Image processing device, image processing method, and program

Abstract

To reduce a risk which causes a false diagnosis.SOLUTION: An image processing device according to an embodiment comprises: a first setting unit which sets a first region of interest, to medical image data; a second setting unit which sets a second region of interest different from the first region of interest set by the first setting unit; and a signal removal processing unit which on the basis of the first region of interest and the second region of interest, executes signal removal processing, for the medical image data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, magnetic resonance imaging apparatuses perform MRA (Magnetic Resonance Angiography) imaging using a time-of-flight (TOF) method to search for unruptured aneurysms in blood vessels as observation targets and to obtain information on cerebral blood flow. When using magnetic resonance image data obtained by MRA imaging using the TOF method for diagnosis, a user applies, for example, maximum intensity projection (MIP) processing to the magnetic resonance image data, and then removes signals corresponding to subcutaneous fat and the like that are not observation targets from the magnetic resonance image data as unnecessary signals.

[0003] In recent years, in order to easily perform the work of removing an unnecessary signal removal region such as subcutaneous fat from magnetic resonance image data, a technology has been proposed in which an unnecessary signal outside the region of interest is automatically removed while the region of interest is automatically set using an algorithm for setting the region of interest or machine learning, or a technology in which an unnecessary signal outside the region of interest is automatically removed while the user manually performs the work of setting the unnecessary signal removal region or region of interest. Some users may use these technologies to automatically perform part or all of the work of removing unnecessary signals from magnetic resonance image data. On the other hand, even though the automation of the work of removing unnecessary signals is progressing, users may be afraid of the risk of overlooking something, the unnecessary signal removal region may differ for each facility or user, or the unnecessary signal removal region or region of interest may be set in detail. For this reason, some users may still perform all of the work of removing unnecessary signals manually. In this way, in the work of removing unnecessary signals, users set the unnecessary signal removal region or region of interest automatically or manually.

[0004] However, when the region of interest is set automatically, since algorithms or machine learning applicable to all clinical scenarios have not been established, there may be regions that should not be removed, such as blood vessels and organs, outside the region set as the unnecessary signal removal region, i.e., outside the region of interest. Even when the region of interest is set manually, there may be regions that should not be removed, such as blood vessels and organs that cannot be seen from the displayed cross section, or blood vessels whose signals have disappeared due to bleeding, outside the region of interest. In this state, if the unnecessary signals outside the region of interest are removed, the regions that should not be removed, such as blood vessels and organs, will also be removed as unnecessary signals, which may lead to a risk of misdiagnosis. In addition, such problems occur not only in magnetic resonance imaging devices, but also in medical image data captured using medical image diagnostic devices such as X-ray CT (Computed Tomography) devices and general X-ray diagnostic devices. If a region that should not be removed, such as blood vessels, exists outside the region of interest, the risk of misdiagnosis can be reduced if the region that should not be removed can be set to include the region of interest. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 7-299053 [Patent Document 2] JP 2020-163100 A [Patent Document 3] JP 2018-68400 A Summary of the Invention [Problem to be solved by the invention]

[0006] One of the problems that the embodiments disclosed in this specification and the drawings aim to solve is to reduce the risk of misdiagnosis. However, the problems that the embodiments disclosed in this specification and the drawings aim to solve are not limited to the above problems. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0007] An image processing device according to an embodiment includes a first setting unit that sets a first region of interest in medical image data, a second setting unit that sets a second region of interest different from the first region of interest set by the first setting unit, and a signal removal processing unit that performs signal removal processing on the medical image data based on the first region of interest and the second region of interest. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the arrangement of an image processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of the arrangement of an image processing apparatus according to a first embodiment. [Diagram 3] FIG. 4 is a flowchart illustrating the contents of a signal removal process executed by the image processing device according to the first embodiment. [Figure 4] FIG. 4 is a flowchart illustrating the contents of a signal removal process executed by the image processing device according to the first embodiment. [Diagram 5] FIG. 2 is a view showing an example of medical image data acquired by the image processing apparatus according to the first embodiment. [Figure 6] FIG. 2 is a view showing an example of medical image data displayed on a display according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of a first region of interest set in medical image data in the first embodiment. [Figure 8] FIG. 4 is a diagram showing an example of a second region of interest set in medical image data in the first embodiment. [Figure 9] FIG. 4 is a diagram showing an example of a second region of interest after editing in the first embodiment. [Figure 10] FIG. 4 is a view showing an example of a first warning screen displayed on a display according to the first embodiment. [Figure 11] FIG. 11 is a view showing an example of a second warning screen displayed on the display according to the first embodiment. [Figure 12] FIG. 4 is a diagram showing an example of a confirmation screen displayed on a display according to the first embodiment. [Figure 13] FIG. 4 is a view showing an example of a processed image displayed on a display according to the first embodiment. [Figure 14] FIG. 11 is a flowchart illustrating the contents of a signal removal process executed by an image processing device according to a second embodiment. [Figure 15] FIG. 11 is a diagram showing an example of medical image data, a first region of interest, and a second region of interest displayed on a display according to a second embodiment. [Figure 16] FIG. 11 is a block diagram showing an example of the arrangement of an image processing system according to a third embodiment. [Figure 17] FIG. 11 is a block diagram showing an example of the arrangement of a magnetic resonance imaging apparatus according to a third embodiment. [Figure 18] FIG. 13 is a block diagram showing an example of the arrangement of an image processing device according to a fourth embodiment. [Figure 19] FIG. 13 is a diagram showing an example of a data flow during learning of an editing function of the image processing device according to the fourth embodiment. [Figure 20] FIG. 13 is a diagram showing an example of a data flow when an editing function of the image processing apparatus according to the fourth embodiment is in operation. [Figure 21] FIG. 13 is a flowchart illustrating the contents of a signal removal process executed by an image processing device according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of an image processing device, an image processing method, and a program will be described with reference to the drawings. In the following description, components having substantially the same functions and configurations are denoted by the same reference numerals, and duplicated descriptions will be given only when necessary.

[0010] [First embodiment] Fig. 1 is a block diagram showing an example of the configuration of an image processing system according to the first embodiment. As shown in Fig. 1, the image processing system 1 according to this embodiment is configured to include a medical image diagnostic device 10, a medical image storage device 20, and an image processing device 30. The medical image diagnostic device 10, the medical image storage device 20, and the image processing device 30 are communicatively connected to each other via an intra-hospital network using a dedicated line within the hospital. The medical image diagnostic device 10, the medical image storage device 20, and the image processing device 30 may be communicatively connected to each other via a network using a public line such as the Internet.

[0011] The medical image diagnostic apparatus 10 captures an image of a subject and generates medical image data. The medical image diagnostic apparatus 10 then transmits the generated medical image data to a medical image storage apparatus 20 and an image processing apparatus 30 via an intra-hospital network. For example, the medical image diagnostic apparatus 10 is an X-ray CT (Computed Tomography) apparatus or a magnetic resonance imaging apparatus. The magnetic resonance imaging apparatus is also called an MRI (Magnetic Resonance Imaging) apparatus. In the following description, the medical image diagnostic apparatus 10 is an MRI apparatus, and the medical image data is magnetic resonance image (hereinafter referred to as MR image) data of the subject's head acquired by the MRI apparatus.

[0012] The medical image storage device 20 stores medical image data generated by the medical image diagnostic device 10 and various images generated by the image processing device 30. The medical image storage device 20 also transmits the stored medical image data and various images to the medical image diagnostic device 10 and to the image processing device 30 via an intra-hospital network. For example, the medical image storage device 20 is an image server such as a PACS (Picture Archiving and Communication System). The medical image storage device 20 may also be realized by a group of servers (cloud) connected to the image processing system 1 via a network.

[0013] The image processing device 30 performs various image processing on medical image data. Specifically, the image processing device 30 sets a first region of interest on the medical image data, sets a second region of interest different from the first region of interest, and executes signal removal processing on the medical image data based on the first region of interest and the second region of interest. For example, the image processing device 30 according to this embodiment is realized by a computer device such as a server or a workstation.

[0014] Here, the first region of interest is a region including an observation target that is included in the medical image data and is observed by a user. The observation target is various parts of a subject, for example, organs such as the heart and liver, bones, blood vessels such as arteries and veins, etc. In the following description, a case where the observation target is blood vessels in the head will be described as an example.

[0015] Furthermore, the second region of interest is a region including an observation target that exists outside the first region of interest. That is, the second region of interest is a region including an observation target that is not included in the first region of interest set in the medical image data, but is included in the entire medical image data. Specifically, when the user is observing blood vessels in the head, the observation target included in this second region of interest is, for example, abnormal blood vessels, blood vessels whose signals have disappeared due to bleeding, blood vessels that cannot be seen from the cross-sectional image displayed on the display 32, fine blood vessels close to fat around the eyes, etc., blood vessels that the user did not include when setting the first region of interest, cerebral blood vessels, etc., and blood vessels included in a region with a difference between the left and right.

[0016] Fig. 2 is a block diagram showing an example of the configuration of the image processing device 30 according to this embodiment. As shown in Fig. 2, the image processing device 30 according to this embodiment includes a memory circuit 31, a display 32, an input interface 33, a communication interface 34, and a processing circuit 35.

[0017] The storage circuitry 31 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, an optical disk, etc. In this embodiment, the storage circuitry 31 stores, for example, various information such as medical image data generated by the medical image diagnostic device 10, medical image data transmitted from the medical image storage device 20, and various image data generated by the image processing device 30.

[0018] The display 32 displays medical image data, various image data, and various information. For example, the display 32 displays a GUI (Graphical User Interface) for receiving various operations from a user. In this embodiment, the display 32 is configured by, for example, a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 32 corresponds to a display unit according to this embodiment.

[0019] The input interface 33 receives various input operations from the user, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuit 35. The input interface 33 is realized by, for example, a mouse, a keyboard, a trackball, a manual switch, a foot switch, a button, a joystick, a touchpad that performs input operations by touching the operation surface, a touch screen in which a display screen and a touchpad are integrated, a non-contact input interface using an optical sensor, and a voice input interface. In this specification, the input interface 33 is not limited to an interface having physical operation parts such as a mouse and a keyboard. For example, an example of the input interface 33 includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to a control circuit. The input interface 33 corresponds to the input unit according to this embodiment.

[0020] The communication interface 34 implements various information communication protocols according to the type of the intra-hospital network. The communication interface 34 realizes communication with other devices via the intra-hospital network in accordance with these various protocols. In this embodiment, the image processing device 30 is connected to the intra-hospital network via the communication interface 34, and realizes communication with the medical image diagnostic device 10 and the medical image storage device 20.

[0021] The processing circuit 35 is an arithmetic circuit that performs various calculations, and is configured with a processor such as a CPU, a GPU, etc. The processing circuit 35 according to the present embodiment, for example, acquires medical image data from the medical image diagnostic device 10 or the medical image storage device 20, and sets a first region of interest and a second region of interest for the medical image data.

[0022] For this reason, the processing circuit 35 according to this embodiment has an acquisition function 351, a display control function 352, a first setting function 353, a second setting function 354, an editing function 355, a first warning function 356, a second warning function 357, and a signal removal processing function 358. The acquisition function 351 corresponds to the acquisition unit according to this embodiment, the display control function 352 corresponds to the display control unit according to this embodiment, the first setting function 353 corresponds to the first setting unit according to this embodiment, the second setting function 354 corresponds to the second setting unit according to this embodiment, the editing function 355 corresponds to the editing unit according to this embodiment, the first warning function 356 corresponds to the first warning unit according to this embodiment, the second warning function 357 corresponds to the second warning unit according to this embodiment, and the signal removal processing function 358 corresponds to the signal removal processing unit according to this embodiment.

[0023] In the embodiment shown in Fig. 2, the processing functions performed by the acquisition function 351, the display control function 352, the first setting function 353, the second setting function 354, the editing function 355, the first warning function 356, the second warning function 357, and the signal removal processing function 358 are stored in the storage circuitry 31 in the form of a program executable by a computer. The processing circuitry 35 is a processor that realizes a function corresponding to each program by reading the program from the storage circuitry 31 and executing it. In other words, the processing circuitry 35 in a state in which each program has been read has each function shown in the processing circuitry 35 of Fig. 2. In FIG. 2, the acquisition function 351, the display control function 352, the first setting function 353, the second setting function 354, the editing function 355, the first warning function 356, the second warning function 357, and the signal removal processing function 358 are described as being realized by a single processing circuit 35. However, it is also possible to configure the processing circuit 35 by combining a plurality of independent processors, and realize these functions by each processor executing a program.

[0024] The acquisition function 351 acquires medical image data. Specifically, the acquisition function 351 acquires medical image data from the medical image storage device 20 that stores the medical image data. The acquisition function 351 may acquire medical image data from the medical image diagnostic device 10 that generates the medical image data. The acquisition function 351 may acquire the medical image data directly via the communication interface 34, or the medical image data acquired via the communication interface 34 may be temporarily stored in the storage circuitry 31, and then the acquisition function 351 may acquire the medical image data from the storage circuitry 31.

[0025] The display control function 352 displays the medical image data on the display 32. The display control function 352 also displays the first region of interest on the medical image data by superimposing the first region of interest on the medical image data by the display 32, and displays the second region of interest on the medical image data by superimposing the second region of interest on the medical image data by the display 32.

[0026] The first setting function 353 sets a first region of interest for the medical image data. The second setting function 354 sets a second region of interest different from the first region of interest set by the first setting function 353. The editing function 355 edits the first region of interest and / or the second region of interest. Editing the first region of interest and / or the second region of interest means adding the first region of interest and / or the second region of interest to the medical image data, deleting the first region of interest and / or the second region of interest set in the medical image data, or changing the range of the first region of interest and / or the second region of interest set in the medical image data.

[0027] The first warning function 356 warns the user that the observation target exists in the area outside the first region of interest and the area outside the second region of interest when the observation target exists in the area outside the first region of interest and the area outside the second region of interest in the medical image data. Here, the area outside the first region of interest and the area outside the second region of interest is an area that does not belong to either the first region of interest or the second region of interest in the medical image data. The second warning function 357 warns the user that the observation target is asymmetric when the observation target included in the first region of interest and / or the second region of interest in the medical image data is asymmetric. The signal removal processing function 358 performs signal removal processing on the medical image data based on the first region of interest and the second region of interest.

[0028] 3 and 4 are flow charts for explaining the contents of the signal removal process executed by the image processing device 30 according to this embodiment. In this signal removal process, medical image data is acquired, and a first region of interest and a second region of interest are set for the medical image data. For example, this signal removal process is a process that is executed when medical image data is acquired.

[0029] 3, first, the image processing device 30 acquires medical image data (step S11). The process of acquiring the medical image data is realized by an acquisition function 351 in the processing circuit 35. Specifically, the image processing device 30 acquires the medical image data from the medical image diagnostic device 10 or the medical image storage device 20 via the communication interface 34.

[0030] Fig. 5 is a diagram showing an example of medical image data acquired by the image processing device 30 according to the first embodiment. In the example shown in Fig. 5, the image processing device 30 acquires three-dimensional MR image data including a plurality of slice image data DS_1 to DS_n (n≧2) as medical image data D1. Here, one slice image data is MR image data extending in the X direction, Y direction, and Z direction shown in Fig. 5, and the slice image data DS_1 to DS_n are arranged in the Z direction to realize a three-dimensional data structure. Furthermore, in the example shown in Fig. 5, MIP processing is performed on the medical image data D1.

[0031] 3, the image processing device 30 displays the medical image data D1 (step S13). The process of displaying the medical image data D1 is realized by the display control function 352 in the processing circuitry 35. Specifically, the image processing device 30 causes the display 32 to display the medical image data D1 acquired in step S11.

[0032] Fig. 6 is a diagram showing an example of medical image data D1 displayed on the display 32 according to the first embodiment. As shown in Fig. 6, the display 32 of the image processing device 30 displays one arbitrary slice image data of the three-dimensional MR image data, which is the medical image data D1.

[0033] 3, the image processing device 30 receives a first input operation (step S15). The process of receiving the first input operation is realized by the first setting function 353 in the processing circuitry 35. Specifically, the image processing device 30 receives a first input operation from a user via the input interface 33, the first input operation being an input operation related to setting a first region of interest in the medical image data D1. Then, the image processing device 30 sets the first region of interest in the medical image data D1 in response to the first input operation.

[0034] More specifically, for example, when the medical image data D1 acquired in step S11 is three-dimensional medical image data D1 as shown in Fig. 5, the first setting function 353 may accept a first input operation for each slice image data, and set a first region of interest for each slice image data in response to the first input operation. Alternatively, the first setting function 353 may set a first region of interest for one slice image data in response to a first input operation for one slice image data, while also setting a first region of interest for other slice image data included in the three-dimensional medical image data D1.

[0035] 7 is a diagram showing an example of a first region of interest set on medical image data D1 in the first embodiment. As shown in FIG. 7, the image processing device 30 receives an input operation from the user via the input interface 33 to superimpose a figure on the MR image data displayed on the display 32 as a first input operation, thereby setting a first region of interest R1 on the MR image data, which is the medical image data D1. The figure superimposed on the medical image data D1 may be a figure drawn freehand by the user, or may be a figure such as a circle or a rectangle. In other words, the shape of the figure superimposed on the medical image data D1 is arbitrary.

[0036] Next, as shown in FIG. 3, the image processing device 30 determines whether the setting of the first region of interest R1 is completed (step S17). The process of determining whether the setting of the first region of interest R1 is completed is realized by the first setting function 353 in the processing circuit 35. Specifically, the image processing device 30 determines whether the setting of the first region of interest R1 is completed by determining whether the user presses a setting completion button (not shown) for the first region of interest R1 displayed on the display 32 via the input interface 33. If the setting of the first region of interest R1 is not completed (step S17: No), the image processing device 30 returns to the above-mentioned step S15 and waits by repeating the process of accepting the first input operation (step S15) and the process of determining whether the setting of the first region of interest R1 is completed (step S17).

[0037] On the other hand, if it is determined in step S17 that the setting of the first region of interest R1 is completed (step S17: Yes), the image processing device 30 sets a second region of interest (step S19). The process of setting this second region of interest is realized by the second setting function 354 in the processing circuitry 35. Specifically, the image processing device 30 according to this embodiment sets the second region of interest based on anatomical feature information on anatomical feature points included in the medical image data D1.

[0038] More specifically, for example, the second setting function 354 extracts anatomical features by executing an anatomical feature extraction process on the medical image data D1, and compares anatomical feature information on the anatomical features of the observation target included in the medical image data D1 with model data of the anatomical features of the observation target stored in the storage circuitry 31. Then, the second setting function 354 sets the second region of interest by extracting the observation target existing outside the first region of interest R1 in the medical image data D1. Note that in the above example, the comparison is performed with the model data of the anatomical feature of the observation target stored in the storage circuitry 31, but when the anatomical feature of the observation target exists outside the first region of interest R1 in the medical image data D1, the second setting function 354 may set the second region of interest by extracting the observation target outside the first region of interest R1 in the medical image data D1 based on the anatomical feature existing outside the first region of interest R1 as the anatomical feature information.

[0039] 3, the image processing device 30 displays the second region of interest (step S21). The process of displaying the second region of interest is realized by the display control function 352 in the processing circuitry 35. Specifically, the image processing device 30 causes the display 32 to display the second region of interest by superimposing it on the medical image data D1.

[0040] FIG. 8 is a diagram showing an example of a second region of interest set in medical image data D1 in the first embodiment. As shown in FIG. 8, the image processing device 30 displays the second region of interest R2 on the display 32 by superimposing it on the MR image data in which the first region of interest R1 is set. In this embodiment, two second regions of interest R21 and R22 are set in the MR image data. Note that the number of second regions of interest R2 set in the MR image data is not limited to two. That is, the number of second regions of interest R2 set in the MR image data is arbitrary, and one second region of interest R2 may be set, or three or more second regions of interest R2 may be set.

[0041] 3, the image processing device 30 determines whether the first region of interest R1 and / or the second region of interest R2 are to be edited (step S23). The process of determining whether the first region of interest R1 and / or the second region of interest R2 are to be edited is realized by the editing function 355 in the processing circuit 35. Specifically, the image processing device 30 determines whether the first region of interest R1 and / or the second region of interest R2 are to be edited by determining whether an editing start button (not shown) displayed on the display 32 has been pressed via the input interface 33.

[0042] Then, in step S23, when the first region of interest R1 and / or the second region of interest R2 is edited (step S23: Yes), the image processing device 30 accepts a second input operation or a third input operation (step S25). The process of accepting the second input operation or the third input operation is realized by the editing function 355 in the processing circuitry 35. Specifically, the image processing device 30 accepts a second input operation, which is an input operation for editing the first region of interest R1 set in the medical image data D1, from the user via the input interface 33, and edits the first region of interest R1 in response to the second input operation. Alternatively, the image processing device 30 accepts a third input operation, which is an input operation for editing the second region of interest R2 set in the medical image data D1, from the user via the input interface 33, and edits the second region of interest R2 in response to the third input operation.

[0043] More specifically, the editing function 355 receives a second input operation from the user via the input interface 33, the second input operation being an input operation for editing the first region of interest R1 set in the medical image data D1, and edits the first region of interest R1 in response to the second input operation. Alternatively, the editing function 355 receives a second input operation from the user via the input interface 33, the second input operation being an input operation for editing the second region of interest R2 set in the medical image data D1, and edits the second region of interest R2 in response to a third input operation.

[0044] For example, when the medical image data D1 acquired in step S11 is three-dimensional medical image data D1 as shown in FIG. 5, the editing function 355 accepts a second input operation and / or a third input operation for each slice image data, and edits the first region of interest R1 and / or the second region of interest R2 in accordance with the second input operation and / or the third input operation.

[0045] The second input operation and the third input operation are, for example, a drag-and-drop operation using a mouse as an example of the input interface 33. Specifically, the mouse cursor is moved onto the first region of interest R1 or the second region of interest R2 displayed on the display 32, and the mouse button is pressed. Then, while the mouse button is still pressed, the mouse cursor is moved to an arbitrary position, and the mouse button is released at an arbitrary position on the XY plane on the medical image data D1. As a result, for example, editing is performed to expand the first region of interest R1 or the second region of interest R2 to the position where the mouse button is released. In this way, the image processing device 30 edits the first region of interest R1 and / or the second region of interest R2.

[0046] Furthermore, the second input operation and the third input operation may be, for example, a click operation using a mouse as an example of the input interface 33. Specifically, a click operation may be performed on the medical image data D1 displayed on the display 32, and the range of the first region of interest R1 and / or the range of the second region of interest R2 may be changed based on the pixel value of the clicked pixel. As a result, for example, editing is performed to include a region having a pixel value similar to that of the clicked pixel in the first region of interest R1 or the second region of interest R2. In this manner, the image processing device 30 may edit the first region of interest R1 and / or the second region of interest R2.

[0047] The second input operation and the third input operation are not limited to a drag-and-drop operation or a click operation using a mouse. That is, the contents of the second input operation and the third input operation are arbitrary, and may be an operation using an input interface 33 other than a mouse, such as a click operation of a delete button on a keyboard, or a scroll operation using a mouse wheel.

[0048] FIG. 9 is a diagram showing an example of the second region of interest R2 after editing in the first embodiment. In the example shown in FIG. 9, the image processing device 30 does not accept the second input operation, which is an input operation for editing the first region of interest R1, from the user, and therefore the first region of interest R1 is not edited. Meanwhile, the image processing device 30 accepts an input operation for deleting the second region of interest R21 from the user as a third input operation, which is an input operation for editing the second region of interest R2, and deletes the second region of interest R21. In addition, the image processing device 30 accepts an input operation for changing the range of the second region of interest R22 from the user as the third input operation, and changes the second region of interest R22 shown in FIG. 8 to the second region of interest R22a shown in FIG. 9.

[0049] Next, as shown in FIG. 3, the image processing device 30 determines whether the editing of the first region of interest R1 and / or the second region of interest R2 is completed (step S27). The process of determining whether the editing is completed is realized by the editing function 355 in the processing circuit 35. Specifically, the image processing device 30 determines whether the editing of the first region of interest R1 and / or the second region of interest R2 is completed by determining whether the user presses an editing completion button (not shown) displayed on the display 32 via the input interface 33. Then, if the editing of the first region of interest R1 and / or the second region of interest R2 is not completed (step S27: No), the image processing device 30 returns to the above-mentioned step S25 and waits by repeating the process of receiving the second input operation or the third input operation (step S25) and the process of determining whether the editing is completed (step S27).

[0050] On the other hand, if the first region of interest R1 and / or the second region of interest R2 are not edited in step S23 (step S23: No), or if the editing of the first region of interest R1 and / or the second region of interest R2 is completed in step S27 (step S27: Yes), the image processing device 30 determines whether or not an instruction to execute a signal removal process has been input (step S29). Specifically, the image processing device 30 determines whether or not an instruction to execute a signal removal process has been input by determining whether or not a user has pressed a signal removal process execution button (not shown) displayed on the display 32 via the input interface 33. If an instruction to execute a signal removal process has not been input (step S29: No), the image processing device 30 returns to the above-mentioned step S23 and waits while repeating the processes from step S23.

[0051] On the other hand, if an instruction to execute the signal removal process is input in step S29 (step S29: Yes), as shown in Fig. 4, the image processing device 30 determines whether or not an observation target exists in a region outside the first region of interest R1 and the second region of interest R2 in the medical image data D1 (step S31). This process of determining whether or not an observation target exists is realized by the first warning function 356 in the processing circuitry 35. Specifically, the image processing device 30 determines whether or not an observation target exists in a region outside the first region of interest R1 and the second region of interest R2 in the medical image data D1 based on the anatomical landmark information included in the medical image data D1. More specifically, for example, when the medical image data D1 acquired in step S11 is three-dimensional medical image data D1 as shown in FIG. 5, the first warning function 356 determines whether or not an object of observation exists in an area outside the first area of ​​interest R1 and the second area of ​​interest R2 based on the first area of ​​interest R1 and the second area of ​​interest R2 set in each of the slice image data included in the medical image data D1.

[0052] In the above-mentioned step S31, it is determined whether or not an observation target exists in an area outside the first region of interest R1 and the second region of interest R2 in the medical image data D1 based on the anatomical feature information included in the medical image data D1. However, the image processing device 30 may newly perform image analysis processing such as threshold processing or edge detection, and determine whether or not an observation target exists in an area outside the first region of interest R1 and the second region of interest R2 in the medical image data D1 based on the result of the image analysis processing.

[0053] Then, in step S31, if an observation target exists in an area outside the first region of interest R1 and outside the second region of interest R2 in the medical image data D1 (step S31: Yes), the image processing device 30 issues a first warning to warn the user of that fact (step S33). The process of issuing the first warning is realized by the first warning function 356 in the processing circuitry 35. Specifically, the image processing device 30 warns the user, via the display 32 or the like, that an observation target exists in an area outside the first region of interest R1 and outside the second region of interest R2.

[0054] Fig. 10 is a diagram showing an example of a first warning screen displayed on the display 32 according to the first embodiment. As shown in Fig. 10, the image processing device 30 displays a first warning screen A1 including text information "OBSERVATION SUBJECT EXISTS OUTSIDE THE REGION OF INTEREST" on the display 32 to warn the user that an observation subject exists outside the first region of interest R1 and the second region of interest R2.

[0055] 10, the user is warned by displaying the first warning screen A1 on the display 32, but the method of warning the user that an observation target exists in an area outside the first region of interest R1 and the second region of interest R2 is not limited to this. In other words, the method of warning the user that an observation target exists in an area outside the first region of interest R1 and the second region of interest R2 is arbitrary, and the user may be warned via an output interface such as a speaker or a printer.

[0056] 10, the first warning screen A1 is displayed with text information, but the display mode of the first warning screen A1 is not limited to this. That is, the display mode of the first warning screen A1 is arbitrary, and the first warning screen A1 may display, for example, slice image data in which an observation target existing in an area outside the first region of interest R1 and the second region of interest R2 is emphasized together with text information, or may display slice image data in which an observation target existing in an area outside the first region of interest R1 and the second region of interest R2 is emphasized instead of text information.

[0057] On the other hand, in step S31, if there is no observation target in the medical image data D1 outside the first region of interest R1 and outside the second region of interest R2 (step S31: No), or after step S33, the image processing device 30 determines whether or not there is a difference between the left and right sides of the observation target included in the first region of interest R1 and / or the second region of interest R2 (step S35). The process of determining whether or not there is a difference between the left and right sides of the observation target is realized by the second warning function 357 in the processing circuit 35. Specifically, the image processing device 30 determines whether or not there is a difference between the left and right sides of the observation target included in the first region of interest R1 and / or the second region of interest R2 based on the anatomical landmark information included in the first region of interest R1 and / or the second region of interest R2. The difference between the left and right sides is, for example, when the observation target included in the first region of interest R1 and / or the second region of interest R2 is asymmetric. More specifically, for example, when the medical image data D1 acquired in step S11 is three-dimensional medical image data D1 as shown in FIG. 5, the second warning function 357 determines whether or not there is a difference between the left and right sides of the observation object contained in the first region of interest R1 and / or the second region of interest R2 based on the first region of interest R1 and the second region of interest R2 set in each of the slice image data contained in the medical image data D1.

[0058] In the above-mentioned step S35, whether or not there is a difference between the left and right sides of the observation target included in the first region of interest R1 and / or the second region of interest R2 is determined based on the anatomical landmark information included in the first region of interest R1 and / or the second region of interest R2. However, the image processing device 30 may newly perform an image analysis process and determine whether or not there is a difference between the left and right sides of the observation target included in the first region of interest R1 and / or the second region of interest R2 based on the result of the image analysis process.

[0059] Then, in step S35, if there is a difference between the left and right of the observation target included in the first region of interest R1 and / or the second region of interest R2 (step S35: Yes), the image processing device 30 issues a second warning to warn the user of that fact (step S37). The process of issuing the second warning is realized by the second warning function 357 in the processing circuit 35. Specifically, the image processing device 30 warns the user via the display 32 or the like that there is a difference between the left and right of the observation target included in the first region of interest R1 and / or the second region of interest R2.

[0060] Fig. 11 is a diagram showing an example of a second warning screen displayed on the display 32 according to the first embodiment. As shown in Fig. 11, the image processing device 30 displays a second warning screen A2 including text information such as "The observation objects included in the regions of interest are different between the left and right. Are you sure?" on the display 32, thereby warning the user that there is a difference between the left and right of the observation objects included in the first region of interest R1 and / or the second region of interest R2.

[0061] 11, the second warning screen A2 is displayed on the display 32 to warn the user, but the method of warning the user that there is a difference between the left and right of the observation target included in the first region of interest R1 and / or the second region of interest R2 is not limited to this. In other words, the method of warning the user that there is a difference between the left and right of the observation target included in the first region of interest R1 and / or the second region of interest R2 may be any method, and the user may be warned via an output interface such as a speaker.

[0062] 11, the second warning screen A2 is displayed with text information, but the display mode of the second warning screen A2 is not limited to this. That is, the display mode of the second warning screen A2 is arbitrary, and the second warning screen A2 may display, for example, slice image data in which the observation target included in the first region of interest R1 and / or the second region of interest R2 has a left-right difference together with text information, or may display slice image data in which the observation target included in the first region of interest R1 and / or the second region of interest R2 has a left-right difference instead of text information.

[0063] On the other hand, in step S35, if there is no laterality in the observation target included in the first region of interest R1 and / or the second region of interest R2 (step S35: No), or after step S37, the image processing device 30 displays a confirmation screen (step S39). The process of displaying this confirmation screen is realized by the signal removal processing function 358 in the processing circuit 35. Specifically, the image processing device 30 displays on the display 32 a confirmation screen that prompts the user to confirm whether or not to execute the signal removal processing.

[0064] Fig. 12 is a diagram showing an example of a confirmation screen displayed on the display according to the first embodiment. In the example shown in Fig. 11, the image processing device 30 displays on the display 32 a confirmation screen C1 including text information of "Remove?", a Yes button B1, and a No button B2.

[0065] Next, as shown in FIG. 4, the image processing device 30 judges whether or not to execute the signal removal process (step S41). The process of judging whether or not to execute the signal removal process is realized by the signal removal process function 358 in the processing circuit 35. Specifically, the image processing device 30 judges whether or not to execute the signal removal process by accepting an input operation related to the execution of the signal removal process on the confirmation screen via the input interface 33. More specifically, the signal removal process function 358 judges whether or not to execute the signal removal process by pressing either the Yes button B1 or the No button B2 included in the confirmation screen C1 as an input operation related to the execution of the signal removal process. Then, in step S41, if it is judged that the signal removal process is not to be executed (step S41: No), the process returns to step S23 and waits while repeating the process from step S23.

[0066] On the other hand, if it is determined in step S41 that the signal removal process is to be executed (step S41: Yes), the image processing device 30 executes the signal removal process (step S43). The process of executing the signal removal process is realized by the signal removal process function 358 in the processing circuit 35. Specifically, the image processing device 30 executes the signal removal by cutting out the first region of interest R1 and the second region of interest R2 set for the medical image data D1 from the medical image data D1. Note that the method of the signal removal process is not limited to this. That is, the method of the signal removal process is arbitrary, and for example, the pixel values ​​of the medical image data D1 outside the regions of interest may be set to 0.

[0067] 4, the image processing device 30 displays the processed image (step S45). This process of displaying the processed image is realized by the display control function 352 in the processing circuit 35. Specifically, the image processing device 30 displays, on the display 32, the processed image, which is the image after the signal removal process has been performed.

[0068] Fig. 13 is a diagram showing an example of a processed image displayed on a display according to the first embodiment. As shown in Fig. 13, the image processing device displays, on the display 32, a processed image P1 which is an image cut out from a first region of interest R1 and a second region of interest R2.

[0069] By executing step S45, the signal removal process according to this embodiment ends.

[0070] As described above, according to the image processing system 1 of the present embodiment, the image processing device 30 sets the first region of interest R1 for the medical image data D1, sets the second region of interest R2 different from the first region of interest R1, and executes the signal removal process based on the first region of interest R1 and the second region of interest R2, thereby reducing the risk of misdiagnosis due to removal of an unnecessary signal in a region that should not be removed. Also, according to the image processing system 1 of the present embodiment, the image processing device 30 sets the second region of interest R2 by extracting the observation target based on the anatomical feature point information of the observation target, so that the observation target can be extracted even when the signal has disappeared due to bleeding.

[0071] Second Embodiment In the first embodiment described above, the image processing device 30 displays the medical image data D1 acquired from the medical image diagnostic device 10, the medical image storage device 20, etc. on the display 32, accepts the first input operation, which is an input operation for setting the first region of interest R1 for the medical image data D1, and sets the first region of interest R1 in response to the first input operation, but the method for setting the first region of interest R1 is not limited to this. In the second embodiment, an image processing device 30 that sets the first region of interest by performing an analysis process on the medical image data D1 acquired from the medical image diagnostic device 10, the medical image storage device 20, etc. will be described below. Note that the configuration of the image processing system 1 and the configuration of the image processing device 30 according to the second embodiment are the same as those in FIG. 1 and FIG. 2, so their description will be omitted.

[0072] Fig. 14 is a flow chart for explaining the contents of the signal removal process executed by the image processing device 30 according to the second embodiment, and corresponds to Fig. 3 in the first embodiment. The signal removal process according to this embodiment differs from the signal removal process according to the first embodiment in that the first region of interest R1 is automatically set. Note that the process of step S11 shown in Fig. 14 is the same as that of the first embodiment, and therefore will not be described.

[0073] After step S11, as shown in FIG. 14, the image processing device 30 sets a first region of interest R1 (step S51). The process of setting the first region of interest R1 is realized by the first setting function 353 in the processing circuit 35. Specifically, the image processing device 30 sets the first region of interest R1 by executing an analysis process on the medical image data D1 acquired in step S11. More specifically, the first setting function 353 extracts the observation target by executing an analysis process on the MR image data, which is the medical image data D1 acquired in step S11, to extract the observation target using an extraction algorithm for extracting the observation target, and sets the first region of interest R1 on the MR image data. Here, the extraction algorithm is, for example, a blood vessel extraction algorithm for extracting blood vessels such as cerebral blood vessels using signal continuity or bilateral symmetry, or a region extraction algorithm. The first setting function 353 may execute an analysis process on the medical image data D1 using a machine learning technique such as deep learning.

[0074] Furthermore, in step S51, when a region of interest is set in a positioning image (also called a locator image) corresponding to medical image data D1, the image processing device 30 may set the first region of interest R1 based on the region of interest set in the positioning image corresponding to medical image data. In this way, when setting the first region of interest R1 based on the region of interest set in the positioning image corresponding to medical image data D1, the image processing device 30 may acquire the medical image data D1 in step S11 and also acquire the positioning image corresponding to medical image data D1.

[0075] Moreover, the process of step S19 after step S51 is similar to that of the first embodiment described above, and therefore the description thereof will be omitted.

[0076] 14, the image processing device 30 displays the medical image data D1, the first region of interest R1, and the second region of interest R2 (step S53). This display process is realized by the display control function 352 in the processing circuitry 35. Specifically, the image processing device 30 causes the display 32 to display the medical image data D1 acquired in step S11, with the first region of interest R1 set in step S51 and the second region of interest R2 set in step S21 superimposed on the medical image data D1.

[0077] Fig. 15 is a diagram showing an example of medical image data D1, a first region of interest R1, and a second region of interest R2 displayed on a display 32 according to the second embodiment, and corresponds to Fig. 8 in the first embodiment described above. As shown in Fig. 15, the image processing device 30 displays the first region of interest R1 and the second region of interest R2 on the MR image data, which is medical image data D1, on the display 32. Note that other explanations are the same as those in Fig. 8 in the first embodiment described above, and therefore will not be repeated.

[0078] The processes from step S23 to step S29 after step S53 are the same as those in the first embodiment, and therefore the description thereof will be omitted. The processes after step S29 shown in Fig. 14 are the same as those in Fig. 4 in the first embodiment, and therefore the description thereof will be omitted.

[0079] As described above, according to the image processing system 1 of this embodiment, as in the first embodiment, the image processing device 30 sets the first region of interest R1 for the medical image data D1, sets the second region of interest R2 different from the first region of interest R1, and executes the signal removal process based on the first region of interest R1 and the second region of interest R2, thereby reducing the risk of misdiagnosis due to removal of an area that should not be removed as an unnecessary signal. Also, according to the image processing system 1 of this embodiment, as in the first embodiment, the image processing device 30 sets the second region of interest R2 by extracting the observation target based on the anatomical feature point information of the observation target, so that the observation target can be extracted even when the signal has disappeared due to bleeding.

[0080] Furthermore, according to the image processing system 1 of this embodiment, the image processing device 30 sets the first region of interest R1 by executing an analysis process on the medical image data D1, so that the user himself / herself does not need to set the first region of interest R1 on the medical image data D1, and the burden on the user in setting the first region of interest R1 can be reduced compared to the first embodiment.

[0081] Third Embodiment In the above-mentioned first and second embodiments, the medical image diagnostic apparatus 10 and the image processing device 30 have been described as separate entities, but the image processing device 30 may be included in the medical image diagnostic apparatus 10. In the third embodiment, a case where the image processing device 30 is included in, for example, a magnetic resonance imaging apparatus as the medical image diagnostic apparatus 10 will be described. Hereinafter, a case where this modification is applied to the above-mentioned first embodiment will be described as the third embodiment, and the parts different from the above-mentioned first embodiment will be described. Note that this modification can also be applied to the second embodiment in the same manner.

[0082] FIG. 16 is a block diagram showing an example of the configuration of an image processing system 1 according to the third embodiment, and corresponds to FIG. 1 in the first embodiment described above. As shown in FIG. 16, the image processing system 1 according to this embodiment differs from the first embodiment in the configuration of the medical image diagnostic apparatus, and is therefore referred to as a medical image diagnostic apparatus 10a in this embodiment. In addition, in this embodiment, the image processing system 1 does not include an image processing apparatus 30, since the image processing apparatus 30 is included in the medical image diagnostic apparatus 10a. In the following description, the medical image diagnostic apparatus 10a will be described as a magnetic resonance imaging apparatus, and therefore the medical image diagnostic apparatus 10a will also be referred to as a magnetic resonance imaging apparatus 10a.

[0083] Fig. 17 is a block diagram showing an example of the configuration of a magnetic resonance imaging apparatus 10a according to the third embodiment, and corresponds to Fig. 2 in the first embodiment described above. As shown in Fig. 17, the magnetic resonance imaging apparatus 10a according to this embodiment includes a static magnetic field magnet 101, a gradient magnetic field coil 103, a gradient magnetic field power supply 104, a bed 105, a bed control circuit 106, a transmission coil 107, a transmission circuit 108, a reception coil 109, a reception circuit 110, a sequence control circuit 120, and a computer system 130. In this embodiment, the computer system 130 in the magnetic resonance imaging apparatus 10a executes the function of the image processing device 30.

[0084] It should be noted that the magnetic resonance imaging apparatus 10 does not include a subject P. The configuration of the magnetic resonance imaging apparatus 10 is not limited to that shown in Fig. 17. That is, the magnetic resonance imaging apparatus 10 may have any configuration. For example, the sequence control circuit 120 and each unit in the computer system 130 may be configured to be integrated or separated as appropriate.

[0085] The static magnetic field magnet 101 is a magnet formed in a hollow, approximately cylindrical shape, and generates a static magnetic field in the internal space. The static magnetic field magnet 101 is, for example, a superconducting magnet, and is excited by receiving a current from a static magnetic field power supply. The static magnetic field power supply supplies power to the static magnetic field magnet 101. As another example, the static magnetic field magnet 101 may be a permanent magnet, in which case the magnetic resonance imaging apparatus 10 does not need to include a static magnetic field power supply. Also, the static magnetic field power supply may be provided separately from the magnetic resonance imaging apparatus 10.

[0086] The gradient magnetic field coil 103 is a coil formed in a hollow cylindrical shape and is disposed inside the static magnetic field magnet 101. The gradient magnetic field coil 103 is formed by combining three coils corresponding to the X-, Y-, and Z-axes that are orthogonal to each other. The Z-axis direction is the same direction as the static magnetic field. The Y-axis direction is the vertical direction, and the X-axis direction is the direction perpendicular to the Z-axis and the Y-axis. The gradient magnetic field coil 103 generates a gradient magnetic field to be superimposed on the static magnetic field. Specifically, the three coils in the gradient magnetic field coil 103 are individually supplied with power from a gradient magnetic field power supply 104, and generate a gradient magnetic field whose magnetic field strength changes along each of the X-, Y-, and Z-axes. The gradient magnetic field power supply 104 supplies a current to the gradient magnetic field coil 103 under the control of the sequence control circuit 120.

[0087] The bed 105 includes a top board 105a on which the subject P is placed. Under the control of a bed control circuit 106, the bed 105 inserts the top board 105a into the imaging aperture with the subject P placed on the top board 105a. Under the control of a computer system 130, the bed control circuit 106 drives the bed 105 to move the top board 105a in the longitudinal direction and the vertical direction.

[0088] The transmission coil 107 receives an RF pulse from the transmission circuitry 108, generates a high-frequency magnetic field, and applies the high-frequency magnetic field to the subject P. The transmission circuitry 108 supplies an RF pulse to the transmission coil 107 under the control of a sequence control circuit 120.

[0089] The receiving coil 109 is disposed inside the gradient magnetic field coil 103, and receives a magnetic resonance signal (MR signal) emitted from the subject P due to the influence of a high frequency magnetic field. The receiving coil 109 outputs the received magnetic resonance signal to a receiving circuit 110. Note that a configuration in which the receiving coil 109 also serves as a transmitting coil may be adopted.

[0090] The receiving circuit 110 detects a magnetic resonance signal output from the receiving coil 109, and generates magnetic resonance data based on the detected magnetic resonance signal. Specifically, the receiving circuit 110 performs analog-to-digital (AD) conversion on the analog magnetic resonance signal output from the receiving coil 109 to generate magnetic resonance data (MR data). The receiving circuit 110 also transmits the generated magnetic resonance data to the sequence control circuit 120. The receiving circuit 110 may be provided on the side of a gantry device including the static magnetic field magnet 101, the gradient magnetic field coil 103, and the like.

[0091] The sequence control circuit 120 performs imaging of the subject P by driving the gradient magnetic field power supply 104, the transmission circuitry 108, and the reception circuitry 110 based on sequence information transmitted from the computer system 130. Here, the sequence information is information that defines a procedure for performing imaging. The sequence information defines the strength of the current that the gradient magnetic field power supply 104 supplies to the gradient magnetic field coil 103 and the timing of supplying the current, the strength of the RF pulse that the transmission circuitry 108 supplies to the transmission coil 107 and the timing of applying the RF pulse, the timing at which the reception circuitry 110 detects a magnetic resonance signal, and the like.

[0092] The sequence control circuit 120 is, for example, an integrated circuit such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA), or an electronic circuit such as a Central Processing Unit (CPU) or a Micro Processing Unit (MPU).

[0093] The computer system 130 performs overall control of the magnetic resonance imaging apparatus 10a, generates magnetic resonance images, etc. As shown in Fig. 17, the computer system 130 is configured to include a memory circuit 131, a display 132, an input interface 133, a communication interface 134, and a processing circuit 135, which are connected to each other via an internal bus. The memory circuit 131, the display 132, the input interface 133, and the communication interface 134 correspond to the memory circuit 31, the display 32, the input interface 33, and the communication interface 34 in the image processing apparatus 30 in the first embodiment described above, respectively, and have substantially the same functions, so that the description will be omitted.

[0094] The processing circuit 135 is a control circuit that performs overall control of the magnetic resonance imaging apparatus 10a, and is also a calculation circuit that performs various calculations. For example, the processing circuit 135 according to this embodiment has a system control function 1351, an image generation function 1352, an acquisition function 1353, a display control function 1354, a first setting function 1355, a second setting function 1356, an editing function 1357, a first warning function 1358, a second warning function 1359, and a signal removal processing function 1360. The system control function 1351 corresponds to the system control unit in this embodiment, the image generation function 1352 corresponds to the image generation unit in this embodiment, the acquisition function 1353 corresponds to the acquisition unit in this embodiment, the display control function 1354 corresponds to the display control unit in this embodiment, the first setting function 1355 corresponds to the first setting unit in this embodiment, the second setting function 1356 corresponds to the second setting unit in this embodiment, the editing function 1357 corresponds to the editing unit in this embodiment, the first warning function 1358 corresponds to the first warning unit in this embodiment, the second warning function 1359 corresponds to the second warning unit in this embodiment, and the signal removal processing function 1360 corresponds to the signal removal processing unit in this embodiment.

[0095] The system control function 1351 controls the magnetic resonance imaging apparatus 10a. Specifically, the system control function 1351 reads out a system control program stored in the storage circuitry 131, develops it on the memory, and controls various circuits and various power supplies in the magnetic resonance imaging apparatus 10a according to the developed system control program. For example, the system control function 1351 reads out an imaging protocol from the storage circuitry 131 based on imaging conditions input by a user via the input interface 133. The system control function 1351 may generate an imaging protocol based on the imaging conditions. The system control function 1351 transmits the imaging protocol to the sequence control circuitry 120 and controls imaging such as locator imaging and main imaging for the subject P.

[0096] The image generating function 1352 generates a locator image based on the magnetic resonance data collected by the locator imaging. Also, the image generating function 1352 generates a reconstructed image by performing reconstruction processing on the k-space data, which is the magnetic resonance data arranged by the system control function 1351. For example, FFT (Fast Fourier Transfer) or the like is used as the reconstruction processing.

[0097] In addition, the acquisition function 1353, the display control function 1354, the first setting function 1355, the second setting function 1356, the editing function 1357, the first warning function 1358, the second warning function 1359, and the signal removal processing function 1360 are equivalent to the acquisition function 351, the display control function 352, the first setting function 353, the second setting function 354, the editing function 355, the first warning function 356, the second warning function 357, and the signal removal processing function 358 of the processing circuit 35 in the first embodiment described above, respectively, and therefore their explanations will be omitted.

[0098] As described above, in the image processing system 1 according to the present embodiment, the magnetic resonance imaging apparatus 10a functions as the image processing apparatus 30, so that the magnetic resonance imaging apparatus 10a sets the first region of interest R1 for the medical image data D1, sets the second region of interest R2 different from the first region of interest R1, and executes the signal removal process based on the first region of interest R1 and the second region of interest R2, thereby reducing the risk of misdiagnosis due to removal of an area that should not be removed as an unnecessary signal. Also, in the image processing system 1 according to the present embodiment, the magnetic resonance imaging apparatus 10a functions as the image processing apparatus 30, so that the magnetic resonance imaging apparatus 10a sets the second region of interest R2 by extracting the observation target based on the anatomical feature point information of the observation target, so that the observation target can be extracted even when the signal has disappeared due to bleeding.

[0099] [Fourth embodiment] In the first to third embodiments described above, the image processing device 30 receives from the user the second input operation, which is an input operation for editing the first region of interest R1, or the third input operation, which is an input operation for editing the second region of interest R2, and manually edits the first region of interest R1 and / or the second region of interest R2 in response to the second input operation or the third input operation, but is not limited to this. The image processing device 30 may automatically edit the first region of interest R1 and / or the second region of interest R2. Hereinafter, a case where this modification is applied to the first embodiment described above will be referred to as the fourth embodiment, and a part different from the first embodiment described above will be described. Note that this modification can be similarly applied to the second and third embodiments. In addition, the configuration of the image processing system 1 according to the fourth embodiment is the same as that in FIG. 1, so a description thereof will be omitted.

[0100] FIG. 18 is a block diagram showing an example of the configuration of an image processing device 30 according to the fourth embodiment, and corresponds to FIG. 2 in the first embodiment. As shown in FIG. 18, the image processing device 30 according to this embodiment is configured by adding a learning function to the processing circuit 35 with respect to the image processing device 1 according to the first embodiment. In addition, since the editing function is different from the editing function 355 in the first embodiment, in this embodiment, it is written as an editing function 355a. Note that the learning function 359 corresponds to the learning unit according to this embodiment. In addition, since the configuration and functions other than the editing function 355a and the learning function 359 are equivalent to those in FIG. 1 in the first embodiment, a description thereof will be omitted.

[0101] The editing function 355a edits the first region of interest R1 and / or the second region of interest R2 set in the medical image data D1. In this embodiment, the editing function 355a edits the first region of interest R1 and / or the second region of interest R2 set in the medical image data D1. Based on pre-edited medical image data, which is medical image data D1 in which the first region of interest R1 before editing and the second region of interest R2 before editing are set, and finalized medical image data, which is medical image data D1 in which the first region of interest R1 after editing and the second region of interest R2 after editing are set, the editing function 355a inputs the pre-edited medical image data to a trained model that has been machine-learned to output edited medical image data, which is medical image data D1 in which the first region of interest R1 after editing and the second region of interest R2 after editing are set, and obtains the edited medical image data output from the trained model, thereby editing the first region of interest R1 and / or the second region of interest R2. That is, the editing function 355a edits the first region of interest R1 and / or the second region of interest R2 by inputting the pre-edited medical image data to a trained model trained using as a training set the pre-edited medical image data in which the pre-edited first region of interest R1 and the pre-edited second region of interest R2 are set, and the finalized medical image data in which the finalized first region of interest R1 and the finalized second region of interest R2 are set. Note that "after finalization" here refers to a state in which all editing has been completed and no further editing will be performed.

[0102] The pre-edited medical image data is medical image data D1 in which a first region of interest R1 before editing and a second region of interest R2 before editing are set, and is medical image data D1 immediately after the first region of interest R1 and the second region of interest R2 are set in the image processing device 30. The finalized medical image data is medical image data D1 in which the first region of interest R1 after the finalization and the second region of interest R2 after the finalization are set, and is medical image data D1 in which the first region of interest R1 and the second region of interest R2 are set when the signal removal process is executed.

[0103] Furthermore, the machine learning may be, for example, machine learning using a support vector machine (SVM) or deep learning using a multi-layer neural network such as a convolutional neural network (CNN) or a convolutional deep belief neural network. In the following description, an example will be given in which the machine learning is deep learning using a neural network, and the machine learning model is a trained model using deep learning.

[0104] The learning function 359 is a function for constructing a trained model. In this embodiment, the learning function 359 constructs a trained model using a training data set in which the unedited medical image data is used as training data and the finalized medical image data is used as teacher data.

[0105] Fig. 19 is an explanatory diagram illustrating an example of a data flow during learning of the editing function 355a. As shown in Fig. 19, during learning, the editing function 355a sequentially updates the parameter data 505 stored in the memory circuitry 31 by performing deep learning using a large number of training data sets.

[0106] Each training data set of the multiple training sets is composed of a set of a learning data group 501 and a teacher data group 503. The learning data group 501 is composed of learning data 5011, 5012, 5013, .... Each learning data is composed of pre-edited medical image data, which is medical image data D1 in which a first region of interest R1 before editing and a second region of interest R2 before editing are set. In addition, the teacher data group 503 corresponding to each learning data is composed of teacher data 5031, 5032, 5033, .... Each teacher data 5031, 5032, 5033, ... is composed of finalized medical image data, which is medical image data D1 in which a first region of interest R1 after finalization and a second region of interest R2 after finalization are set.

[0107] The editing function 355a is controlled by the learning function 359, and updates the parameter data 505 each time a training data set is provided so that the result of processing the learning data by the neural network 507 approaches the teacher data, that is, the editing function 355a performs learning. In general, when the rate of change of the parameter data 505 converges within a threshold, learning is determined to be completed. Hereinafter, the parameter data 505 after learning is completed is particularly referred to as learned parameter data 505t.

[0108] Fig. 20 is an explanatory diagram illustrating an example of a data flow during operation of the editing function 355a. As shown in Fig. 20, during operation, the editing function 355a inputs pre-edited medical image data to a trained model, and obtains edited medical image data D1, which is medical image data in which a first region of interest after editing and a second region of interest after editing are set, output from the trained model, thereby editing the first region of interest R1 and / or the second region of interest R2.

[0109] The trained parameter data 505t and the neural network 507 constitute the trained model 50. Various methods can be used for this type of training and for constructing the trained model. The neural network 507 is stored in the memory circuitry 31 in the form of a program. The trained parameter data 505t may be stored in the memory circuitry 31 or in a storage medium connected to the processing circuitry 35. When the trained model 50 (the neural network 507 and the trained parameter data 505t) is stored in the memory circuitry 31, the editing function 355a realized by the processor of the processing circuitry 35 reads the trained model 50 from the memory circuitry 31 and executes it, inputting pre-edited medical image data into the trained model 50 and acquiring edited medical image data output from the trained model 50, thereby editing the first region of interest R1 and / or the second region of interest R2. In addition, the trained model 50 may be constructed using an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0110] 20, the edited medical image data output from the trained model has the first region of interest after editing and the second region of interest after editing set, but the edited medical image data may have the first region of interest before editing and the second region of interest after editing set, or the first region of interest after editing and the second region of interest before editing set. That is, both the first region of interest and the second region of interest set in the edited medical image data output from the trained model do not necessarily have to be edited, and at least one of the first region of interest and the second region of interest set in the edited medical image data output from the trained model may be edited.

[0111] Next, the signal removal process executed by the image processing device 30 according to this embodiment will be described with reference to Fig. 21. Fig. 21 is a flow chart for explaining the contents of the signal removal process executed by the image processing device 30 according to this embodiment, and corresponds to Fig. 3 in the first embodiment described above. The signal removal process according to this embodiment differs from the signal removal process according to the first embodiment in that the first region of interest R1 and / or the second region of interest R2 are automatically edited. Note that the processes of steps S11 to S23 shown in Fig. 21 are similar to those of the first embodiment described above, and therefore will not be described.

[0112] Then, in step S23, if the first region of interest R1 and / or the second region of interest R2 are edited (step S23: Yes), the image processing device 30 edits the first region of interest R1 and / or the second region of interest R2 (step S61). The process of editing the first region of interest R1 and / or the second region of interest R2 is realized by the editing function 355a in the processing circuitry 35. Specifically, the image processing device 30 inputs pre-edited medical image data to the trained model 50 and acquires edited medical image data output from the trained model 50, thereby editing the first region of interest R1 and / or the second region of interest R2.

[0113] 21, the image processing device 30 displays the edited medical image data (step S63). The process of displaying the edited medical image data is realized by the display control function 352 in the processing circuitry 35. Specifically, the image processing device 30 displays on the display 32 the edited medical image data, which is medical image data D1 in which the edited first region of interest R1 and the edited second region of interest R2 are set.

[0114] The process of step S29 after step S63 is the same as that of the first embodiment described above, and therefore the description thereof will be omitted. Also, the process of step S29 after the step shown in Fig. 21 is the same as that of Fig. 4 in the first embodiment described above, and therefore the description thereof will be omitted.

[0115] As described above, according to the image processing system 1 of this embodiment, as in the first embodiment, the image processing device 30 sets the first region of interest R1 for the medical image data D1, sets the second region of interest R2 different from the first region of interest R1, and executes the signal removal process based on the first region of interest R1 and the second region of interest R2, so that it is possible to reduce the risk of misdiagnosis due to the region that should not be removed being removed as an unnecessary signal. Also, according to the image processing system 1 of this embodiment, as in the first embodiment, the image processing device 30 sets the second region of interest R2 by extracting the observation target based on the anatomical feature point information of the observation target, so that it is possible to extract the observation target even when the signal has disappeared due to bleeding.

[0116] According to the image processing system 1 of the present embodiment, the image processing device 30 automatically edits the first region of interest R1 and / or the second region of interest R2 set in the medical image data D1 using the trained model 50, so that the burden on the user in editing the first region of interest R1 and / or the second region of interest R2 can be reduced. Furthermore, since the first region of interest R1 and / or the second region of interest R2 set in the medical image data D1 are automatically edited using the trained model 50, it is possible to learn the tendency of editing the first region of interest R1 and / or the second region of interest R2 for each user, each medical image diagnostic device, or each hospital facility, and it is possible to present the first region of interest R1 and the second region of interest R2 in accordance with the tendency of editing the first region of interest R1 and / or the second region of interest R2 to the user.

[0117] In the above-described fourth embodiment, the processing circuitry 35 of the image processing device 30 may not include the learning function 359. In other words, the trained model 50 may not be constructed in the image processing device 30, and the first region of interest R and / or the second region of interest R2 may be edited by inputting pre-edited medical image data to the trained model 50 constructed by performing learning in a device different from the image processing device 30.

[0118] Specifically, for example, when a learning device equipped with a learning function 359 different from the image processing device 30 is used, the learning device uses the unedited medical image data and the finalized medical image data to update the parameter data 505 so that the result of processing the learning data with the neural network 507 approaches the teacher data, and constructs a trained model 50 composed of the trained parameter data 505t and the neural network. Then, the image processing device 30 may edit the first region of interest R and / or the second region of interest R2 by inputting the unedited medical image data into the trained model 50 trained by the learning device.

[0119] Furthermore, in the above-described fourth embodiment, the image processing device 30 may be configured to allow the user to manually edit the first region of interest R1 and / or the second region of interest R2 after displaying the edited medical image data on the display 42 in step S63. That is, the image processing device 30 may be configured to accept the second input operation and / or the third input operation by the user after displaying the edited medical image data on the display 42 in step S63.

[0120] [Modifications of the first to fourth embodiments] In the above-mentioned first to fourth embodiments, the second setting function 354 of the processing circuit 35 in the image processing device 30 can also apply a segmentation process to the medical image data D1 to set the second region of interest R2. Specifically, the second setting function 354 may use the segmentation process to extract an observation target in the medical image data D1 and set the second region of interest R2 so as to include the observation target included outside the first region of interest R1 in the medical image data D1 as the region of interest. More specifically, the second setting function 354 extracts the observation target using a signal difference between a signal of the observation target such as a blood vessel and a background signal of the observation target by the segmentation process, and sets the second region of interest R2.

[0121] The method of this segmentation processing is not particularly limited. That is, the method of the segmentation processing is arbitrary, and the segmentation processing may be performed using threshold processing, edge detection, region growing method, etc. Also, the segmentation processing may be performed by a machine learning method. For example, a trained model is generated by using medical image data D1 in which the first region of interest R1 is set as learning data, and the determined medical image data in which the determined first region of interest R1 and the determined second region of interest R2 are set as a training set. Then, the second setting function 354 of the processing circuit 35 in the image processing device 30 may input the medical image data D1 in which the first region of interest R1 is set to the trained model trained to perform the segmentation processing in this way, thereby acquiring the medical image data D1 in which the first region of interest R1 and the second region of interest R2 are set, which are output by the trained model, and setting the second region of interest R2.

[0122] In the first embodiment described above, the second setting function 354 of the processing circuit 35 in the image processing device 30 may set the second region of interest R2 based on information related to the first region of interest R1. Specifically, the second setting function 354 may extract and specify a tissue included in the first region of interest R1 in the medical image data D1 as information related to the first region of interest R1, extract the tissue included outside the first region of interest R1 in the medical image data D1 based on the tissue included in the specified first region of interest R1, and set the second region of interest R2 so as to include the tissue included outside the first region of interest R1 as a region of interest. The second setting function 354 may generate pixel value information on pixel values ​​included in the first region of interest R1 in the medical image data D1 as information related to the first region of interest R1, extract a region of the pixel value included outside the first region of interest R1 in the medical image data D1 based on the generated pixel value information, and set the second region of interest so as to include the extracted region.

[0123] Furthermore, in the first to fourth embodiments described above, the image processing device 30 may use the first warning function 356 to determine whether or not an observation target exists in an area outside the first region of interest R1 and the second region of interest R2 in the medical image data D1 during editing of the first region of interest R1 and / or the second region of interest R2, for example, each time the second input operation or the third input operation is received, and may warn the user if an observation target exists in an area outside the first region of interest R1 and the second region of interest R2. Alternatively, the image processing device 30 may, when editing of the first region of interest R1 and / or the second region of interest R2 is completed in step S27, by the first warning function 356, for example, when the editing function 355 accepts completion of editing of the first region of interest R1 and / or the second region of interest R2, determine whether or not an object of observation exists in an area outside the first region of interest R1 and the second region of interest R2, and warn the user if an object of observation exists in an area outside the first region of interest R1 and the second region of interest R2.

[0124] In the first to fourth embodiments described above, the image processing device 30 may determine whether or not there is a difference between the left and right sides of the observation target included in the first region of interest R1 and / or the second region of interest R2 at any timing during editing of the first region of interest R1 and / or the second region of interest R2, for example, every time the second input operation or the third input operation is received, and may warn the user if there is a difference between the left and right sides of the observation target. Alternatively, the image processing device 30 may determine whether or not there is a difference between the left and right sides of the observation target included in the first region of interest R1 and / or the second region of interest R2, for example, when the editing function 355 receives the completion of editing of the first region of interest R1 and / or the second region of interest R2 in step S27, and may warn the user if there is a difference between the left and right sides of the observation target.

[0125] In the above-described first embodiment, when the image processing device 30 acquires the medical image data D1 in step S11, the image processing device 30 may extract anatomical features of the medical image data D1 to warn the user based on the anatomical features of the medical image data D1 when setting the first region of interest R1. Specifically, the image processing device 30 may determine whether or not an observation target exists in a region outside the first region of interest R1 in the medical image data D1 based on the anatomical features of the medical image data D1 during setting of the first region of interest R1 in step S15 of the signal removal process, for example, each time the first setting function 353 accepts a first input operation, and warn the user when an observation target exists in a region outside the first region of interest R1.

[0126] Furthermore, in the first embodiment described above, when the setting of the first region of interest R1 is completed in step S17 of the signal removal process, for example, when the first setting function 353 accepts the completion of the setting of the first region of interest R1, the image processing device 30 may determine, based on anatomical landmarks of the medical image data D1, whether or not an observation target is present in a region outside the first region of interest R1, and may warn the user if an observation target is present in a region outside the first region of interest R1, by the second warning function 357.

[0127] In the first to fourth embodiments described above, the signal removal process steps S23 to S41 may be omitted. That is, the image processing device 30 may execute the signal removal process when the first region of interest R1 and the second region of interest R2 are set for the medical image data D1. In this case, the processing circuitry 35 of the image processing device 30 may not include the editing function 355, the first warning function 356, and the second warning function 357.

[0128] Furthermore, in the above-described first to fourth embodiments, three-dimensional medical image data D1 is used, but two-dimensional medical image data D1 including single-slice medical image data D1 or the like may be used.

[0129] Furthermore, in the above-described first to fourth embodiments, MR image data captured by a magnetic resonance imaging apparatus has been described as an example, but medical image data captured by other medical image diagnostic apparatuses, such as CT image data captured by an X-ray CT apparatus, may also be used as the medical image data D1.

[0130] In the first to fourth embodiments described above, when the second setting function 354 sets the second region of interest R2 by extracting an observation target in the medical image data D1 based on the anatomical feature information, the first warning function 356 or the second warning function 357 may warn the user when a blood vessel or the like that should be present is not present in the medical image data D1. Specifically, in step S19 of the signal removal process, the second setting function 354 sets the second region of interest R2, and the first warning function 356 or the second warning function 357 compares the anatomical feature information on the anatomical feature of the observation target included in the medical image data D1 with the model data of the anatomical feature of the observation target stored in the storage circuitry 31 to determine whether there is a difference between the anatomical feature information and the model data. If there is a difference between the anatomical feature information and the model data, the user may be warned in step S21 or step S53 that a blood vessel or the like that should be present is not found. In this way, it is possible to warn the user when a signal that should be present in the medical image data D1 is not present.

[0131] The term "processor" used in the above description means a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). The processor realizes its function by reading and executing a program stored in a memory circuit. Instead of storing a program in a memory circuit, the processor may be configured to directly incorporate the program in the circuit. In this case, the processor realizes its function by reading and executing the program incorporated in the circuit. The processor is not limited to being configured as a single processor circuit, and may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the multiple components in FIG. 1 may be integrated into a single processor to realize its function.

[0132] Although several embodiments have been described above, these embodiments are presented only as examples and are not intended to limit the scope of the invention. The novel device and method described in this specification can be implemented in various other forms. In addition, various omissions, substitutions, and modifications can be made to the forms of the device and method described in this specification without departing from the spirit of the invention. The appended claims and their equivalents are intended to include such forms and modifications that fall within the scope and spirit of the invention. [Explanation of symbols]

[0133] 1... image processing system, 10... medical image diagnostic device, 20... medical image storage device, 30... image processing device, 31... memory circuit, 32... display, 33... input interface, 34... communication interface, 35... processing circuit, 50... trained model, 72... unedited medical image data, 74... edited medical image data

Claims

1. a first setting unit that sets a first region of interest in the medical image data; a second setting unit that sets a second region of interest different from the first region of interest set by the first setting unit; a signal removal processing unit that performs a signal removal process on the medical image data based on the first region of interest and the second region of interest; An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the first setting unit receives a first input operation from a user via an input unit, the first input operation being an input operation related to setting the first region of interest in the medical image data, and sets the first region of interest in accordance with the first input operation.

3. The image processing apparatus according to claim 1 , wherein the first setting unit sets the first region of interest by executing an analysis process on the medical image data.

4. The image processing apparatus according to claim 1 , wherein the first setting unit sets the first region of interest based on a region of interest set in a positioning image corresponding to the medical image data.

5. The image processing device according to claim 1 , wherein the second setting unit sets the second region of interest based on anatomical feature information that is information on anatomical feature points included in the medical image data.

6. The image processing device according to claim 1 , wherein the second setting unit applies a segmentation process to the medical image data to set the second region of interest.

7. The image processing device according to claim 1 , wherein the second setting unit sets the second region of interest based on information related to the first region of interest.

8. The image processing device according to claim 1 , wherein the signal removal processing unit displays a confirmation screen for prompting a user to confirm whether or not to execute the signal removal processing when an instruction to execute the signal removal processing is input.

9. The image processing device according to claim 1 , further comprising an editing unit that edits the first region of interest and / or the second region of interest.

10. 10. The image processing device according to claim 9, wherein the editing unit receives a second input operation from a user via an input unit, the second input operation being an input operation for editing the first region of interest set in the medical image data, and edits the first region of interest in accordance with the second input operation.

11. 10. The image processing device according to claim 9, wherein the editing unit receives a third input operation from a user via an input unit, the third input operation being an input operation for editing the second region of interest set in the medical image data, and edits the second region of interest in accordance with the third input operation.

12. 10. The image processing device according to claim 9, wherein the editing unit edits the first region of interest and / or the second region of interest by inputting the pre-edited medical image data to a trained model trained using pre-edited medical image data in which the first region of interest before editing and the second region of interest before editing are set, and finalized medical image data in which the first region of interest after confirmation and the second region of interest after confirmation are set, as a training set.

13. The image processing apparatus according to claim 1 , further comprising a display control unit that causes a display unit to display the second region of interest superimposed on the medical image data.

14. the first region of interest is a region including an observation target included in the medical image data, The second region of interest is a region including an observation target existing outside the first region of interest. The image processing device according to claim 1 .

15. The image processing device according to claim 14 , wherein the observation target is a blood vessel.

16. The image processing device according to claim 14, further comprising a first warning unit that, when the object to be observed is present in an area outside the first region of interest and outside the second region of interest in the medical image data, warns a user that the object to be observed is present in an area outside the first region of interest and outside the second region of interest.

17. The image processing device according to claim 14, further comprising a second warning unit that, when there is a laterality in the object of observation included in the first region of interest and / or the second region of interest in the medical image data, warns a user that there is a laterality in the object of observation.

18. The image processing apparatus according to claim 1 , further comprising an acquisition unit configured to acquire the medical image data from a medical image diagnostic apparatus that generates the medical image data or a medical image storage apparatus that stores the medical image data.

19. An image processing method by an image processing device, comprising: Setting a first region of interest in medical image data; setting a second region of interest different from the set first region of interest; performing a signal removal process on the medical image data based on the first region of interest and the second region of interest; An image processing method comprising:

20. On the computer, Setting a first region of interest in medical image data; setting a second region of interest different from the set first region of interest; performing a signal removal process on the medical image data based on the first region of interest and the second region of interest; A program that executes the following.

Images