Medical image processing device and medical image processing method

Abstract

To enable setting of an appropriate region of interest.SOLUTION: A medical image processor comprises an acquisition unit, a setting unit, an analysis unit, and an output unit. The acquisition unit acquires a medical image. The setting unit sets a plurality of candidate regions on the medical image as candidates for a region of interest to be set on the medical image. The analysis unit executes quantitative analysis on an analyte's composition for each of the plurality of candidate regions. The output unit outputs a plurality of analysis results corresponding to the respective candidate regions.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The embodiments disclosed in this specification relate to a medical image processing device. [Background technology]

[0002] Techniques for performing quantitative analysis on regions of interest set on medical images are becoming widespread. Known quantitative analysis techniques include bone mineral density (BMD) and breast density (BD). The results of quantitative analysis may be affected by the region of interest set. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-104798 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the problems that the embodiments disclosed herein aim to solve is to enable appropriate setting of a region of interest. However, the problems solved by the embodiments disclosed herein are not limited to the above problem. Problems corresponding to the effects of the configurations described in the embodiments below can also be considered as other problems solved by the embodiments disclosed herein. [Means for solving the problem]

[0005] A medical image processing device according to an embodiment includes an acquisition unit, a setting unit, an analysis unit, and an output unit. The acquisition unit acquires a medical image. The setting unit sets a plurality of candidate regions on the medical image as candidates for a region of interest to be set on the medical image. The analysis unit performs quantitative analysis of the composition of the subject for each of the plurality of candidate regions. The output unit outputs a plurality of analysis results corresponding to each of the plurality of candidate regions. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a medical information processing system according to the first embodiment. [Figure 2] FIG. 2 is a diagram for explaining bone mineral quantification measurement according to the first embodiment. [Figure 3A] FIG. 3A is a diagram for explaining bone mineral quantification measurement according to the first embodiment. [Figure 3B] FIG. 3B is a diagram for explaining bone mineral quantification measurement according to the first embodiment. [Figure 3C] FIG. 3C is a diagram for explaining bone mineral quantification measurement according to the first embodiment. [Figure 4] FIG. 4 is a diagram for explaining bone mineral density measurement according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating the candidate region according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing a display example according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing a display example according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing a display example according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing a display example according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing a display example according to the first embodiment. [Figure 11]FIG. 11 is a flowchart for explaining the flow of a series of processes performed by the medical image processing apparatus according to the first embodiment. [Figure 12] FIG. 12 is a diagram for explaining breast density measurement according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, an embodiment of a medical image processing apparatus will be described in detail with reference to the accompanying drawings.

[0008] (First embodiment) The following describes an example of a medical information processing system 1 including a medical image diagnostic device 10, an image storage device 20, and a medical image processing device 30. For example, the medical image diagnostic device 10, the image storage device 20, and the medical image processing device 30 are connected to each other via a network NW as shown in FIG.

[0009] The medical image diagnostic apparatus 10 is an apparatus that captures an image of a subject P to collect medical images. When the medical image diagnostic apparatus 10 is an X-ray diagnostic apparatus, the medical image diagnostic apparatus 10 collects X-ray images by irradiating X-rays onto the subject P and detecting the X-rays that have passed through the subject P. In addition, the medical image diagnostic apparatus 10 transmits the collected X-ray images to the image storage apparatus 20 or the medical image processing apparatus 30 via a network NW.

[0010] The image storage device 20 stores various medical images. For example, the image storage device 20 receives X-ray images collected by the medical image diagnostic device 10 and stores them in a memory provided inside or outside the device. For example, the image storage device 20 is a server for a PACS (Picture Archiving and Communication System).

[0011] The medical image processing device 30 performs various processes described below using the medical images collected by the medical image diagnostic device 10. For example, as shown in FIG. 1, the medical image processing device 30 includes an input interface 31, a display 32, a memory 33, and a processing circuit 34.

[0012] The input interface 31 accepts various input operations from the user, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuit 34. For example, the input interface 31 may be implemented by a mouse, keyboard, trackball, switch, button, joystick, a touchpad that allows input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit using an optical sensor, a voice input circuit, etc. The input interface 31 may also be configured as a tablet terminal or the like that can wirelessly communicate with the medical image processing device 30. The input interface 31 may also be a circuit that accepts input operations from the user through motion capture. For example, the input interface 31 can accept the user's body movements, gaze, etc. as input operations by processing signals acquired via a tracker and images collected about the user. The input interface 31 is not limited to those that include physical operating components such as a mouse and keyboard. For example, an example of the input interface 31 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the medical image processing device 30 and outputs this electrical signal to the processing circuit 34.

[0013] The display 32 displays various types of information. For example, under the control of the processing circuitry 34, the display 32 displays medical images collected by the medical image diagnostic apparatus 10 and the results of analysis performed using the medical images. In addition, for example, the display 32 displays a GUI (Graphical User Interface) for receiving various instructions, settings, and the like from a user via the input interface 31. For example, the display 32 is a liquid crystal display or a CRT (Cathode Ray Tube) display. The display 32 may be a desktop type, or may be configured as a tablet terminal or the like capable of wireless communication with the main body of the medical image processing apparatus 30.

[0014] 1, the medical image processing apparatus 30 is described as including the display 32, but the medical image processing apparatus 30 may include a projector instead of or in addition to the display 32. The projector can project onto a screen, a wall, a floor, the body surface of the subject P, etc. under the control of the processing circuitry 34. As an example, the projector can also project onto any plane, object, space, etc. by projection mapping.

[0015] The memory 33 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, an optical disk, etc. For example, the memory 33 stores a program that enables a circuit included in the medical image processing device 30 to realize its function. The memory 33 also stores medical images collected by the medical image diagnostic device 10. The memory 33 may also be realized by a group of servers (cloud) connected to the medical image processing device 30 via a network NW.

[0016] The processing circuitry 34 controls the operation of the entire medical image processing device 30 by executing a control function 34a, an acquisition function 34b, a setting function 34c, an analysis function 34d, an output function 34e, and a reception function 34f. The acquisition function 34b is an example of an acquisition unit. The setting function 34c is an example of a setting unit. The analysis function 34d is an example of an analysis unit. The output function 34e is an example of an output unit. The reception function 34f is an example of a reception unit.

[0017] For example, the processing circuit 34 reads out a program corresponding to the control function 34a from the memory 33 and executes it to control various functions such as the acquisition function 34b, the setting function 34c, the analysis function 34d, the output function 34e, and the reception function 34f based on various input operations received from the user via the input interface 31.

[0018] Furthermore, for example, the processing circuitry 34 acquires medical images by reading and executing a program corresponding to the acquisition function 34b from the memory 33. For example, the acquisition function 34b acquires, via the network NW, X-ray images collected by the medical image diagnostic apparatus 10 and stored in the image storage device 20. Alternatively, the acquisition function 34b may acquire X-ray images directly from the medical image diagnostic apparatus 10 without going through the image storage device 20.

[0019] Furthermore, for example, the processing circuitry 34 reads from the memory 33 and executes a program corresponding to the setting function 34c to set multiple candidate regions on the medical image as candidates for a region of interest to be set on the medical image. For example, the processing circuitry 34 reads from the memory 33 and executes a program corresponding to the analysis function 34d to perform quantitative analysis of the composition of the subject P for each of the multiple candidate regions. For example, the processing circuitry 34 reads from the memory 33 and executes a program corresponding to the output function 34e to output multiple analysis results corresponding to each of the multiple candidate regions. For example, the processing circuitry 34 reads from the memory 33 and executes a program corresponding to the reception function 34f to receive an operation from a user who has referenced the analysis results to select one of the multiple candidate regions as a region of interest. The processing performed by the setting function 34c, the analysis function 34d, the output function 34e, and the reception function 34f will be described later.

[0020] 1, each processing function is stored in the form of a computer-executable program in memory 33. Processing circuitry 34 is a processor that realizes the function corresponding to each program by reading and executing the program from memory 33. In other words, once a program has been read, the processing circuitry 34 has the function corresponding to the read program.

[0021] 1, the control function 34a, the acquisition function 34b, the setting function 34c, the analysis function 34d, the output function 34e, and the reception function 34f are described as being realized by a single processing circuit 34, but the processing circuit 34 may be configured by combining multiple independent processors, and each processor may execute a program to realize the functions. Furthermore, each processing function of the processing circuit 34 may be realized by being appropriately distributed or integrated into a single or multiple processing circuits.

[0022] The processing circuitry 34 may also realize its functions by using a processor of an external device connected via a network NW. For example, the processing circuitry 34 reads and executes a program corresponding to each function from the memory 33, and realizes each function shown in Fig. 1 by using a group of servers (cloud) connected to the medical image processing apparatus 30 via the network NW as a computational resource.

[0023] The above describes an example configuration of the medical information processing system 1 including the medical image processing device 30. With this configuration, the medical image processing device 30 sets a region of interest on an X-ray image collected by the medical image diagnostic device 10, and performs quantitative analysis according to the region of interest.

[0024] For example, the medical image processing device 30 performs bone mineral quantitative measurement as a quantitative analysis to calculate bone mineral density (BMD). In this case, the medical image diagnostic device 10 uses a plurality of X-ray energies to acquire X-ray images corresponding to each X-ray energy. For example, the medical image diagnostic device 10 performs dual energy acquisition to acquire a first X-ray image corresponding to a first X-ray energy and a second X-ray image corresponding to a second X-ray energy. As an example, the medical image diagnostic device 10 can perform dual energy acquisition by controlling the tube voltage supplied to the X-ray tube. Furthermore, the medical image diagnostic device 10 transmits the first X-ray image and the second X-ray image to the image storage device 20 or the medical image processing device 30 via a network NW.

[0025] Next, the acquisition function 34b acquires the first X-ray image and the second X-ray image from the image archiving device 20 or the medical image diagnostic device 10. Next, the acquisition function 34b generates a bone density image based on the first X-ray image and the second X-ray image.

[0026] For example, the acquisition function 34b performs material decomposition processing to separate bone components present in the imaging target region from other components. For example, the acquisition function 34b defines calcium (Ca) and water as reference materials and estimates, for each pixel, the abundance ratio of calcium to water and the abundance ratio of water to calcium. This allows the acquisition function 34b to generate a material-decomposed image corresponding to the calcium component as a bone density image. While the acquisition function 34b has been described as generating a bone density image, the acquisition function 34b may also acquire a bone density image generated by the medical image diagnostic device 10 or another device via the network NW.

[0027] Next, the setting function 34c sets a region of interest on the bone density image. For example, the setting function 34c sets a region of interest R11 as shown in FIG. 2. For example, the region of interest R11 is soft tissue. The region of interest R11 may be set automatically by the setting function 34c or may be set based on a user operation. Note that FIG. 2 is a diagram for explaining bone mineral density measurement according to the first embodiment. In FIG. 2, a case where bone mineral density measurement of a vertebra is performed will be described.

[0028] The region of interest R11 shown in Figure 2 may include transverse processes of vertebrae or bones other than vertebrae (such as ribs and ilium). In this case, the BMD may be calculated to be low. The setting function 34c further sets a neutral region as the region of interest, as shown in Figures 3A, 3B, and 3C. Figures 3A, 3B, and 3C are diagrams for explaining bone mineral density measurement according to the first embodiment.

[0029] For example, the setting function 34c sets a region of interest R21 corresponding to vertebrae as shown in Figure 3A. The setting function 34c also sets a region of interest R22 corresponding to transverse processes of vertebrae and bones other than vertebrae as shown in Figure 3B. That is, the region of interest R22 is a neutral region that does not correspond to either vertebrae or soft tissue. The setting function 34c also sets a region of interest R23 corresponding to soft tissue as shown in Figure 3C.

[0030] The analysis function 34d performs bone mineral quantification measurement based on the set region of interest. Hereinafter, a case where a region of interest corresponding to bone and a region of interest corresponding to soft tissue are set as shown in Fig. 4 will be described. Fig. 4 is a diagram for explaining bone mineral quantification measurement according to the first embodiment. In Fig. 4, the horizontal axis of the bone density image is the x-axis and the vertical axis is the y-axis.

[0031] When calculating BMD, bone regions are extracted from bone density images and a region of interest (ROI) is set. Therefore, the BMD value is affected by the ROI, and if the ROI is not set appropriately, the BMD value may be inappropriate. However, the boundaries between tissues, such as between bone and soft tissue, can be ambiguous, making it difficult to set the ROI. Several methods exist for setting the ROI, and methods using artificial intelligence (AI) have also been proposed. However, neither method guarantees that the ROI will always be set appropriately. Furthermore, while users can fine-tune the set ROI based on their preferences or the patient, it is difficult to understand the impact of these fine-tuning adjustments on quantitative analysis.

[0032] The medical image processing device 30 enables appropriate setting of a region of interest through the process described in detail below. Specifically, first, the acquisition function 34b acquires a medical image. For example, the acquisition function 34b generates a bone density image based on a first X-ray image and a second X-ray image acquired with dual energy. Alternatively, the acquisition function 34b acquires a bone density image generated by another device via the network NW.

[0033] Next, the setting function 34c sets a plurality of candidate regions on the medical image as candidates for the region of interest to be set on the medical image. This will be described below with reference to FIG. 5. FIG. 5 is a diagram for explaining the candidate regions according to the first embodiment. For ease of explanation, FIG. 5 shows only the vertebrae included in the bone density image. For example, as shown in FIG. 5, the setting function 34c sets candidate region R31, candidate region R32, and candidate region R33 as candidates for the region of interest corresponding to the vertebrae.

[0034] The setting function 34c can set the candidate region using various methods. For example, the setting function 34c can set the candidate region using a trained model that is configured to accept input of a medical image and estimate a region of interest. The trained model may be generated by the setting function 34c or may be generated by a device other than the medical image processing device 30. The trained model is stored, for example, in the memory 33 and is read and used by the setting function 34c as appropriate.

[0035] For example, the trained model is configured by a convolutional neural network. A convolutional neural network is a network that propagates information from the input layer to the output layer while maintaining the relationships between pixels. For example, a region of interest that was previously set on a medical image and for which no abnormal values ​​were found in the results of bone mineral density measurement based on that region of interest is used as training data. By using a medical image as input data and training data for the region of interest as output data and training a multi-layered neural network, a trained model is generated that has parameters that generate training data for the region of interest from the input data. Note that the multi-layered neural network is configured, for example, by an input layer, multiple intermediate layers (hidden layers), and an output layer.

[0036] The setting function 34c inputs the bone density image acquired by the acquisition function 34b into the trained model, and sets the estimation result of the region of interest obtained by the trained model as a candidate region.

[0037] Note that multiple trained models may be used, and multiple estimation results of regions of interest obtained by the respective trained models may be set as candidate regions. In this case, the trained models are generated based on training data with different strictness of region determination, creators, or groups.

[0038] The setting function 34c may set the candidate region by threshold processing. For example, the setting function 34c may estimate a region corresponding to bone by comparing the pixel value of each pixel in the bone density image acquired by the acquisition function 34b with a threshold, and set the result as the candidate region.

[0039] The setting function 34c may also set the candidate region by performing graph cut processing on the bone density image.

[0040] The setting function 34c may also set a candidate region through a user operation. For example, the output function 34e displays the bone density image acquired by the acquisition function 34b on the display 32, and the user performs an operation to draw the outline of the candidate region while referring to the bone density image. In this case, the setting function 34c may set the region drawn by the user as the candidate region. The output function 34e displays multiple candidate regions on the display 32, and the user performs an operation to adjust the displayed candidate region. In this case, the setting function 34c sets the region adjusted by the user as a new candidate region.

[0041] The setting function 34c may also set the candidate region by performing morphological processing. For example, the setting function 34c sets the candidate region by enlarging or reducing the region output from the trained model as the region of interest estimation result. For example, the setting function 34c sets the region output from the trained model as the region of interest estimation result as candidate region R32, sets the region obtained by reducing candidate region R32 as candidate region R31, and sets the region obtained by enlarging candidate region R32 as candidate region R33.

[0042] For example, the setting function 34c receives a user instruction to select one of multiple pre-set parameters and performs morphological processing based on the selected parameter. For example, the setting function 34c receives a user instruction to select one of parameters such as "80%," "90%, "110%,," or "120%." For example, if "110%" is selected, the setting function 34c enlarges the area output from the trained model as the estimated region of interest to the size of "110%" while maintaining its shape and center position. Alternatively, the setting function 34c may perform morphological processing based on any parameter received from the user. That is, the setting function 34c may perform discrete or continuous morphological processing.

[0043] In another example, the output function 34e displays a bone density image on the display 32, and the user specifies an arbitrary point on the bone density image. The setting function 34c sets a candidate region by enlarging or reducing the region output from the trained model as an estimation result of the region of interest so that it passes through the point specified by the user. For example, the output function 34e displays candidate region R31, candidate region R32, and candidate region R33 shown in FIG. 5 on the display 32. The user specifies an arbitrary point in the region between candidate region R31 and candidate region R32 or in the region between candidate region R32 and candidate region R33. For example, when a point in the region between candidate region R31 and candidate region R32 is specified, the setting function 34c enlarges or reduces candidate region R31 or candidate region R32 so that it passes through the point specified by the user, thereby setting the candidate region.

[0044] Although the case where a candidate region is set by performing morphology processing on a region output from a trained model has been described, the embodiment is not limited to this. For example, the setting function 34c may set a candidate region by performing morphology processing on a region set by threshold processing, a region set by graph cut processing, a region created by user operation, or the like.

[0045] Furthermore, the setting function 34c may set candidate regions including cortical bone and candidate regions not including cortical bone. Generally, the bone density of cortical bone differs from that of the cancellous bone within the cortical bone, and therefore the pixel values ​​on the bone density image also differ. Therefore, the setting function 34c can extract cortical bone regions from the bone density image and set candidate regions including cortical bone and candidate regions not including cortical bone. The output function 34e may display multiple candidate regions so that it can be identified whether each candidate region includes cortical bone or not.

[0046] The analysis function 34d performs bone mineral quantification measurement for each of the plurality of candidate regions set by the setting function 34c. For example, the analysis function 34d calculates the BMD for each of the plurality of candidate regions R31 to R33.

[0047] The output function 34e outputs the analysis results obtained by the analysis function 34d. The output function 34e displays a plurality of analysis results in association with a plurality of candidate regions. For example, the output function 34e displays a plurality of candidate regions R31 to R33 in association with their respective calculated BMDs on the display 32.

[0048] For example, the output function 34e displays on the display 32 a graph, as shown in Figure 6, with the type of candidate region (ROI TYPE) such as candidate region R31, candidate region R32, and candidate region R33 on the horizontal axis and the BMD value on the vertical axis.

[0049] The user decides whether to select a candidate region based on the diagram displayed on the display 32. The user roughly estimates the BMD value based on various information, such as patient information about the subject P, such as the age and sex of the subject P, the symptoms the subject P is complaining of, past test results about the subject P, and impressions based on the bone density image. The diagram shown in FIG. 6 serves as a basis for deciding which candidate region is appropriate as a region of interest.

[0050] Specifically, the user refers to the display of the bone density image and candidate regions shown in Figure 5 and selects one of candidate regions R31, R32, and R33. Here, it is conceivable that the user may determine that candidate region R33 is inappropriate, but may be unsure which of candidate region R31 and candidate region R32 is more appropriate. In such a case, the user can refer to the diagram shown in Figure 6 to select, as the region of interest, candidate region R31 or candidate region R32, which has a more appropriate BMD value.

[0051] The reception function 34f accepts an operation to select a region of interest. For example, a user may select a region of interest by clicking a mouse, tapping a touch panel, or by voice input. For example, the reception function 34f accepts an operation to specify a position on the graph shown in FIG. 6 or a position on text such as "R31," "R32," or "R33" as an operation to select a region of interest. The reception function 34f also accepts an operation to specify a position on the bone density image shown in FIG. 5 as an operation to select a region of interest. The reception function 34f also accepts an operation to specify a position on the legend shown in FIG. 5 as an operation to select a region of interest.

[0052] The output function 34e outputs the analysis results corresponding to the selected region of interest. For example, when a candidate region R32 is selected as the region of interest, the output function 34e displays a report containing the BMD of the candidate region R32 on the display 32. This allows the user to perform various diagnoses, such as understanding the effects of treatment on the subject P and formulating a treatment plan.

[0053] The user may be a doctor or a laboratory technician. For example, the reception function 34f receives a selection of a region of interest from a doctor, and the output function 34e displays the analysis results corresponding to the selected region of interest to the doctor. Also, for example, the reception function 34f receives a selection of a region of interest from a laboratory technician, and the output function 34e displays the analysis results corresponding to the selected region of interest to the doctor.

[0054] Furthermore, although the case where the output function 34e controls the display of the display 32 has been described, the embodiment is not limited to this. For example, the output function 34e may control a projector to project the graph of FIG. 6. For example, the output function 34e may transmit the graph of FIG. 6 to another device, and a display of the other device may display it to the user. For example, the output function 34e may output the graph of FIG. 6 from a printer and provide it to the user. The same applies to various displays described below.

[0055] 6 is an example, and various modifications are possible. The output function 34e may display the BMD calculated based on candidate region R31, the BMD calculated based on candidate region R32, and the BMD calculated based on candidate region R33 in a table or text format, in association with candidate region R31, candidate region R32, and candidate region R33, respectively.

[0056] The output function 34e may also display an image showing the candidate region in association with the analysis results. For example, as shown in FIG. 7, the output function 34e displays an image showing the candidate region R31 on a graph, linked to a plot of BMD calculated based on the candidate region R31. Similarly, the output function 34e displays an image showing the candidate region R32 on a graph, linked to a plot of BMD calculated based on the candidate region R32. Similarly, the output function 34e displays an image showing the candidate region R33 on a graph, linked to a plot of BMD calculated based on the candidate region R33.

[0057] Furthermore, when bone mineral quantification measurements are performed on subject P multiple times, output function 34e may display the analysis results corresponding to each of the multiple candidate regions in association with the time axis. This will be described below with reference to Fig. 8. Fig. 8 shows a case where bone mineral quantification measurements are performed on subject P at measurement date and time T1, measurement date and time T2, and measurement date and time T3. For example, setting function 34c sets candidate regions R41, R42, R43, R44, and R45 at each measurement date and time.

[0058] As an example, the setting function 34c inputs a bone density image collected from the subject P at measurement date and time T1 into the trained model, and sets the region output from the trained model as the candidate region R41. The setting function 34c also inputs a bone density image collected from the subject P at measurement date and time T2 into the trained model, and sets the region output from the trained model as the candidate region R41. The setting function 34c also inputs a bone density image collected from the subject P at measurement date and time T3 into the trained model, and sets the region output from the trained model as the candidate region R41. In other words, the setting function 34c sets the region output from the trained model as the candidate region R41 for each measurement date and time.

[0059] As another example, the setting function 34c sets a candidate region R42 at each measurement date and time by performing morphology processing on the region output from the first trained model. The setting function 34c also sets a region output from a second trained model, which is different from the first trained model, as a candidate region R43 at each measurement date and time. As another example, the setting function 34c sets a candidate region R44 at each measurement date and time by performing threshold processing. As another example, the setting function 34c sets a candidate region R45 at each measurement date and time by performing graph cut processing.

[0060] For example, as shown in Fig. 8, the output function 34e displays a three-axis graph consisting of an axis showing the type of candidate region (ROI TYPE), an axis showing the BMD value, and an axis showing the measurement date and time. The user selects a region of interest by referring to the BMD calculated this time as well as BMDs calculated in the past. In other words, the user can select a region of interest by taking into account the track record and reliability of each candidate region.

[0061] Here, the output function 34e may detect outlier candidates from multiple analysis results. For example, in FIG. 8, the BMD corresponding to the candidate region R44 set at measurement date and time T3 is exceptionally high. In such a case, the output function 34e detects the BMD of (R44, T3) as an outlier candidate. For example, the output function 34e calculates the standard deviation of the BMD values ​​for each measurement date and time and each candidate region, and sets an error range based on the standard deviation. For example, if the bone density image includes multiple vertebrae, the output function 34e can set an error range by performing statistical processing on the BMD calculated for each vertebra. For another example, the output function 34e can estimate the number of photons based on the pixel values ​​of the bone density image and set an error range based on the estimation result. For example, the output function 34e can estimate the amount of quantum noise according to a Poisson distribution based on the estimated photon number, and set an error range by summing the amount of quantum noise and a predetermined amount of noise corresponding to circuit noise. If the BMD value is outside the error range, the output function 34e detects it as a possible outlier. For example, the output function 34e detects a BMD value that is outside a predetermined interval (e.g., a 3σ range) on the probability distribution as a possible outlier.

[0062] Alternatively, the output function 34e may detect outlier candidates for each measurement date and time. For example, the output function 34e calculates the standard deviation of the BMD value for each candidate region for each measurement date and time, and compares it with an error range to detect outlier candidates. Alternatively, the output function 34e may detect outlier candidates for each candidate region. For example, the output function 34e calculates the standard deviation of the BMD value for each measurement date and time for each candidate region, and compares it with an error range to detect outlier candidates.

[0063] The error range may be set based on a smaller candidate region. That is, the smaller the candidate region, the less likely it is that an inappropriate region will be included within the candidate region. For example, when setting a region of interest corresponding to a bone, the smaller the candidate region, the less likely it is that soft tissue, etc. will be included within the candidate region. On the other hand, the smaller the candidate region, the fewer pixels it contains, making it more susceptible to statistical error. Here, the output function 34e can improve the accuracy of detecting outlier candidates by applying the error range set based on a smaller candidate region to other candidate regions.

[0064] In the following, we will explain an example in which candidate region R41 in Figure 8 is the smallest, followed by candidate region R42, candidate region R43, candidate region R44, and candidate region R45 in that order. For example, the output function 34e sets an error range for each candidate region based on candidate region R41. If the error of candidate region R41 is σ based on past measurements, device specifications, etc., the error in region R4X is equivalent to "σ × region area of ​​R41 / region area of ​​R4X." Then, the output function 34e detects outlier candidates by comparing the BMD of each candidate region with the error range set for that region.

[0065] Furthermore, if an abnormal value candidate is detected, the output function 34e may display an image showing a candidate region associated with the abnormal value candidate. For example, if a BMD at (R44, T3) is detected as an abnormal value candidate, the output function 34e may display an image showing candidate region R44 set at measurement date and time T3, as shown in Fig. 9. Furthermore, the output function 34e may display an image showing candidate region R44 set at measurement dates and times T1 and T2, which were not detected as an abnormal value candidate, so that images of the same candidate region but measured at different dates and times can be compared.

[0066] Furthermore, the output function 34e may highlight areas where bone density has changed significantly by coloring or the like. This allows the user to estimate the cause of the abnormal value candidate. For example, if there are many areas where bone density has changed significantly, it is estimated that the abnormal value candidate has occurred due to an actual change in bone density. On the other hand, if there are no areas where bone density has changed significantly, it is estimated that the abnormal value candidate has occurred because the set candidate region is inappropriate.

[0067] Note that the display in FIG. 9 is an example, and various modifications are possible. For example, the output function 34e may provide an image display area in addition to the area displaying the graph shown in FIG. 9, and display an image showing the candidate region R44 set at measurement date and time T3 and an image showing the candidate region R44 set at measurement date and time T2 in a stack. Furthermore, although FIGS. 8 and 9 illustrate the display of three-dimensional graphs, the output function 34e may also be configured to appropriately switch to display two-dimensional graphs. For example, the output function 34e may be configured to switch between displaying a two-dimensional graph consisting of two axes representing the type of candidate region (ROI TYPE) and the BMD value and a two-dimensional graph consisting of two axes representing the measurement date and time and the BMD value, based on a user operation.

[0068] Furthermore, the output function 34e may change the display target based on the outlier candidate. For example, if the BMD of (R44, T3) is detected as an outlier candidate, the output function 34e may not display the BMD value corresponding to the candidate region R44, as shown in FIG. 10. Such a change in the display target may be reflected from the measurement date and time T3 when the outlier candidate was detected, or may be reflected from the next measurement onwards. Furthermore, when each candidate region is used as learning data for machine learning, the candidate region detected as an outlier candidate may be excluded from the learning data. Note that FIG. 10 is a diagram showing a display example according to the first embodiment.

[0069] While the above description concerns the case where a region of interest is set corresponding to a vertebra, the same can be applied to the case where multiple regions of interest are set. For example, the setting function 34c sets candidate regions R211 and R212 as candidates for the region of interest corresponding to the vertebrae. Furthermore, the setting function 34c sets candidate regions R221 and R222 as candidates for the region of interest corresponding to the neutral region. Furthermore, the setting function 34c sets candidate regions R231 and R232 as candidates for the region of interest corresponding to the soft tissue. That is, when multiple regions of interest are set, the setting function 34c sets multiple candidate regions for each region of interest.

[0070] Next, the analysis function 34d performs bone mineral quantification for each combination of candidate regions. For example, the analysis function 34d calculates BMD based on the combination of candidate region R211, candidate region R221, and candidate region R231. The analysis function 34d also calculates BMD based on the combination of candidate region R211, candidate region R221, and candidate region R232. The analysis function 34d also calculates BMD based on the combination of candidate region R211, candidate region R222, and candidate region R231. The analysis function 34d also calculates BMD based on the combination of candidate region R211, candidate region R222, and candidate region R232. The analysis function 34d also calculates BMD based on the combination of candidate region R212, candidate region R221, and candidate region R231. The analysis function 34d also calculates the BMD based on a combination of candidate regions R212, R221, and R232. The analysis function 34d also calculates the BMD based on a combination of candidate regions R212, R222, and R231. The analysis function 34d also calculates the BMD based on a combination of candidate regions R212, R222, and R232.

[0071] Next, the output function 34e outputs the analysis results. For example, the output function 34e displays the BMD calculated based on the combination of candidate regions in association with the combination of candidate regions used to calculate the BMD. For example, the output function 34e displays a graph on the display 32, with the horizontal axis representing the combination of candidate regions and the vertical axis representing the BMD. The receiving function 34f then receives a user operation to select one of the multiple candidate regions as a region of interest. For example, if the user selects a BMD value calculated based on the combination of candidate regions R211, R221, and R231, the receiving function 34f can receive a user operation by regarding candidate region R211 as the region of interest corresponding to the vertebrae, candidate region R221 as the candidate region of interest corresponding to the neutral region, and candidate region R231 as the candidate region of interest corresponding to the soft tissue.

[0072] Next, an example of the processing procedure by the medical image processing apparatus 30 will be described with reference to Fig. 11. Fig. 11 is a flowchart for explaining a series of processing flows of the medical image processing apparatus 30 according to the first embodiment. Step S101 corresponds to the acquisition function 34b. Step S102 corresponds to the setting function 34c. Step S103 corresponds to the analysis function 34d. Step S104 corresponds to the output function 34e. Steps S105 and S106 correspond to the reception function 34f.

[0073] The processing circuitry 34 acquires a medical image (step S101). For example, the processing circuitry 34 generates a bone density image based on a first X-ray image and a second X-ray image. For example, the processing circuitry 34 acquires the bone density image via a network NW. Next, the processing circuitry 34 sets a plurality of candidate regions (step S102). Next, the processing circuitry 34 performs quantitative analysis (step S103). For example, the processing circuitry 34 performs bone mineral density (BMD) measurement for each of the plurality of candidate regions.

[0074] Next, processing circuitry 34 displays the analysis results (step S104). For example, processing circuitry 34 displays BMDs corresponding to each of the multiple candidate regions on display 32 by displaying Figures 6, 7, 8, 9, 10, etc. Next, processing circuitry 34 accepts selection of a region of interest (step S105). That is, processing circuitry 34 accepts an operation to select one of the multiple candidate regions as a region of interest from a user who has referred to the analysis results corresponding to each of the multiple candidate regions in step S104.

[0075] Next, the processing circuitry 34 determines whether a region of interest has been selected (step S106). If a region of interest has not been selected (step S106: No), the processing circuitry 34 enters a standby state. Alternatively, if a region of interest has not been selected, the processing circuitry 34 may return to step S102 and reset multiple candidate regions. Here, the processing circuitry 34 may set a candidate region different from the previous one, for example, by using a trained model different from the previous one or by changing the threshold value in the threshold processing. On the other hand, if a region of interest has been selected (step S106: Yes), the processing circuitry 34 ends the processing.

[0076] As described above, according to the first embodiment, the acquisition function 34b acquires a medical image. The setting function 34c sets multiple candidate regions on the medical image as candidates for a region of interest to be set on the medical image. The analysis function 34d performs quantitative analysis on the composition of the subject P for each of the multiple candidate regions. The output function 34e outputs multiple analysis results corresponding to each of the multiple candidate regions.

[0077] Therefore, the medical image processing apparatus 30 according to the first embodiment can set an appropriate region of interest. Specifically, the user can refer to multiple analysis results corresponding to each of the multiple candidate regions and determine which of the multiple candidate regions to select as the region of interest. That is, the user can select a more appropriate region of interest by further using the analysis results obtained when using the candidate region in addition to information such as the medical image itself and the shape of the candidate region set on the medical image.

[0078] As described above, when multiple regions of interest are set on a medical image, the setting function 34c sets multiple candidate regions for each region of interest. The analysis function 34d then performs quantitative analysis for each combination of candidate regions. Therefore, the medical image processing device 30 can set appropriate regions of interest even when performing quantitative analysis using multiple regions of interest, such as bone mineral density measurement.

[0079] Furthermore, as described above, when quantitative analysis has been performed multiple times on the subject P, the output function 34e displays the multiple analysis results in association with a time axis. For example, as shown in FIG. 8, the output function 34e displays a three-axis graph on the display 32, consisting of an axis indicating the type of candidate region, an axis indicating the BMD value, and an axis indicating the measurement date and time. Therefore, the medical image processing device 30 can enable more appropriate setting of a region of interest. Specifically, when determining which of multiple candidate regions to select as a region of interest, the user can further use analysis results obtained in the past using that candidate region as a basis for selection, thereby selecting a more appropriate region of interest.

[0080] Furthermore, as described above, according to the first embodiment, the output function 34e detects outlier candidates from multiple analysis results. Therefore, the medical image processing device 30 can set a more appropriate region of interest. For example, when an outlier candidate is detected, the output function 34e can display an image showing a candidate region associated with the outlier candidate. This allows the user to determine whether the outlier candidate has actually arisen due to a change in bone density or the like, or whether the candidate region is inappropriate. Furthermore, if it is determined that the outlier candidate has arisen due to an inappropriate candidate region, the user can avoid selecting such a candidate region as a region of interest.

[0081] When performing bone mineral quantification measurements on the subject P multiple times, the medical image processing device 30 may basically perform the above-described processing only in the first bone mineral quantification measurement. That is, in the first bone mineral quantification measurement, the setting function 34c sets multiple candidate regions, the analysis function 34d performs bone mineral quantification measurements for each of the candidate regions, and the output function 34e outputs multiple analysis results corresponding to each of the candidate regions. This allows the user to select an appropriate region of interest. Here, in subsequent bone mineral quantification measurements, the region of interest set in the first bone mineral quantification measurement can be used. This allows the medical image processing device 30 to standardize the analysis conditions for multiple bone mineral quantification measurements, making it easier to evaluate changes over time.

[0082] However, due to changes in the bone mineral density of the subject P, it may be inappropriate to use the region of interest set in the first bone mineral quantification measurement in subsequent bone mineral quantification measurements. In this case, the medical image processing device 30 may also perform the above-described processing in subsequent bone mineral quantification measurements. That is, the setting function 34c may set multiple candidate regions in subsequent bone mineral quantification measurements, the analysis function 34d may perform bone mineral quantification measurements for each of the candidate regions, and the output function 34e may output multiple analysis results corresponding to each of the candidate regions. The medical image processing device 30 may be configured to perform the above-described processing in subsequent bone mineral quantification measurements, or may perform the above-described processing in response to a user request. For example, the region of interest set in the first bone mineral quantification measurement may be used in subsequent bone mineral quantification measurements, and a report may be created. If a user viewing the report determines that the BMD value listed in the report is inappropriate, the medical image processing device 30 may additionally perform the above-described processing.

[0083] (Second embodiment) In the above-described embodiment, a bone density image is generated and a candidate region is set. However, the setting function 34c may preprocess the bone density image prior to setting the candidate region and set the candidate region in the preprocessed bone density image. For example, the setting function 34c performs a cortical bone removal process on the bone density image as preprocessing. Specifically, the setting function 34c extracts a region corresponding to the cortical bone from the bone density image and replaces the extracted region with background components, etc. When such preprocessing is performed, the candidate region is set based on the cancellous bone, excluding the cortical bone. Generally, changes in bone density are small in the cortical bone and large in the cancellous bone. Therefore, calculating BMD based on the cancellous bone can provide more useful information to the user.

[0084] In the above-described embodiment, the trained model is configured using a convolutional neural network, but the trained model may be configured using other machine learning techniques, such as extracting multiple region of interest candidates by extracting features using a digital filter or pattern matching, and then classifying and estimating the final region of interest using techniques such as boosting or a support vector machine.

[0085] In the above-described embodiment, bone mineral quantification measurement has been described as an example of quantitative analysis. However, the embodiment is not limited to this, and can be similarly applied to various quantitative analyses related to the composition of the subject P.

[0086] For example, the analysis function 34d may perform breast density measurement as a quantitative analysis of the composition of the subject P and calculate breast density (BD). In this case, the medical information processing system 1 includes, for example, a mammography device as the medical image diagnostic device 10. For example, the medical image diagnostic device 10 captures images of the breast of the subject P and collects mammography images such as MLO (Mediolateral-Oblique) images, CC (Cranio-Caudal) images, and tomosynthesis images. In addition, the acquisition function 34b acquires mammography images from the image storage device 20 or the medical image diagnostic device 10.

[0087] In breast density measurement, two regions of interest are set, such as region of interest R51 and region of interest R52 in FIG. 12. Region of interest R51 is a region corresponding to the breast. Region of interest R52 is a region where the mammary glands are present. For example, analysis function 34d can calculate BD by dividing the area of ​​region of interest R52 by the area of ​​region of interest R51. Here, as in the case of BMD, if the regions of interest are not set appropriately, BD may also have an inappropriate value. Note that FIG. 12 is a diagram for explaining breast density measurement according to the second embodiment.

[0088] For example, the setting function 34c sets candidate regions R511 and R512 as candidates for the region of interest R51. The setting function 34c also sets candidate regions R521 and R522 as candidates for the region of interest R52. That is, the setting function 34c sets multiple candidate regions for each of the multiple regions of interest.

[0089] Next, the analysis function 34d calculates BD based on the combination of candidate region R511 and candidate region R521. The analysis function 34d also calculates BD based on the combination of candidate region R511 and candidate region R522. The analysis function 34d also calculates BD based on the combination of candidate region R512 and candidate region R521. The analysis function 34d also calculates BD based on the combination of candidate region R512 and candidate region R522. That is, the analysis function 34d performs breast density measurement for each combination of candidate regions.

[0090] Next, the output function 34e outputs the analysis results. For example, the output function 34e displays the BD calculated based on the combination of candidate regions in association with the combination of candidate regions used to calculate the BD. For example, the output function 34e displays on the display 32 a graph with the combination of candidate regions on the horizontal axis and the BD on the vertical axis.

[0091] Then, the reception function 34f receives an operation from the user to select one of the multiple candidate regions as the region of interest. For example, if the user selects a value of BD calculated based on a combination of candidate region R511 and candidate region R521, the reception function 34f can receive an operation from the user by regarding candidate region R511 as the region of interest R51 and candidate region R521 as the region of interest R52.

[0092] When breast density measurement is performed multiple times on subject P, medical image processing device 30 may basically perform the above-described process each time. That is, since breast density measurement basically requires setting a region of interest each time, setting function 34c sets multiple candidate regions for each breast density measurement, analysis function 34d performs breast density measurement for each of the multiple candidate regions, and output function 34e outputs multiple analysis results corresponding to each of the multiple candidate regions.

[0093] In the above-described embodiment, X-ray images such as bone density images and mammography images have been described as examples of medical images. However, the embodiment is not limited to this. For example, the present invention can be similarly applied to cases where quantitative analysis is performed using medical images such as X-ray CT (Computed Tomography) images, PET (Positron Emission Computed Tomography) images, SPECT (Single Photon Emission Computed Tomography) images, ultrasound images, and MR (Magnetic Resonance) images.

[0094] 1, the medical image diagnostic apparatus 10, the image storage apparatus 20, and the medical image processing apparatus 30 may be installed in any location as long as they are connectable via the network NW. For example, the medical image processing apparatus 30 may be installed in a hospital different from the hospital where the medical image diagnostic apparatus 10 is installed. In other words, the network NW may be configured as a closed local network within the hospital, or may be a network via the Internet.

[0095] Furthermore, in the above-described embodiment, the medical image diagnostic apparatus 10 and the medical image processing apparatus 30 are described as separate entities, but the medical image processing apparatus 30 may be included in the medical image diagnostic apparatus 10. For example, a console device in the medical image diagnostic apparatus 10 may execute the functions of the medical image diagnostic apparatus 10. In this case, the acquisition function 34b can acquire medical images by capturing an image of the subject P.

[0096] The term "processor" used in the above description refers to a circuit such as a CPU, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA)). When the processor is a CPU, for example, the processor realizes its function by reading and executing a program stored in a memory circuit. On the other hand, when the processor is an ASIC, for example, instead of storing a program in a memory circuit, the function is directly incorporated into the processor circuit as a logic circuit. Note that each processor in the embodiments is not limited to being configured as a single circuit, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, multiple components in each figure may be integrated into a single processor to realize its function.

[0097] 1, a single memory 33 is described as storing programs corresponding to each processing function of the processing circuit 34. However, the embodiment is not limited to this. For example, a configuration may be adopted in which multiple memories 33 are distributed and the processing circuit 34 reads corresponding programs from individual memories 33. Furthermore, instead of storing programs in the memory 33, a configuration may be adopted in which the programs are directly embedded in the circuitry of the processor. In this case, the processor realizes the functions by reading and executing the programs embedded in the circuitry.

[0098] The components of each device according to the above-described embodiments are conceptual and functionally independent, and are not necessarily physically configured as shown in the drawings. In other words, the specific form of distribution and integration of each device is not limited to that shown in the drawings, and all or part of each device can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0099] Furthermore, the medical image processing method described in the above-described embodiment can be realized by executing a prepared medical image processing program on a computer such as a personal computer or a workstation. This medical image processing program can be distributed via a network such as the Internet. Furthermore, this medical image processing program can be recorded on a non-transitory computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and can be executed by being read from the recording medium by a computer.

[0100] According to at least one of the embodiments described above, it is possible to set an appropriate region of interest.

[0101] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0102] 1 Medical information processing system 10 Medical imaging diagnostic equipment 30 Medical image processing device 34 Processing circuit 34a Control Functions 34b Acquisition function 34c Setting Function 34d analysis function 34e Output Function 34th floor Reception function

Claims

1. an acquisition unit for acquiring medical images; a setting unit that sets a plurality of candidate regions on the medical image as candidates for a region of interest to be set on the medical image; an analysis unit that performs quantitative analysis of the composition of the subject for each of the plurality of candidate regions; an output unit that displays an image showing the plurality of candidate regions and information associating the plurality of candidate regions with analysis results that are results of the quantitative analysis performed on each of the plurality of candidate regions; a receiving unit that receives an operation to select the region of interest from a user who has referred to the display by the output unit, The output unit outputs an analysis result relating to the selected region of interest in response to the operation.

2. The medical image processing apparatus according to claim 1 , wherein the accepting unit accepts, as the operation for selecting the region of interest, an operation for specifying positions of a plurality of legends corresponding to a plurality of the candidate regions.

3. When a plurality of regions of interest are set on the medical image, the setting unit sets a plurality of the candidate regions for each of the regions of interest; The medical image processing apparatus according to claim 1 , wherein the analysis unit executes the quantitative analysis for each combination of the candidate regions.

4. The medical image processing device according to any one of claims 1 to 3, wherein the setting unit inputs the medical image to a trained model that is functionally configured to accept the input of the medical image and estimate the region of interest, and sets a region output from the trained model as an estimation result of the region of interest as one of the plurality of candidate regions.

5. The medical image processing apparatus according to claim 4 , wherein the setting unit inputs the medical image to each of the plurality of trained models.

6. The medical image processing apparatus according to claim 1 , wherein the setting unit sets at least one of the plurality of candidate regions by performing threshold processing on pixel values ​​of the medical image.

7. The medical image processing apparatus according to claim 6 , wherein the setting unit sets at least two of the plurality of candidate regions by performing the threshold processing a plurality of times with different thresholds.

8. The medical image processing apparatus according to claim 1 , wherein the setting unit sets at least one of the plurality of candidate regions by performing graph cut processing on the medical image.

9. The medical image processing apparatus according to claim 1, wherein the setting unit sets a region created by a user operation as one of the plurality of candidate regions.

10. The medical image processing apparatus according to claim 1 , wherein the setting unit sets at least one of the plurality of candidate regions by morphological processing.

11. The medical image processing apparatus according to claim 1 , wherein the output unit further displays an image showing the candidate region in association with a plurality of the analysis results in the information.

12. The medical image processing device according to any one of claims 1 to 11, wherein when the quantitative analysis is performed multiple times on the same subject, the output unit displays the multiple analysis results further corresponding to a time axis.

13. The medical image processing apparatus according to claim 1, wherein the output unit further detects outlier candidates from among the plurality of analysis results.

14. The medical image processing apparatus according to claim 13 , wherein the output unit further displays an image showing the candidate region associated with the abnormal value candidate.

15. The medical image processing apparatus according to claim 14 , wherein the output unit displays an image showing the candidate region so that changes over time before and after the occurrence of the abnormal value candidate can be identified.

16. The medical image processing device according to any one of claims 13 to 15, wherein the output unit sets an error range for each of the plurality of candidate areas based on a smaller candidate area, and detects the outlier candidates by comparing the analysis results corresponding to each of the plurality of candidate areas with the error range set for the candidate area.

17. The medical image processing apparatus according to claim 13, wherein the setting unit further adjusts a method for setting the candidate region based on the abnormal value candidate.

18. The medical image processing apparatus according to claim 1, wherein the analysis unit executes bone mineral quantification measurement as the quantitative analysis.

19. The medical image processing apparatus according to any one of claims 1 to 17, wherein the analysis unit performs breast density measurement as the quantitative analysis.

20. Acquire medical images; setting a plurality of candidate regions on the medical image as candidates for a region of interest to be set on the medical image; performing a quantitative analysis of the composition of the analyte for each of the plurality of candidate regions; displaying an image showing the plurality of candidate regions and information associating the plurality of candidate regions with analysis results that are results of the quantitative analysis performed on each of the plurality of candidate regions; receiving an operation to select the region of interest from a user who has referred to the display; In response to the operation, an analysis result regarding the selected region of interest is output. A medical image processing method comprising:

Images