Medical image processing equipment, ultrasound diagnostic equipment, and programs

Abstract

The goal is to reduce the effort required to select the appropriate frame and enable proper detection of lesions. [Solution] The medical image processing apparatus of the embodiment comprises an acquisition unit and a detection unit. The acquisition unit acquires contrast-enhanced images of a subject to which a contrast agent has been administered during the process from the arterial-dominant phase through the portal vein-dominant phase to the post-vascular phase, at least from the portal vein-dominant phase onward. The detection unit detects areas in the contrast-enhanced images from the portal vein-dominant phase onward where the contrast agent has been washed out as defective areas.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to medical image processing apparatuses, ultrasonic diagnostic apparatuses, and programs.

Background Art

[0002] There is a diagnostic technique for imaging an image (video) of a subject and detecting a lesion based on the obtained image. In this technique, for example, a contrast agent is administered to the subject, and a lesion is detected based on how the administered contrast agent stains. For example, a site where the contrast agent has washed out and disappeared after staining is detected as a lesion.

[0003] In order to observe how the contrast agent stains, an operator selects an appropriate image from, for example, a plurality of frames. However, since the number of images to be selected is enormous, it takes time to select an appropriate image. If an appropriate image cannot be selected, it is difficult to appropriately detect a lesion.

Prior Art Documents

Patent Documents

[0004] <00​​​​​​​​​​​​​​​​​​​​​​ [Means for solving the problem]

[0006] The medical image processing apparatus of the embodiment comprises an acquisition unit and a detection unit. The acquisition unit acquires contrast-enhanced images of a subject to which a contrast agent has been administered during the process from the arterial-dominant phase through the portal vein-dominant phase to the retrovascular phase, at least from the portal vein-dominant phase onward. The detection unit detects areas in the contrast-enhanced images from the portal vein-dominant phase onward where the contrast agent has been washed out as missing areas. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram showing an example of the configuration of the ultrasound diagnostic device 1 of the first embodiment. [Figure 2] A diagram showing an example of an image displayed on display 130. [Figure 3] A diagram showing an example of a phase change after administration of a contrast agent. [Figure 4] A schematic diagram showing contrast-enhanced and tissue images. [Figure 5] A diagram showing an example of TIC. [Figure 6] A flowchart illustrating an example of the processing performed by the medical image processing device 100. [Figure 7] A flowchart illustrating an example of the processing performed by the medical image processing device 100. [Figure 8] A flowchart showing an example of the process for updating the first trained model 184. [Figure 9] This figure schematically shows the internal images in the arterial-dominant phase and the portal vein-dominant phase of the second embodiment. [Figure 10] This figure schematically shows tissue images, arterial-dominant phase, and portal vein-dominant phase before contrast agent administration in the third embodiment. [Figure 11] A block diagram showing one example configuration of the medical information processing device 300 according to the fourth embodiment. [Modes for carrying out the invention]

[0008] The following describes the embodiment of the medical image processing device, ultrasound diagnostic device, medical information processing device, and program with reference to the drawings.

[0009] (First Embodiment) The medical image processing device of the first embodiment is provided, for example, in an ultrasound diagnostic device. An ultrasound diagnostic device is a medical device that captures medical images of a subject. Medical image processing devices include, for example, ultrasound diagnostic devices, X-ray CT (Computed Tomography) devices, PET (positron emission tomography)-CT devices, MRI (Magnetic Resonance Imaging) devices, and the like.

[0010] Figure 1 is a block diagram showing an example of the configuration of an ultrasound diagnostic apparatus 1 according to the first embodiment. The ultrasound diagnostic apparatus 1 comprises, for example, an ultrasound probe 10 and a medical image processing device 100. The ultrasound diagnostic apparatus 1 is installed, for example, in a medical institution such as a hospital. The ultrasound diagnostic apparatus 1 is operated by an operator, for example, a technician or a doctor, and captures and stores medical images of the inside of a subject, who is a patient. Based on the stored medical images, the ultrasound diagnostic apparatus 1 creates findings of the subject.

[0011] In examinations using ultrasound diagnostic equipment 1, for example, a contrast agent is administered to the subject, and the presence or absence of defects is detected according to the staining and washout status of the administered contrast agent. If a defect is detected, findings are created based on the size of the defect, the presence or absence of staining in the arterial phase, the presence or absence of washout, and the timing and degree of washout. Washout refers to, for example, the decrease in the concentration of the contrast agent seen in the contrast-enhanced image. The more the staining disappears in the contrast-enhanced image, the more advanced the washout is.

[0012] The ultrasonic probe 10 is, for example, operated by an operator and pressed against a part of the subject (the part to be examined or diagnosed). The ultrasonic probe 10 transmits (irradiates) ultrasonic waves to the subject at regular intervals, for example, in order to acquire an image of the inside of the subject. The ultrasonic probe 10 receives the echo (reflected wave) of the transmitted ultrasonic wave.

[0013] The ultrasonic probe 10 generates data of the received echo signal (hereinafter referred to as echo data). The ultrasonic probe 10 outputs the generated echo data to the medical image processing device 100. The ultrasonic probe 10 receives the echo of the ultrasonic wave at regular intervals and generates echo data. For this reason, echo data at regular intervals is output to the medical image processing device 100.

[0014] The medical image processing device 100 includes, for example, a communication interface 110, an input interface 120, a display 130, a processing circuit 140, and a memory 180. The communication interface 110 communicates with an external device via a communication network NW. The communication interface 110 includes, for example, a communication interface such as a NIC (Network Interface Card).

[0015] The input interface 120 receives various input operations from the operator of the ultrasonic diagnostic apparatus 1, converts the received input operations into electrical signals, and outputs them to the processing circuit 140. For example, the input interface 120 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch panel, etc. The input interface 120 may be, for example, a user interface that receives voice input such as a microphone. When the input interface 120 is a touch panel, the input interface 120 may also have the display function of the display 130.

[0016] Note that in this specification, the input interface is not limited to only those equipped with physical operation components such as a mouse and a keyboard. For example, an electric signal processing circuit that receives an electric signal corresponding to an input operation from an external input device provided separately from the apparatus and outputs this electric signal to the control circuit is also included in the examples of the input interface.

[0017] The display 130 displays various kinds of information. For example, the display 130 displays an image generated by the processing circuit 140, a GUI (Graphical User Interface) for receiving various input operations from the operator, and the like. For example, the display 130 is an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube) display, an organic EL (Electro Luminescence) display, or the like.

[0018] The processing circuit 140 includes, for example, an acquisition function 141, an image generation function 142, a display control function 143, a detection function 144, a time determination function 145, a boundary setting function 146, a defect degree determination function 147, a time curve generation function 148, a peak frame selection function 149, and a finding creation function 150. The processing circuit 140 realizes these functions, for example, by a hardware processor (computer) executing a program stored in the memory 180 (storage circuit).

[0019] A hardware processor refers to circuits such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC), and programmable logic devices (e.g., Simple Programmable Logic Device (SPLD) or Complex Programmable Logic Device (CPLD), Field Programmable Gate Array (FPGA)). Instead of storing the program in memory 180, the hardware processor may be configured to directly incorporate the program into its circuitry. In this case, the hardware processor functions by reading and executing the program incorporated into the circuitry. The program may be stored in memory 180 beforehand, or it may be stored in a non-temporary storage medium such as a DVD or CD-ROM, and installed from the non-temporary storage medium to memory 180 when the non-temporary storage medium is mounted in the drive device (not shown) of the ultrasound diagnostic device 1. A hardware processor is not limited to being a single circuit; it may also be composed of multiple independent circuits combined to perform various functions. Alternatively, multiple components may be integrated into a single hardware processor to perform various functions.

[0020] Memory 180 can be implemented by semiconductor memory elements such as RAM (Random Access Memory), flash memory, hard disks, or optical discs. These non-transient storage media may also be implemented by other storage devices connected via a communication network NW, such as NAS (Network Attached Storage) or external storage server devices. Memory 180 may also include non-transient storage media such as ROM (Read Only Memory) or registers. Memory 180 stores tissue image data 181, contrast-enhanced image data 182, annotation information 183, a first trained model 184, a second trained model 185, and a third trained model 186.

[0021] The acquisition function 141 acquires imaging data output from the echo data output by the ultrasound probe 10. The echo data and imaging data are accompanied by time information indicating the time when the ultrasound probe 10 received the echo that was the source of the echo data (the time when the imaging data was captured). The acquisition function 141 acquires the time information along with the echo data.

[0022] The acquisition function 141 receives input operations from the operator to the input interface 120. For example, after the ultrasound diagnostic device 1 has started examining a subject, if a contrast agent is administered to the subject, the operator can input the administration information. When the input interface 120 receives the administration information, it outputs an electrical signal corresponding to the administration information to the acquisition function 141. The acquisition function 141 is an example of an acquisition unit.

[0023] The image generation function 142 generates internal images, which are images showing the internal state of a subject, based on the echo data acquired by the acquisition function 141. The image generation function 142 generates and acquires internal images, specifically images based on echo data of tissue images (hereinafter referred to as tissue images) and images based on echo data of contrast agents (hereinafter referred to as contrast images). The image generation function 142 distinguishes between echo data of tissue images and echo data of contrast agents based, for example, on the imaging method or imaging bandwidth used when visualizing the echo data. Time information added to the echo data of tissue images and echo data of contrast agents is added to the internal images. The image generation function 142 is an example of an acquisition unit.

[0024] The image generation function 142 generates internal video (imaging video and contrast-enhanced video) by arranging multiple internal images (tissue images and contrast-enhanced images) as frames in a time series, for example. The image generation function 142 stores the generated tissue images and contrast-enhanced images in the memory 180 as tissue image data 181 and contrast-enhanced image data 182, respectively. Tissue image data 181 includes tissue images and tissue video, and contrast-enhanced image data 182 includes contrast-enhanced images and contrast-enhanced video.

[0025] The display control function 143 displays the internal images on the display 130 after the series of internal images during the examination have been generated by the image generation function 142 and the tissue image data 181 and contrast-enhanced image data 182 have been stored in the memory 180. The image generation function 142 generates internal images based on echo data at regular intervals. Therefore, the internal images generated by the image generation function 142 are, so to speak, a continuous series of images.

[0026] Figure 2 shows an example of an image displayed on display 130. In the center of display 130, the tissue image GA11 and the contrast-enhanced image GA12 are displayed side by side. In the upper right of display 130, the first thumbnail image GA21 to the eighth thumbnail image GA28 are displayed.

[0027] The first thumbnail image GA21 to the eighth thumbnail image GA28 are examples of internal images for each time period, arranged chronologically after the administration of a contrast agent to the subject. The phases after the administration of the contrast agent to the subject transition in the order of arterial-dominant phase, portal vein-dominant phase, and retrovascular phase. The image generation function 142, for example, classifies the tissue images and contrast images after the administration of the contrast agent into arterial-dominant phase, portal vein-dominant phase, and retrovascular phase, generates them as videos for each time period, and stores them in memory 180. The period between the arterial-dominant phase and the portal vein-dominant phase is, for example, the first intermediate phase, and the period between the portal vein-dominant phase and the retrovascular phase is, for example, the second intermediate layer.

[0028] Figure 3 shows an example of phase changes after administration of a contrast agent. The arterial-dominant phase, portal-dominant phase, and retrovascular phase are defined, for example, based on annotation information 183. Annotation information 183 may be information set for the course of the examination, and the arterial-dominant phase, portal-dominant phase, and retrovascular phase may be determined based on time information measured by a timer.

[0029] For example, the period between 10 and 30 seconds after the administration of the contrast agent (information indicating the time elapsed since the administration of the contrast agent) is defined as the arterial-dominant phase, the period between 60 and 90 seconds as the portal vein-dominant phase, and the period after 5 minutes or more as the retrovascular phase. In this embodiment, the arterial-dominant phase, portal vein-dominant phase, and retrovascular phase utilize time information measured by a timer.

[0030] In the contrast-enhanced images, the arterial-dominant phase is the image acquired during the time period corresponding to, for example, the first thumbnail image GA21. The portal vein-dominant phase is the image acquired during the time period corresponding to, for example, the sixth thumbnail image GA26. The retrovascular phase is the image acquired during the time period corresponding to, for example, the eighth thumbnail image GA28. For the first thumbnail image GA21 to the eighth thumbnail image GA28, for example, contrast-enhanced images acquired at the beginning of each time period are used. Other images may be used as thumbnail images.

[0031] When the operator performs an input operation on any of the first thumbnail images GA21 to the eighth thumbnail image GA28, the internal image (internal video) corresponding to the time period of the input thumbnail image is displayed in the center of the display 130. For example, when the operator performs an input operation on the first thumbnail image GA21, the arterial-dominant phase tissue image GA11 and contrast-enhanced image GA12 are displayed in the center of the display 130. The internal image displayed in response to the thumbnail input operation is, for example, a moving image.

[0032] When a tissue image is displayed on the display 130, the operator sets a detection ROI (Region of Interest) at the location of a suspected lesion in the tissue image. The operator uses the input interface 120 to input the operation to set the detection ROI. The input interface 120 outputs an electrical signal corresponding to the operator's input to the processing circuit 140. The acquisition function 141 of the processing circuit 140 acquires the set position of the detection ROI according to the electrical signal output by the input interface 120.

[0033] The detection function 144 detects the presence or absence of defects based on the contrast-enhanced images after the portal vein-dominant phase. In detecting defects, the detection function 144 utilizes the first pre-trained model 184 stored in memory. The first pre-trained model 184 is a pre-trained model that generates training data using multiple contrast-enhanced images in which defects are detected and multiple contrast-enhanced images in which defects are not detected, both in the contrast-enhanced images after the portal vein-dominant phase. The first pre-trained model 184 may also be a pre-trained model that generates training data using multiple contrast-enhanced images in which defects are detected.

[0034] The detection function 144 detects contrast agent washout based on the output result obtained by inputting contrast-enhanced images after the portal vein dominant phase into the first trained model 184. Based on the contrast agent washout status, the detection function 144 detects the presence or absence of defects. The detection function 144 is an example of a detection unit.

[0035] If, after detecting the presence or absence of a defect using contrast-enhanced images from the portal vein-dominant phase onward as input data, no defect is detected, the detection function 144 detects the presence or absence of a lesion based on the output result obtained by inputting tissue images from the arterial-dominant phase as input data to the second pre-trained model 185. The second pre-trained model 185 is a pre-trained model that is generated using multiple tissue images in which lesions are detected and multiple tissue images in which lesions are not detected as training data. The second pre-trained model 185 may also be a pre-trained model that is generated using multiple tissue images in which lesions are detected as training data. Tissue images from the portal vein-dominant phase onward may be used instead of, or in addition to, the arterial-dominant phase tissue images.

[0036] If the detection function 144 detects a lesion based on the output result obtained by inputting an arterial-dominant phase tissue image as input data to the second trained model 185, it needs to identify the location of the lesion in the contrast-enhanced image. For this reason, the detection function 144 selects the frame with the highest likelihood from among multiple contrast-enhanced image frames. Based on the location of the lesion in the tissue image of the selected frame, the detection function 144 identifies the lesion in the contrast-enhanced image.

[0037] If the detection function 144 fails to detect a lesion using an arterial-dominant phase tissue image as input data, it inputs an arterial-dominant phase contrast-enhanced image as input data to the third pre-trained model 186 and detects the presence or absence of a lesion based on the output data. The third pre-trained model 186 is a pre-trained model that generates training data using multiple contrast-enhanced images in which lesions are detected and multiple contrast-enhanced images in which lesions are not detected in the arterial-dominant phase. The third pre-trained model 186 may also be a pre-trained model that generates training data using multiple contrast-enhanced images in which lesions are detected.

[0038] If the detection function 144 detects multiple defects, the display control function 143 displays the contrast-enhanced image containing the multiple defects on the display 130. When the input interface 120 receives an input operation from the operator to specify one of the multiple defects, it outputs an electrical signal corresponding to the input operation to the processing circuit 140. The acquisition function 141 receives the electrical signal output by the input interface 120 and acquires the designation information of the defect specified by the operator.

[0039] The time identification function 145 acquires time information attached to the frame of the contrast-enhanced image in which the defect was first detected (hereinafter referred to as the defect detection frame) when a defect is detected by the detection function 144. Based on the acquired time information, the time identification function 145 identifies the time when the image of the defect detection frame was captured (hereinafter referred to as the defect start time). The time identification function 145 is an example of a time identification unit. The display control function 143 displays the defect start time identified by the time identification function 145 on the display 130.

[0040] The boundary setting function 146 performs segmentation in the contrast-enhanced image detected by the detection function 144, setting a boundary line between the defective area and the surrounding area. When performing segmentation, the boundary setting function 146 sets an ROI that includes the defective area and its surrounding area in the defective detection frame. The ROI can be set in any way; for example, it may be set in the shape of a square where the area ratio of the defective area and the surrounding area are equal.

[0041] The boundary setting function 146 sets a boundary between the defective area and the surrounding area within the set ROI. The defect degree determination function 147 determines the degree of defect in the defective area based on the difference in intensity between the defective area and the surrounding area in the contrast-enhanced image segmented by the boundary setting function 146. The difference in intensity between the defective area and the surrounding area appears as a difference in brightness in the contrast-enhanced image. The boundary setting function 146 is an example of a boundary setting area, and the surrounding area is an example of a non-defective area.

[0042] The defect degree determination function 147 determines, for example, that a marked washout with a large degree of washout is present when the average strength of the peripheral area and the average strength of the defective area satisfy equation (1) below. Furthermore, it determines that a mild washout with a gradual degree of washout is present when the average strength of the defective area satisfies equation (2) below. The defect degree determination function 147 is an example of a defect degree determination unit. Average intensity of the defect / Average intensity of the surrounding area < First threshold ···(1) First threshold ≤ average intensity of the defect / average intensity of the peripheral area < second threshold ... (2)

[0043] The time curve generation function 148, for example, uses detected ROI information in the tissue image (hereinafter referred to as detected ROI information) to correct the movement of the contrast-enhanced image in the arterial phase in the reverse time direction from the portal venous phase and generates a time intensity curve (hereinafter referred to as TIC). The TIC shows the time change in contrast agent staining (intensity of the echo signal) in the lesion and is determined by the time change in brightness of the lesion in the contrast-enhanced image. Figure 4 is a schematic diagram showing the contrast-enhanced image and the tissue image. The position of the defect image GA31 in the contrast-enhanced image GA30 corresponds to the position of the detected ROI GA36 in the tissue image GA35.

[0044] The time curve generation function 148 acquires the TIC of the lesion by correcting for motion in the reverse time direction using the detected ROIGA36 in the tissue image GA35. Figure 5 shows an example of TIC. The solid line L1 shows the time change in the intensity of contrast agent staining in the lesion. The dashed line L2 shows the time change in the intensity of contrast agent staining around the defect. This is an example of the time curve generation unit.

[0045] The peak frame selection function 149 selects a peak frame where contrast agent staining peaks, based on the TIC generated by the time curve generation function 148. The peak frame selection function 149 refers to the TIC to determine the peak time T1 where contrast agent staining peaks in the lesion. The peak frame selection function 149 selects the contrast-enhanced image at peak time T1 as the peak frame. The peak frame selection function 149 stores the selected peak frame in the memory 180. The display control function 143 displays the peak frame selected by the peak frame selection function 149 on the display 130 along with the tissue image at peak time T1. This is an example of the peak frame selection unit.

[0046] The findings creation function 150 creates findings of arterial phase staining based on the signal intensity difference, for example, brightness difference, between the defective area and the peripheral area in the peak frame selected by the peak frame selection function 149. For example, it creates findings of "high" when the staining of the lesion is greater than that of the peripheral area, "iso" when the staining of the lesion and the peripheral area are about the same, and "low" when the staining of the lesion is less than that of the peripheral area. The findings creation function 150 stores the created arterial staining findings in the memory 180. The findings creation function 150 is an example of a findings creation unit.

[0047] Next, an example of the processing performed by the medical image processing device 100 will be described. In the medical image processing device 100, first, the subject is examined and examination data is acquired, and then the examination results are obtained based on the acquired examination data. For this reason, the process for acquiring the examination data will be described first, followed by the process using the acquired examination data.

[0048] Figures 6 and 7 are flowcharts illustrating an example of processing by the medical image processing device 100. Figure 6 shows the processing when acquiring examination data. In the ultrasound diagnostic device 1, when the operator operates the ultrasound probe 10, the ultrasound probe 10 transmits echo data to the medical image processing device 100. The medical image processing device 100 receives the echo data transmitted by the ultrasound probe 10 in its acquisition function 141. The image generation function 142 generates and acquires a tissue image based on the echo data of the tissue image acquired by the acquisition function 141 (step S101), and stores it in the memory 180.

[0049] Next, the image generation function 142 determines whether or not a contrast agent has been administered to the subject based on whether or not an electrical signal corresponding to the administration information has been output via the input interface 120 (step S103). If it is determined that no contrast agent has been administered to the subject, the image generation function 142 returns to step S101 and acquires a tissue image.

[0050] If it is determined that a contrast agent has been administered to the subject, the image generation function 142 acquires a tissue image and, based on the echocardiographic data of the contrast agent acquired by the acquisition function 141, begins acquiring a contrast-enhanced image (step S105). Subsequently, while the phase after the administration of the contrast agent is predominantly arterial, the image generation function 142 acquires an internal image of the predominantly arterial phase and saves it to the memory 180 (step S107).

[0051] While the phase after contrast agent administration is the portal vein-dominant phase, the image generation function 142 acquires internal images of the portal vein-dominant phase and saves them to the memory 180 (step S109). While the phase after contrast agent administration is the retrovascular phase, the image generation function 142 acquires internal images of the retrovascular phase and saves them to the memory 180 (step S111). In this way, the medical image processing device 100 completes the process shown in Figure 6. After completing the process shown in Figure 6, the memory 180 stores multiple tissue images before contrast agent administration, multiple tissue images after contrast agent administration, and a contrast image.

[0052] Next, the process for obtaining examination results based on acquired examination data will be explained. Figure 7 shows the process for obtaining examination results based on acquired examination data. In the medical image processing device 100, the detection function 144 reads the contrast-enhanced image from the portal vein dominant phase onward stored in the memory 180 (step S201). Subsequently, the detection function 144 inputs the read-out contrast-enhanced image from the portal vein dominant phase onward into the first trained model 184 and detects washout based on the output result obtained (step S203).

[0053] The detection function 144 determines whether or not a washout has been detected (step S205). If a washout has been detected, the detection function 144 detects the missing portion in the contrast-enhanced image (step S207). Subsequently, the time identification function 145 identifies the start time of the missing portion (step S209), and the display control function 143 displays the start time of the missing portion on the display 130. The start time of the missing portion displayed on the display control function 143 can be automatically entered as a finding, for example, or added to the findings by the operator through an input operation on the input interface 120.

[0054] Next, the boundary setting function 146 performs segmentation on the missing frame in which the detection function 144 first detected a missing portion (step S211), identifying the missing portion and its surrounding area. Subsequently, the degree of missing portion determination function 147 determines the degree of missing portion in the segmented missing frame (step S213).

[0055] Next, the time curve generation function 148 uses the detected ROI information in the tissue image to correct for motion due to respiratory fluctuations, etc., and generates a TIC of the lesion in the contrast-enhanced image (step S215). Next, the peak frame selection function 149 selects a peak frame based on the TIC generated by the time curve generation function 148 (step S217). Next, the findings creation function 150 creates findings of arterial phase staining based on the brightness difference between the lesion and the surrounding area in the peak frame selected by the peak frame selection function 149 (step S219). In this way, the medical image processing device 100 completes the processing shown in Figure 7.

[0056] In step S205, if it is determined that no washout has been detected, the detection function 144 detects the presence or absence of a lesion based on the results obtained by inputting the arterial-dominant phase tissue image into the second trained model 185. As a result, the detection function 144 determines whether or not a lesion has been detected (step S221).

[0057] If the detection function 144 determines that a lesion has been detected, it identifies the lesion and selects the frame with the highest probability from among multiple contrast-enhanced image frames (step S223). Subsequently, the detection function 144 uses the selected frame as a starting point to perform motion correction in both the temporal and reverse temporal directions and, based on the position of the lesion in the tissue image, determines the position of the lesion in the contrast-enhanced image (step S225). After that, the medical image processing device 100 segments the contrast-enhanced image (step S226) and proceeds to step S215.

[0058] If it is determined in step S221 that no lesion has been detected, the detection function 144 determines whether or not a lesion has been detected from the arterial-dominant phase contrast image (step S227). If the detection function 144 determines that a lesion has been detected, the medical image processing device 100 proceeds to step S223.

[0059] If the detection function 144 determines that no missing portion has been detected, the display control function 143 displays the internal image on the display 130 (step S229). The operator, after viewing the internal image displayed on the display 130, identifies the missing portion (lesion) based on the internal image and performs a manual operation to input findings to the input interface 120. The input interface 120 outputs an electrical signal corresponding to the manual operation to the processing circuit 140. The acquisition function 141 accepts the input based on the examiner's manual operation (step S231). In this way, the medical image processing device 100 completes the process shown in Figure 7.

[0060] During the process of performing an examination using the medical image processing device 100, the medical image processing device 100 simultaneously updates the first trained model 184 to the third trained model 186 based on the input data. The process of updating the first trained model 184 will be described below.

[0061] Figure 8 is a flowchart showing an example of the process for updating the first trained model 184. The medical image processing device 100 determines whether the acquisition function 141 has acquired contrast-enhanced images after the portal vein dominant phase (step S301). If it determines that the acquisition function 141 has not acquired contrast-enhanced images after the portal vein dominant phase, the medical image processing device 100 returns to step S301.

[0062] When the acquisition function 141 determines that it has acquired a contrast-enhanced image from the portal vein dominant phase onward, the medical image processing device 100 updates the first trained model 184 stored in the memory 180 and the acquired contrast-enhanced image (step S303). Subsequently, the medical image processing device 100 stores the updated first trained model 184 in the memory 180 (step S305). In this way, the medical image processing device 100 completes the process shown in Figure 8. The medical image processing device 100 also updates the second trained model 185 and the third trained model 186 using the same procedure.

[0063] The medical image processing device 100 of the first embodiment detects washout based on the contrast-enhanced image after the portal vein-dominant phase. After the portal vein-dominant phase, the changes in the contrast-enhanced image become smaller than in the arterial-dominant phase. Therefore, washout detection becomes easier, reducing the effort required to select appropriate frames such as defect detection frames and peak frames, and enabling appropriate detection of lesions.

[0064] (Second embodiment) Next, a second embodiment will be described. In the first embodiment, internal images were continuously acquired after administering a contrast agent to the subject, but in the second embodiment, contrast-enhanced images in the arterial phase and contrast-enhanced images in the portal vein phase are acquired independently. In this respect, the second embodiment differs mainly from the first embodiment.

[0065] When arterial-dominant phase and portal vein-dominant phase are imaged independently, if a washout is detected in the portal vein-dominant phase and a defect (lesion) is detected, it becomes difficult to identify the lesion in the arterial-dominant phase. Therefore, in the second embodiment, the detection function 144 searches for a pattern corresponding to the location of the defect in the tissue image frame during the time period in which the defect detection frame was acquired, from among frames of any tissue image in the later stage of the arterial-dominant phase. Based on the movement of the searched pattern, the detection function 144 detects the lesion in the contrast-enhanced image of the arterial-dominant phase. Specifically, the detection function 144 searches for the region where the searched pattern best matches the pattern. The detection function 144 detects the lesion by identifying the region in the contrast-enhanced image corresponding to the searched region in the tissue image of the arterial-dominant phase as the lesion.

[0066] Figure 9 schematically shows internal images in the arterial-dominant phase and portal vein-dominant phase of the second embodiment. In Figure 9, contrast-enhanced image GA40 and tissue image GA45 are shown as internal images in the arterial-dominant phase, and contrast-enhanced image GA50 and tissue image GA55 are shown as internal images in the portal vein-dominant phase.

[0067] The detection function 144 extracts, for example, the pattern of the corresponding position image GA56 in the tissue image GA55, which corresponds to the defect image GA51 in the contrast-enhanced image GA50 in the portal vein-dominant phase. Subsequently, the detection function 144 searches for the corresponding position image GA46 in the tissue image GA45 in the arterial-dominant phase that has the pattern that best matches the pattern of the corresponding position image GA56 in the tissue image GA55 in the portal vein-dominant phase. Once the detection function 144 has found the corresponding position image GA46, it identifies the part of the contrast-enhanced image GA40 corresponding to the corresponding position image GA46 as the defect image GA41.

[0068] In the second embodiment, the arterial-dominant phase and the portal vein-dominant phase videos are acquired independently, and the arterial-dominant phase and portal vein-dominant phase videos are discontinuous (not continuous). However, similar to the first embodiment, even if the arterial-dominant phase and portal vein-dominant phase videos are discontinuous, the defect detected in the portal vein-dominant phase can be identified in the contrast-enhanced image in the arterial-dominant phase.

[0069] In the second embodiment, the medical image processing device 100 defines the lesion in the arterial-dominant phase as the position corresponding to the corresponding position image GA46, which has a pattern that best matches the pattern of the corresponding position image GA56 of the tissue image GA55 in the portal vein-dominant phase. Alternatively, the medical image processing device 100 may identify the lesion in the arterial-dominant phase by, for example, manually correcting the movement of the tissue image GA55 in the portal vein-dominant phase in the reverse time direction.

[0070] (Third embodiment) Next, a third embodiment will be described. In the first and second embodiments, internal images acquired during the examination are captured up to the post-vascular phase before identifying the defect and creating findings. However, in the third embodiment, the identification of the defect and creation of findings are performed simultaneously with the examination. The third embodiment differs from the first and second embodiments mainly in this respect.

[0071] Figure 10 schematically shows tissue images, arterial-dominant phase, and portal vein-dominant phase before contrast agent administration in the third embodiment. In the ultrasound diagnostic device 1, the subject is examined by the operator manipulating the ultrasound probe 10. While the subject is being examined, the medical image processing device 100 receives echo data output by the ultrasound probe 10.

[0072] In the first time zone TZ1, before the contrast agent is administered to the subject, the medical image processing device 100 generates a tissue image GA60 in its image generation function 142 based on the echo data of the tissue image output by the ultrasound probe 10. The display control function 143 displays the tissue image GA60 generated by the image generation function 142 based on the echo data of the tissue image output by the ultrasound probe 10 on the display 130.

[0073] As the examination progresses, the second time zone TZ2, immediately after the contrast agent is administered to the subject, is predominantly arterial. The medical image processing device 100 generates a contrast-enhanced image GA61 in its image generation function 142 based on the echo data output by the ultrasound probe 10. The contrast-enhanced image GA61 generated here is a contrast-enhanced image in the predominantly arterial phase. Furthermore, the image generation function 142 generates a tissue image GA62 based on the echo data of the tissue image output by the ultrasound probe 10.

[0074] As the examination progresses, the third time zone TZ3, after the arterial-dominant phase, becomes the portal vein-dominant phase. The medical image processing device 100, similar to the second time zone TZ2, generates a contrast-enhanced image GA63 in its image generation function 142 based on the echo data output by the ultrasound probe 10. The contrast-enhanced image GA63 generated here is a contrast-enhanced image in the portal vein-dominant phase. Furthermore, the image generation function 142 generates a tissue image GA64 based on the echo data of the tissue image output by the ultrasound probe 10.

[0075] In the third embodiment, the medical image processing device 100 sets a detection ROI in the tissue image GA60 during the first time zone TZ1 before administering a contrast agent to the subject. Furthermore, in the detection function 144, the medical image processing device 100 detects the presence or absence of a lesion based on the output result obtained by inputting the tissue image GA60 generated by the image generation function 142 into the second trained model 185.

[0076] In the second time zone TZ2 following the first time zone TZ1, the medical image processing device 100 detects the presence or absence of a lesion based on the output result obtained by inputting the arterial-dominant phase contrast-enhanced image GA61 generated by the image generation function 142 into the third trained model 186 using the detection function 144. If a lesion is detected by the detection function 144, the time curve generation function 148 generates a TIC and the peak frame selection function 149 selects a peak frame. Then, the findings creation function 150 creates findings based on the peak frame selected by the peak frame selection function 149.

[0077] In the third time zone TZ3, following the second time zone TZ2, the medical image processing device 100 detects the presence or absence of missing portions based on the output result obtained by inputting the portal vein-dominant phase contrast-enhanced image GA63 generated by the image generation function 142 into the first trained model 184 using the detection function 144. If a missing portion is detected in the contrast-enhanced image GA63 by the detection function 144, the time identification function 145 obtains the start time of the missing portion.

[0078] After the detection function 144 detects a defect in the second time zone TZ2 or the third time zone TZ3, the boundary setting function 146 performs segmentation to set boundaries around the defect, and the defect degree determination function 147 determines the degree of defect in the defect. Subsequently, if no lesion is detected in the arterial-dominant phase, the time curve generation function 148 generates a TIC, and the peak frame selection function 149 selects a peak frame. Then, the findings creation function 150 creates findings based on the peak frame selected by the peak frame selection function 149.

[0079] The medical image processing apparatus of the third embodiment provides the same effects as the medical image processing apparatus of the first embodiment. Furthermore, the medical image processing apparatus of the third embodiment can detect lesions simultaneously with performing an examination of the subject.

[0080] (Fourth embodiment) Next, a fourth embodiment will be described. The fourth embodiment differs from the first embodiment in that the functions that were incorporated into the processing circuit 140 of the ultrasound diagnostic device 1 are incorporated into the medical information processing device 300. For this reason, the following description will focus on the differences from the first embodiment, and the points that are common to the first embodiment will not be explained. In the description of the fourth embodiment, the same parts as in the first embodiment will be denoted by the same reference numerals.

[0081] Figure 11 is a block diagram showing an example configuration of a medical information processing device 300 according to a fourth embodiment. The medical information processing device 300 receives multiple internal images captured by the ultrasound diagnostic device 1 via a communication network NW. The medical information processing device 300 includes, for example, a communication interface 310, an input interface 320, a display 330, a processing circuit 340, and a memory 380.

[0082] The communication interface 310 communicates with external devices such as the ultrasound diagnostic device 1 via a communication network NW. The communication interface 310 includes, for example, a communication interface such as a NIC.

[0083] The input interface 320 receives various input operations from the operator of the medical information processing device 300, converts the received input operations into electrical signals, and outputs them to the processing circuit 340. For example, the input interface 320 includes a mouse, keyboard, trackball, switch, button, joystick, touch panel, etc. The input interface 320 may also be a user interface that accepts audio input, such as from a microphone. If the input interface 320 is a touch panel, the input interface 320 may also incorporate the display function of the display 330.

[0084] In this specification, the term "input interface" is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device located separately from the device and outputs this electrical signal to a control circuit is also included as an example of an input interface.

[0085] The display 330 displays various types of information. For example, the display 330 displays images generated by the processing circuit 340, or a GUI for receiving various input operations from the operator. For example, the display 330 may be an LCD, a CRT display, or an organic EL display.

[0086] The processing circuit 340 includes, for example, an acquisition function 341, a display control function 343, a detection function 344, a time identification function 345, a boundary setting function 346, a data loss degree determination function 347, a time curve generation function 348, a peak frame selection function 349, and a findings creation function 350. The processing circuit 340 realizes these functions, for example, by a hardware processor (computer) executing a program stored in the memory 380 (storage circuit).

[0087] A hardware processor refers to a circuit such as a CPU, GPU, application-specific integrated circuit (ASIC), or programmable logic device (e.g., a simple programmable logic device (SPLD) or complex programmable logic device (CPLD), or a field-programmable gate array (FPGA)). Instead of storing the program in memory 380, the hardware processor may be configured to directly incorporate the program into its circuitry. In this case, the hardware processor performs its functions by reading and executing the program incorporated into the circuitry. The program may be pre-stored in memory 380, or it may be stored in a non-temporary storage medium such as a DVD or CD-ROM, and installed from the non-temporary storage medium to memory 380 when the non-temporary storage medium is mounted in the drive device (not shown) of the medical information processing device 300. The hardware processor is not limited to being configured as a single circuit; it may be configured as a single hardware processor by combining multiple independent circuits to realize each function. Alternatively, multiple components may be integrated into a single hardware processor to realize each function.

[0088] The acquisition function 341 acquires information of internal images transmitted by the ultrasound diagnostic device 1. The internal information includes tissue image data 381 and contrast-enhanced image data 382. The acquisition function 341 stores the acquired tissue image data 381 and contrast-enhanced image data 382 in the memory 380.

[0089] The display control function 343 displays various images, such as tissue images and contrast-enhanced images, on the display 330 based on the tissue image data 381 and contrast-enhanced image data 382 stored in the memory 380. The detection function 344 has the same function as the detection function 144 of the first embodiment. The detection function 344 detects the presence or absence of defects based on contrast-enhanced images after the portal vein dominant phase, for example.

[0090] The time identification function 345, boundary setting function 346, data loss degree determination function 347, time curve generation function 348, peak frame selection function 349, and finding creation function 350 each have the same functions as the time identification function 145, boundary setting function 146, data loss degree determination function 147, time curve generation function 148, peak frame selection function 149, and finding creation function 150 of the first embodiment.

[0091] The fourth embodiment described above provides the same effects as the first embodiment. Furthermore, the fourth embodiment reduces the effort required to select an appropriate frame and allows for the appropriate detection of lesions. Moreover, it enables the simultaneous detection of defects and creation of findings in multiple ultrasound diagnostic devices 1. In the fourth embodiment, in the medical information processing device 300, the acquisition function 341 acquires echo data and imaging data transmitted by the ultrasound diagnostic device 1, and the image generation function 342 generates and acquires contrast-enhanced images based on the echo data and imaging data. However, the acquisition function 341 may also generate and acquire contrast-enhanced images based on data provided by modalities other than the ultrasound diagnostic device 1.

[0092] In the first embodiment described above, the medical image processing device 100 detects or generates data used for detecting defects. Alternatively, for example, an operator may operate the input interface 120, and the acquisition function 141 may acquire correction instruction information for the data used for detecting defects output by the input interface 120, and the data used for detecting defects may be corrected based on the acquired information. Furthermore, the elements of the first to fourth embodiments may be combined or rearranged as appropriate.

[0093] According to at least one embodiment described above, by having an acquisition unit that acquires contrast-enhanced images of a subject to which a contrast agent has been administered during the process from the arterial-dominant phase through the portal vein-dominant phase to the retrovascular phase, at least from the portal vein-dominant phase onward, and a detection unit that detects areas in the contrast-enhanced images from the portal vein-dominant phase onward where the contrast agent has been washed out as missing areas, the effort required to select an appropriate frame is reduced, and lesions can be appropriately detected.

[0094] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0095] 1… Ultrasound diagnostic equipment 10… Ultrasound probe 100... Medical image processing equipment 110...Communication Interface 120... Input Interface 130…Display 140… Processing circuit 141... Acquisition function 142…Image generation function 143…Display control function 144...Detection function 145…Time specific function 146... Boundary setting function 147... Function to determine the degree of data loss 148…Time curve generation function 149…Peak frame selection function 150... Findings creation function 180...memory 181... Organizational image data 182...Contrast-enhanced image data 183... Annotation information 184...First trained model 185...Second pre-trained model 186...Third trained model 300... Medical information processing device 310...Communication Interface 320…Input Interface 330…Display 340… Processing circuit 341... Acquisition function 342…Image generation function 343…Display control function 344...Detection function 345…Time specific function 346... Boundary setting function 347... Function to determine the degree of data loss 348…Time curve generation function 349... Peak frame selection function 350... Report creation function 380...memory 381... Organizational image data 382...Contrast-enhanced image data

Claims

1. An acquisition unit that acquires contrast-enhanced images of a subject to which a contrast agent has been administered during the process from the arterial-dominant phase through the portal vein-dominant phase to the retrovascular phase, at least from the portal vein-dominant phase onward. The system includes a detection unit that detects areas where the contrast agent has been washed out in the contrast-enhanced image after the portal vein-dominant phase as defective areas. Medical image processing equipment.

2. A time identification unit identifies the time at which the frame of the contrast-enhanced image in which the defect was detected was captured, among a plurality of contrast-enhanced images. A boundary setting unit that sets the boundary between the missing portion and the non-missing portion, A defect degree determination unit for determining the degree of defect in the defective portion, A time curve generation unit generates a time intensity curve showing the time change of staining in the defective area, A peak frame selection unit selects a peak frame in which the contrast agent staining in the defect reaches its peak. The system further comprises a findings generation unit that generates findings of contrast agent staining in the peak frame. The medical image processing apparatus according to claim 1.

3. The detection unit detects the missing portion based on the output result obtained by inputting the contrast-enhanced image of the subject into a first trained model, which is generated by learning a contrast-enhanced image including the missing portion as training data. The medical image processing apparatus according to claim 1 or 2.

4. The portal vein dominant phase is defined based on time information measured by a timer or annotation information set for the examination process. A medical image processing apparatus according to any one of claims 1 to 3.

5. When the detection unit detects multiple missing parts, the acquisition unit further acquires the designation information of the missing parts specified by the operator. A medical image processing apparatus according to any one of claims 1 to 4.

6. The acquisition unit further acquires tissue images of the subject, which are captured together with the contrast-enhanced image. The detection unit searches for a pattern of tissue images at a location corresponding to the defect in the contrast-enhanced image from among the tissue images in the arterial-dominant phase, and detects the most matching region as the defect in the arterial-dominant phase contrast-enhanced image. A medical image processing apparatus according to any one of claims 1 to 5.

7. The acquisition unit further acquires tissue images of the subject, which are captured together with the contrast-enhanced image. If no defect is detected in the contrast-enhanced image in the portal vein-dominant phase, the detection unit detects the lesion in the tissue image and selects the frame with the highest likelihood. A time curve generation unit generates a time intensity curve by performing motion correction in the reverse time direction, starting from the frame with the highest likelihood, A peak frame selection unit selects a peak frame in which the contrast agent staining peaks in the area corresponding to the lesion. The system further comprises a findings generation unit that generates findings of contrast agent staining in the peak frame. The medical image processing apparatus according to claim 1.

8. The detection unit identifies the lesion based on the output result obtained by inputting the tissue image into a second trained model, which is generated using the tissue image including the defect as training data. The medical image processing apparatus according to claim 7.

9. The acquisition unit further acquires the contrast-enhanced image of the subject in the arterial-dominant phase, The detection unit detects the lesion from the arterial-dominant phase contrast image. A medical image processing apparatus according to any one of claims 1 to 8.

10. The detection unit detects the lesion based on the output result obtained by inputting the arterial-dominant phase contrast image, which includes the lesion, into a third trained model generated using the arterial-dominant phase contrast image as training data. The medical image processing apparatus according to claim 9.

11. The acquisition unit acquires correction instruction information for the data used to detect the missing portion. A medical image processing apparatus according to any one of claims 1 to 10.

12. The acquisition unit acquires tissue images of the subject taken before administering the contrast agent to the subject and contrast-enhanced images of the subject after administering the contrast agent. The detection unit detects the defect areas where the contrast agent has been washed out in the tissue image of the subject, as well as in the contrast-enhanced image in the arterial phase and the contrast-enhanced image in the portal vein phase and beyond. The medical image processing apparatus according to claim 1.

13. A medical image processing apparatus according to any one of claims 1 to 12, and an ultrasound probe that transmits ultrasound and receives echoes of transmitted ultrasound, The acquisition unit acquires the contrast-enhanced image generated based on the ultrasound echoes received by the ultrasound probe. Ultrasound diagnostic equipment.

14. The acquisition unit acquires the contrast-enhanced image provided by a modality connected via a network. A medical image processing apparatus according to any one of claims 1 to 12.

15. On the computer, Among the contrast-enhanced images of a subject who has been administered contrast agent during the process from the arterial-dominant phase through the portal vein-dominant phase to the retrovascular phase, at least the contrast-enhanced images of the subject from the portal vein-dominant phase onward are obtained. In contrast-enhanced images after the portal vein-dominant phase, areas where the contrast agent has been washed out are detected as defects. program.

Images