Medical image processing equipment

Abstract

To enable definite diagnosis for a tumor without depending on an experience or a subjective view of an operator such as a doctor.SOLUTION: A medical image processing device includes an image acquisition unit and a processing unit. The image acquisition unit acquires subject contrast image data with the classification into mutually different time phase zones generated by imaging the subject. The processing unit acquires information for identifying a type of a lesioned part related to the subject by inputting the subject contrast image data with the classification into mutually different time phase zones to a model that has learned for identifying a type of a lesioned part by the input of the contrast image data with the classification into mutually different time phase zones.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] An embodiment of the present invention relates to a medical image processing apparatus. [Background technology]

[0002] In the medical field, ultrasonic diagnostic devices are used to image the inside of a subject using ultrasonic waves generated by multiple transducers (piezoelectric transducers) of an ultrasonic probe. The ultrasonic diagnostic device transmits ultrasonic waves into the subject from an ultrasonic probe connected to the ultrasonic diagnostic device, generates echo signals based on the reflected waves, and obtains a desired ultrasonic image by image processing.

[0003] In contrast examinations using ultrasound diagnostic devices, after the injection of a contrast agent, the tumor is classified into a number of time phases according to the degree of contrast staining by the contrast agent. For example, when the imaging site is the liver, the time phases are classified into early vascular phase, arterial dominant phase, portal vein dominant phase, and late vascular phase according to the elapsed time from the start of the injection of the contrast agent. A definitive diagnosis of a tumor is performed in real time or by post-processing using stored images using contrast image data obtained in these time phases or multiple contrast image data obtained in different time phases. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2011 / 0282194 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to enable a definitive diagnosis of a tumor that is not dependent on the experience or subjectivity of an operator such as a doctor. [Means for solving the problem]

[0006] A medical image processing apparatus according to an embodiment includes an image acquisition unit and a processing unit. The image acquisition unit acquires subject contrast image data having different time phase classifications, which is generated by imaging a subject. The processing unit acquires information for identifying the type of lesion site in the subject by inputting the subject contrast image data having different time phase classifications to a trained model that identifies the type of lesion site by inputting the contrast image data having different time phase classifications. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of an ultrasonic diagnostic apparatus equipped with a medical image processing apparatus according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing the functions of an ultrasound diagnostic apparatus including a medical image processing apparatus according to an embodiment. [Diagram 3] FIG. 3 is a flowchart showing the operation of an ultrasound diagnostic apparatus including a medical image processing apparatus according to an embodiment. [Figure 4] FIG. 4(A) is an explanatory diagram showing an example of data flow during learning in a medical image processing apparatus according to an embodiment, and FIG. 4(B) is a diagram showing an example of training data in a table in the medical image processing apparatus according to an embodiment. [Diagram 5] FIG. 5 is an explanatory diagram showing an example of a data flow during operation in the medical image processing apparatus according to the embodiment. [Figure 6] FIG. 6 is a diagram showing a concept of associating contrast image data with moving image data in the medical image processing apparatus according to the embodiment. [Figure 7] FIG. 7(A) is an explanatory diagram showing an example of data flow during learning in the case of a first time phase band in a medical image processing apparatus according to an embodiment, and FIG. 7(B) is an explanatory diagram showing an example of data flow during learning in the case of a second time phase band in a medical image processing apparatus according to an embodiment. [Figure 8]FIG. 8(A) is an explanatory diagram showing an example of data flow during operation of a medical image processing apparatus according to an embodiment in a first time phase band, and FIG. 8(B) is an explanatory diagram showing an example of data flow during operation of a medical image processing apparatus according to an embodiment in a second time phase band. [Figure 9] FIG. 9(A) is an explanatory diagram showing an example of data flow during learning in a medical image processing apparatus according to an embodiment, and FIG. 9(B) is a diagram showing an example of training data in a table in the medical image processing apparatus according to an embodiment. [Figure 10] FIG. 10 is a block diagram showing the functions of an ultrasound diagnostic apparatus including a medical image processing apparatus according to an embodiment. [Figure 11] FIG. 11 is a flowchart showing the operation of an ultrasonic diagnostic apparatus including a medical image processing apparatus according to an embodiment. [Figure 12] FIG. 12(A) is an explanatory diagram showing an example of data flow during learning in a medical image processing apparatus according to an embodiment, and FIG. 12(B) is a diagram showing an example of training data in a table in the medical image processing apparatus according to an embodiment. [Figure 13] FIG. 13 is an explanatory diagram showing an example of a data flow during operation in the medical image processing apparatus according to the embodiment. [Figure 14] FIG. 14 is a view showing an example of display of a diagnosis result in the medical image processing apparatus according to the embodiment. [Figure 15] FIG. 15(A) is an explanatory diagram showing an example of data flow during learning in a medical image processing apparatus according to an embodiment when combining multiple time phase bands corresponding to a first time phase band or earlier, and FIG. 15(B) is an explanatory diagram showing an example of data flow during learning in a medical image processing apparatus according to an embodiment when combining multiple time phase bands corresponding to a second time phase band or earlier. [Figure 16]FIG. 16(A) is an explanatory diagram showing an example of data flow during operation when combining multiple time phase bands corresponding to a first time phase band or earlier in a medical image processing apparatus according to an embodiment, and FIG. 16(B) is an explanatory diagram showing an example of data flow during operation when combining multiple time phase bands corresponding to a second time phase band or earlier in a medical image processing apparatus according to an embodiment. [Figure 17] 17A and 17B are diagrams showing an example of a time phase setting screen in the medical image processing apparatus according to the embodiment. [Figure 18] FIG. 18 is a view showing an example of display of definitive diagnosis result data in the medical image processing apparatus according to the embodiment. [Figure 19] FIG. 19 is a schematic diagram showing the configuration of a medical image display device including a medical image processing device according to an embodiment. [Figure 20] FIG. 20 is a block diagram showing functions of a medical image display device including a medical image processing device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of a medical image processing apparatus will be described in detail with reference to the drawings.

[0009] The medical image processing device according to the embodiment is provided in a medical image generating device (also called a "modality") capable of acquiring contrast image data, such as an ultrasound diagnostic device, an X-ray CT (Computed Tomography) device, or an MRI (Magnetic Resonance Imaging) device. The medical image processing device according to the embodiment is provided in a medical image display device, such as a workstation, capable of acquiring contrast image data from a medical image generating device (or a medical image management device) via a network. Alternatively, the medical image processing device according to the embodiment is provided in an offline medical image display device capable of acquiring contrast image data from a medical image generating device (or a medical image management device) via a portable recording medium.

[0010] When the medical image processing device is provided in an ultrasound diagnostic device serving as a medical image generating device, a time band after contrast agent injection can be classified into a plurality of time bands (e.g., an early vascular phase, an arterial dominant phase, a portal vein dominant phase, a posterior vascular phase, etc.) by a contrast scan of the liver. When the medical image processing device is provided in an X-ray CT device serving as a medical image generating device, a time band after contrast agent injection can be classified into a plurality of time bands (e.g., an arterial dominant phase, a portal vein dominant phase, a equilibrium phase, etc.) by a contrast scan of the liver. When the medical image processing device is provided in an MRI device serving as a medical image generating device, a time band after contrast agent injection can be classified into a plurality of time bands (e.g., a dynamic phase, a hepatocyte contrast phase, etc.) by a contrast scan of the liver.

[0011] In the following, the first to eighth embodiments will be described taking as an example a case where a medical image processing apparatus is provided in an ultrasonic diagnostic apparatus, and the ninth embodiment will be described taking as an example a case where a medical image processing apparatus is provided in a medical image display device.

[0012] In addition, a case where the imaging site is the liver will be described. However, the imaging site is not limited to this case. For example, the imaging site may be tissue such as the breast or pancreas, which can identify a specific staining state of a lesion site by a contrast agent.

[0013] 1. Medical image processing apparatus according to the first embodiment FIG. 1 is a schematic diagram showing the configuration of an ultrasonic diagnostic apparatus including a medical image processing apparatus according to the first embodiment.

[0014] FIG. 1 shows an ultrasound diagnostic device 10 equipped with a medical image processing device according to the first embodiment. FIG. 1 also shows an ultrasound probe 20, an input interface 30, and a display 40. Note that an apparatus in which at least one of the ultrasound probe 20, the input interface 30, and the display 40 is added to the ultrasound diagnostic device 10 may be referred to as an ultrasound diagnostic device. In the following description, a case will be described in which the ultrasound probe 20, the input interface 30, and the display 40 are all provided outside the ultrasound diagnostic device.

[0015] The ultrasound diagnostic device 10 includes a transmission / reception circuit 11, a B-mode processing circuit 12, a Doppler processing circuit 13, an image generating circuit 14, an image memory 15, a network interface 16, a processing circuit 17, and a main memory 18. The circuits 11 to 14 are configured by an application specific integrated circuit (ASIC) or the like. However, the present invention is not limited to this case, and all or part of the functions of the circuits 11 to 14 may be realized by the processing circuit 17 executing a program. The processing circuit 17 and the main memory 18 configure the medical image processing device M1 according to the first embodiment. The medical image processing device M1 according to the first embodiment may be configured by adding at least one of an input interface 30 and a display 40 to the processing circuit 17 and the main memory 18.

[0016] The transmission / reception circuit 11 has a transmission circuit and a reception circuit (not shown). The transmission / reception circuit 11 controls the transmission directivity and reception directivity in transmitting and receiving ultrasonic waves under the control of the processing circuit 17. Although the case where the transmission / reception circuit 11 is provided in the ultrasonic diagnostic device 10 will be described, the transmission / reception circuit 11 may be provided in the ultrasonic probe 20, or may be provided in both the ultrasonic diagnostic device 10 and the ultrasonic probe 20. The transmission / reception circuit 11 is an example of a transmission / reception unit.

[0017] The transmission circuit has a pulse generation circuit, a transmission delay circuit, a pulsar circuit, etc., and supplies a drive signal to the ultrasonic transducer. The pulse generation circuit repeatedly generates rate pulses for forming a transmission ultrasonic wave at a predetermined rate frequency. The transmission delay circuit provides each rate pulse generated by the pulse generation circuit with a delay time for each piezoelectric transducer required to focus the ultrasonic waves generated from the ultrasonic transducer of the ultrasonic probe 20 into a beam and determine the transmission directivity. In addition, the pulsar circuit applies a drive pulse to the ultrasonic transducer at a timing based on the rate pulse. The transmission delay circuit arbitrarily adjusts the transmission direction of the ultrasonic beam transmitted from the piezoelectric transducer surface by changing the delay time provided to each rate pulse.

[0018] The receiving circuit has an amplifier circuit, an A / D (Analog to Digital) converter, an adder, etc., and receives the echo signal received by the ultrasonic transducer and performs various processes on this echo signal to generate echo data. The amplifier circuit amplifies the echo signal for each channel and performs gain correction processing. The A / D converter A / D converts the gain-corrected echo signal and gives the digital data a delay time required to determine the reception directivity. The adder performs addition processing of the echo signal processed by the A / D converter to generate echo data. The addition processing by the adder emphasizes the reflection component from the direction corresponding to the reception directivity of the echo signal.

[0019] The B-mode processing circuit 12 receives echo data from the receiving circuit under the control of the processing circuit 17, and performs logarithmic amplification, envelope detection processing, and the like to generate data (2D or 3D data) in which the signal intensity is expressed by the brightness of luminance. This data is generally called B-mode data. The B-mode processing circuit 12 is an example of a B-mode processing unit.

[0020] The B-mode processing circuit 12 can change the detection frequency by filtering to change the frequency band to be imaged. By using the filtering function of the B-mode processing circuit 12, it is possible to perform harmonic imaging such as contrast harmonic imaging (CHI) and tissue harmonic imaging (THI). That is, the B-mode processing circuit 12 can separate reflected wave data (harmonic data or sub-frequency data) of harmonic components whose reflection source is the contrast agent (microbubbles, bubbles) from reflected wave data of a subject into which a contrast agent has been injected, and reflected wave data (fundamental wave data) of fundamental components whose reflection source is tissue in the subject. The B-mode processing circuit 12 can also generate B-mode data for generating contrast image data from reflected wave data (received signal) of harmonic components, and can also generate B-mode data for generating fundamental image data from reflected wave data (received signal) of fundamental components.

[0021] The Doppler processing circuit 13, under the control of the processing circuit 17, performs frequency analysis of velocity information from echo data from the receiving circuit, and generates data (2D or 3D data) in which moving object movement information such as average velocity, variance, and power is extracted for multiple points. This data is generally called Doppler data. Here, the moving object is, for example, blood flow, tissue such as the heart wall, and contrast agent. The Doppler processing circuit 13 is an example of a Doppler processing unit.

[0022] Under the control of the processing circuitry 17, the image generating circuitry 14 generates ultrasonic image data expressed in a predetermined brightness range based on the echo signal received by the ultrasonic probe 20. For example, the image generating circuitry 14 generates, as ultrasonic image data, 2D B-mode image data in which the intensity of the reflected wave is expressed as brightness from the 2D B-mode data generated by the B-mode processing circuitry 12. The image generating circuitry 14 also generates, as ultrasonic images, average velocity images, variance images, power images, or 2D color Doppler image data as a combination image thereof, from the 2D Doppler data generated by the Doppler processing circuitry 13. Hereinafter, 2D ultrasonic image data such as 2D B-mode image data and 2D color Doppler image data will be simply referred to as "2D image data".

[0023] Here, the image generating circuit 14 generally converts (scan converts) a scan line signal sequence of ultrasonic scanning into a scan line signal sequence of a video format represented by a television or the like, and generates image data for display. Specifically, the image generating circuit 14 generates image data for display by performing coordinate conversion according to the ultrasonic scanning form of the ultrasonic probe 20. In addition to scan conversion, the image generating circuit 14 also performs various image processing, such as image processing (smoothing processing) for regenerating an average brightness image using multiple image frames after scan conversion, and image processing (edge ​​enhancement processing) using a differential filter within the image. The image generating circuit 14 also combines character information of various parameters, scales, body marks, etc. with the ultrasonic image data.

[0024] That is, the B-mode data and the Doppler data are image data before the scan conversion process, and the data generated by the image generating circuit 14 is image data for display after the scan conversion process. Note that the B-mode data and the Doppler data are also called raw data. The image generating circuit 14 generates 2D image data for display from the 2D image data before the scan conversion process.

[0025] Furthermore, the image generating circuitry 14 generates 3D B-mode image data by performing coordinate conversion on the 3D B-mode data generated by the B-mode processing circuitry 12. The image generating circuitry 14 also generates 3D Doppler image data by performing coordinate conversion on the 3D Doppler data generated by the Doppler processing circuitry 13. The image generating circuitry 14 generates the 3D B-mode image data and the 3D Doppler image data as 3D ultrasound image data (volume data).

[0026] Furthermore, the image generation circuit 14 performs rendering processing on the 3D image data to generate various 2D image data for displaying the 3D image data on the display 40. As the rendering processing, the image generation circuit 14 performs, for example, volume rendering (VR) processing to generate 2D image data reflecting 3D information. The image generation circuit 14 also performs processing to generate MPR image data from the 3D image data by performing multi-planar reconstruction (MPR). The image generation circuit 14 is an example of an image generation unit.

[0027] The image memory 15 includes a two-dimensional memory that is a memory having a plurality of memory cells in two axial directions per frame, the memory having a number of such memory cells for a plurality of frames. The two-dimensional memory as the image memory 15 stores 2D image data relating to one frame or multiple frames generated by the image generating circuit 14 under the control of the processing circuit 17. The image memory 15 is an example of a storage unit.

[0028] Under the control of the processing circuit 17, the image generating circuit 14 performs 3D reconstruction by performing interpolation processing as necessary on the 2D image data arranged in the 2D memory serving as the image memory 15, thereby generating the 2D image data as 3D image data in the 3D memory serving as the image memory 15. A known technique is used as the interpolation processing method.

[0029] The image memory 15 may include a three-dimensional memory that is a memory having a plurality of memory cells in three axial directions (X-axis, Y-axis, and Z-axis directions). The three-dimensional memory as the image memory 15 stores the 2D image data generated by the image generating circuit 14 as 3D image data under the control of the processing circuit 17.

[0030] The network interface 16 implements various information communication protocols according to the form of the network. The network interface 16 connects the ultrasound diagnostic apparatus 10 to other devices such as an external medical image management apparatus 60 and a medical image display apparatus 70 in accordance with the various protocols. This connection can be an electrical connection via an electronic network. Here, the electronic network refers to a general information communication network that uses electrical communication technology, and includes not only a wireless / wired hospital-based LAN (Local Area Network) and the Internet network, but also a telephone communication line network, an optical fiber communication network, a cable communication network, and a satellite communication network.

[0031] The network interface 16 may also implement various protocols for non-contact wireless communication. In this case, the ultrasound diagnostic device 10 can directly transmit and receive data to and from, for example, the ultrasound probe 20 without going through a network. The network interface 16 is an example of a network connection unit.

[0032] The processing circuit 17 refers to a dedicated or general-purpose CPU (Central Processing Unit), MPU (Micro Processor Unit), or GPU (Graphics Processing Unit), as well as an ASIC and a programmable logic device, etc. Examples of the programmable logic device include a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA).

[0033] Furthermore, the processing circuit 17 may be formed by a single circuit, or may be formed by a combination of multiple independent circuit elements. In the latter case, the main memory 18 may be provided separately for each circuit element, or a single main memory 18 may store programs corresponding to the functions of multiple circuit elements.

[0034] The main memory 18 is composed of a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, etc. The main memory 18 may be composed of a portable medium such as a USB (Universal Serial Bus) memory or a DVD (Digital Video Disk). The main memory 18 stores various processing programs (including application programs, an OS (Operating System), etc.) used in the processing circuit 17 and data required for executing the programs. The OS may also include a GUI (Graphical User Interface) that uses a lot of graphics to display information to the operator on the display 40 and allows basic operations to be performed by the input interface 30. The main memory 18 is an example of a storage unit.

[0035] The ultrasonic probe 20 has a number of tiny transducers (piezoelectric elements) on the front surface, and transmits and receives ultrasonic waves to and from an area including the imaging site. Each transducer is an electroacoustic conversion element, and has the function of converting an electric pulse into an ultrasonic pulse during transmission, and converting a reflected wave into an electric signal (received signal) during reception. The ultrasonic probe 20 is small and lightweight, and is connected to the ultrasonic diagnostic device 10 via a cable (or wireless communication).

[0036] The ultrasonic probe 20 is divided into types such as linear type, convex type, and sector type depending on the difference in the scanning method. Also, the ultrasonic probe 20 is divided into types such as 1D array probes in which a plurality of transducers are arranged one-dimensionally (1D) in the azimuth direction and 2D array probes in which a plurality of transducers are arranged two-dimensionally (2D) in the azimuth direction and the elevation direction depending on the difference in the array arrangement dimension. The 1D array probe includes a probe in which a small number of transducers are arranged in the elevation direction.

[0037] Here, when imaging such as a panoramic scan is performed, a 2D array probe equipped with a scanning method such as a linear type, a convex type, or a sector type is used as the ultrasonic probe 20. Alternatively, when imaging is performed, a 1D probe equipped with a scanning method such as a linear type, a convex type, or a sector type and equipped with a mechanism for mechanically swinging in the elevation direction is used as the ultrasonic probe 20. The latter probe is also called a mechanical 4D probe.

[0038] The input interface 30 includes an input device that can be operated by an operator, and an input circuit that inputs a signal from the input device. The input device can be realized by a trackball, a switch, a mouse, a keyboard, a touchpad that performs an input operation by touching the operation surface, a touchscreen in which a display screen and a touchpad are integrated, a non-contact input device using an optical sensor, a voice input device, etc. When the input device is operated by the operator, the input circuit generates a signal according to the operation and outputs it to the processing circuit 17. The input interface 30 is an example of an input unit.

[0039] The display 40 is configured by a general display output device such as a liquid crystal display, an OLED (Organic Light Emitting Diode) display, etc. The display 40 displays various information under the control of the processing circuit 17. The display 40 is an example of a display unit.

[0040] 1 also shows a medical image management device 60 and a medical image display device 70, which are external devices of the ultrasound diagnostic apparatus 10. The medical image management device 60 is, for example, a Digital Imaging and Communications in Medicine (DICOM) server, and is connected to devices such as the ultrasound diagnostic apparatus 10 so as to be able to transmit and receive data via a network N. The medical image management device 60 manages medical images such as ultrasound images generated by the ultrasound diagnostic apparatus 10 as DICOM files.

[0041] The medical image display device 70 is connected to devices such as the ultrasound diagnostic device 10 and the medical image management device 60 so as to be able to transmit and receive data via the network N. Examples of the medical image display device 70 include a workstation that performs various image processing on the ultrasound images generated by the ultrasound diagnostic device 10, and a portable information processing terminal such as a tablet terminal. Note that the medical image display device 70 may be an offline device that can read out the ultrasound images generated by the ultrasound diagnostic device 10 via a portable storage medium.

[0042] Next, the functions of the ultrasonic diagnostic apparatus 10 equipped with the medical image processing apparatus M1 will be described. FIG. 2 is a block diagram showing the functions of an ultrasonic diagnostic apparatus 10 including a medical image processing apparatus M1.

[0043] The processing circuitry 17 realizes an image acquisition function 171, a necessity information generation function 172, and a storage control function 173 by reading and executing a program stored in the main memory 18 or directly incorporated in the processing circuitry 17. The following description will be given taking as an example a case where the functions 171 to 173 function as software, but all or a part of the functions 171 to 173 may be provided in the ultrasound diagnostic device 10 as a circuit such as an ASIC.

[0044] The image acquisition function 171 includes a function of controlling the transmission / reception circuit 11, the B-mode processing circuit 12, the Doppler processing circuit 13, the image generation circuit 14, etc. to execute ultrasonic imaging using the ultrasonic probe 20 and acquire a plurality of ultrasonic image data (for example, contrast image data) in time series. The image acquisition function 171 is an example of an image acquisition unit.

[0045] Here, it is also possible to link each contrast image data with information on time bands classified according to the staining state of the lesion site by the contrast agent contained in the contrast image data (hereinafter referred to as "time bands" or "time band information"). This is because time band information is generally classified according to the elapsed time based on the start of the injection of the contrast agent. Here, the time band information means the early vascular phase, the arterial dominant phase, the portal vein dominant phase, the posterior vascular phase, etc.

[0046] The necessity information generating function 172 includes a function of generating necessity information for storing contrast image data of a subject by inputting the contrast image data of the subject to a trained model for generating information (necessity information) regarding necessity for storage based on contrast image data required for a definitive diagnosis of a tumor among a plurality of contrast image data related to each time phase. Note that the necessity information generating function 172 is an example of a processing unit.

[0047] The storage control function 173 includes a function of controlling storage of the subject's contrast image data in the main memory 18 in accordance with the necessity information generated by the necessity information generating function 172. In addition, the storage control function 173 preferably has a function of storing, in the main memory 18, contrast image data determined to need to be stored among a plurality of frames of contrast image data of the subject generated by ultrasonic imaging in association with moving image data consisting of the plurality of frames of contrast image data. In this case, an operator such as a doctor who refers to the moving image data on the ultrasound diagnostic apparatus 10 or the medical image display device 70 can easily refer to the contrast image data of the extracted frames. The storage control function 173 is an example of a storage control unit.

[0048] The functions 171 to 173 will be described in detail with reference to FIGS.

[0049] 3 is a flowchart showing the operation of the ultrasound diagnostic device 10 equipped with the medical image processing device M1. In FIG. 3, the reference characters "ST" followed by numbers indicate each step of the flowchart.

[0050] First, the image acquisition function 171 starts injection of a contrast agent (step ST1), and controls the transmission / reception circuit 11, the B-mode processing circuit 12, the Doppler processing circuit 13, the image generation circuit 14, etc. to start ultrasonic imaging (for example, CHI (Contrast Harmonic Imaging)) using the ultrasonic probe 20 (step ST2). The image acquisition function 171 controls the image generation circuit 14, etc. to generate contrast image data and store it in the image memory 15 (step ST3). The image acquisition function 171 also causes the contrast image data generated in step ST3 to be live displayed on the display 40 (step ST4).

[0051] The necessity information generating function 172 generates necessity information regarding storage of the contrast image data based on the contrast image data of the latest frame generated in step ST3 (step ST5). Here, it is also possible to link each contrast image data with "phase band information".

[0052] Here, the necessity information generating function 172 performs a process of generating necessity information regarding the storage of the contrast image data based on the contrast image data. For example, a lookup table (LUT) that associates the contrast image data with necessity information regarding storage may be used for this process. Machine learning may also be used for this process. As the machine learning, deep learning using a multi-layer neural network such as a CNN (Convolutional Neural Network) or a Convolutional Deep Belief Network (CDBN) may also be used.

[0053] In the description of the first embodiment, an example will be given in which the necessity information generating function 172 includes a neural network N1, and generates necessity information regarding the storage of contrast image data based on the contrast image data by using deep learning.

[0054] FIG. 4(A) is an explanatory diagram showing an example of data flow during learning. FIG. 4(B) is a diagram showing an example of training data in a table. In FIG. 4(B), each contrast image data is linked to time-phase band information classified according to the staining state of the lesion site by the contrast agent contained in the contrast image data. When the necessity information generating function 172 generates necessity information in approximately real time (or live) for a plurality of contrast image data sequentially acquired by the image acquiring function 171, the time-phase band information can be linked to each contrast image data. Note that "approximately real time" includes a case where an ultrasound image is generated (or displayed) simultaneously with the transmission of ultrasound, as well as a case where there is a time lag between the transmission of ultrasound and the generation (or display) of an ultrasound image, which corresponds to the processing time from the transmission of ultrasound to the generation (or display) of the ultrasound image.

[0055] The necessity information generating function 172 sequentially updates the parameter data P1 by learning from a large amount of input training data. The training data is composed of a combination of contrast image data S1, S2, S3, ... as training input data, and necessity information T1, T2, T3, .... The contrast image data S1, S2, S3, ... constitute a training input data group S. The necessity information T1, T2, T3, ... constitute a training output data group T.

[0056] The necessity information generating function 172 performs so-called learning, updating the parameter data P1 each time training data is input so that the result of processing the contrast image data S1, S2, S3, ... by the neural network N1 approaches the necessity information T1, T2, T3, .... In general, when the rate of change of the parameter data P1 converges within a threshold, learning is determined to be completed. Hereinafter, the parameter data P1 after learning is particularly referred to as learned parameter data P1' (shown in FIG. 5).

[0057] Note that the type of training input data should match the type of input data during operation shown in Fig. 5. For example, if the input data during operation is contrast image data of a subject, the training input data group S during learning is also contrast image data.

[0058] Furthermore, the "image data" includes raw data generated by a medical image generating device such as an ultrasonic diagnostic device, etc. In other words, the input data of the neural network N1 may be raw data before scan conversion.

[0059] FIG. 5 is an explanatory diagram showing an example of data flow during operation.

[0060] During operation, the necessity information generation function 172 inputs contrast image data Sa of a subject to be diagnosed to the learned model D1, and outputs necessity information Ta regarding the storage of the subject's contrast image data using the learned parameter data P1'.

[0061] The neural network N1 and the trained parameter data P1' configure a trained model D1. The neural network N1 is stored in the main memory 18 in the form of a program. The trained parameter data P1' may be stored in the main memory 18, or in a storage medium connected to the ultrasound diagnostic device 10 via the network N. In this case, the necessity information generating function 172 realized by the processor of the processing circuitry 17 reads out the trained model D1 from the main memory 18 and executes it to generate necessity information regarding storage based on the contrast image data. The trained model D1 may be constructed by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0062] In addition, the necessity information Ta regarding the storage of contrast image data of a subject output by the trained model D1 including the necessity information generation function 172 may be data indicating, for example, either "1" indicating necessity or "0" indicating no necessity.

[0063] In order to improve the accuracy of judgment by the necessity information generation function 172, in addition to the contrast image data, identification information including at least one of the height, weight, medical history, and medical history of blood relatives of the subject may be used as input data.

[0064] In this case, during learning, the identification information of the subject of each of the contrast image data S1, S2, S3, ... as training input data is also input to the neural network N1 as training input data. During operation, the necessity information generation function 172 outputs necessity information Ta regarding the storage of the subject's contrast image data by inputting the identification information of the subject together with the contrast image data Sa of the subject to be diagnosed to the trained model D1 read from the main memory 18. By using the contrast image data and the identification information of the subject as input data, trained parameter data P1' that has been trained according to the type of subject can be generated, thereby improving the diagnostic accuracy compared to the case where only the contrast image data is used as input data.

[0065] Returning to the explanation of FIG. 3, the storage control function 173 judges whether or not it is necessary to store the contrast image data generated in step ST3 according to the necessity information regarding storage generated in step ST5 (step ST6). If the judgment in step ST6 is YES, that is, if it is judged that it is necessary to store the contrast image data generated in step ST3, the storage control function 173 stores the contrast image data generated in step ST3 in the main memory 18 (step ST7). The storage in step ST7 is performed separately from the storage in step ST3. It is preferable that the storage control function 173 has a function of storing the contrast image data, which is judged to be required to be stored, in the main memory 18 in association with moving image data consisting of a plurality of frames of contrast image data.

[0066] FIG. 6 is a diagram showing the concept of associating contrast image data with video image data.

[0067] 6, contrast image data of a subject that has been determined to need to be stored is stored in main memory 18 in association with moving image data consisting of a plurality of frames of contrast image data. It is preferable that at least one piece of contrast image data is determined to need to be stored in each time phase.

[0068] Returning to the explanation of Figure 3, after step ST7, and if the judgment in step ST6 is NO, that is, if it is judged that there is no need to store the contrast image data generated in step ST3, the image acquisition function 171 judges whether or not to terminate the ultrasound imaging started in step ST2 (step ST8).

[0069] If the determination in step ST8 is NO, that is, if it is determined that the ultrasonic imaging started in step ST2 is not to be ended, the image acquisition function 171 controls the image generation circuit 14 and the like to generate contrast image data for the next frame and store it in the image memory 15 (step ST3). That is, the ultrasonic imaging is continued.

[0070] On the other hand, if the judgment in step ST8 is YES, i.e., if it is judged that the ultrasound imaging started in step ST2 is to be terminated, the image acquisition function 171 judges that the operator has given an instruction to terminate via the input interface 30, and terminates the ultrasound imaging.

[0071] That is, according to the flowchart shown in FIG. 3, the ultrasonic diagnostic apparatus 10 repeats a set of steps ST3 to ST8 to generate storage necessity information for each frame and performs storage accordingly.

[0072] The contrast image data is not limited to one frame of image data. The contrast image data may be video data consisting of a plurality of consecutive frames. In this case, video data consisting of a plurality of contrast image data up to the present time is used as training input data, and video image data of a subject to be diagnosed is used as contrast image data Sa and input to the trained model D1. This makes it possible to learn the dynamic movement of bubbles contained in the contrast agent.

[0073] 3, the necessity information generating function 172 generates necessity information every time the contrast image data of each successive frame is generated, but is not limited to this case. For example, the necessity information generating function 172 may generate necessity information every time the contrast image data of each frame generated at a certain frame interval is generated, or may generate necessity information every time the first frame of contrast image data of each time phase is generated.

[0074] According to the medical image processing device M1, by storing contrast image data assumed to be necessary for diagnosis separately from the video image data of the entire frame, post-processing using the stored images enables a definitive diagnosis of a tumor that is not dependent on the experience or subjectivity of an operator such as a doctor, and also shortens the diagnosis time.

[0075] 2. Medical image processing apparatus according to the second embodiment In the above-described first embodiment, a single neural network is configured regardless of the time phase, that is, the necessity information generating function 172 generates necessity information regarding the storage of the contrast image data of the subject in each time phase by inputting the contrast image data of the subject in each time phase to one trained model. However, the present invention is not limited to this case.

[0076] For example, one neural network may be configured for each time phase, that is, the necessity information generating function 172 may generate necessity information regarding the storage of the contrast image data of a subject by inputting the contrast image data of a subject to a trained model corresponding to the time phase among a plurality of trained models corresponding to a plurality of time phases. This case will be described as a medical image processing device M2 according to a second embodiment (shown in FIG. 1 and FIG. 2).

[0077] The configuration and functions of the medical image processing apparatus M2 are the same as those shown in FIGS. 1 and 2, and the operation of the medical image processing apparatus M2 is the same as that shown in FIG. 3, so a description thereof will be omitted.

[0078] In step ST5 shown in FIG. 3, necessity information generating function 172 shown in FIG. 2 generates necessity information regarding storage of the contrast image data based on the contrast image data of the latest frame generated in step ST3.

[0079] Here, the necessity information generating function 172 performs a process of generating necessity information regarding preservation according to a time phase corresponding to the target frame. For example, a lookup table for each time phase that associates contrast image data with necessity information regarding preservation may be used for this process. Machine learning may also be used for this process. As the machine learning, deep learning using a multi-layered neural network such as CNN or convolutional deep belief network may also be used.

[0080] In the explanation of the second embodiment, an example is shown in which the necessity information generation function 172 includes a plurality of neural networks N2 (for example, “N21” and “N22” shown in (A) and (B) of FIG. 7)) and generates necessity information regarding preservation based on contrast image data using deep learning.

[0081] Fig. 7(A) is an explanatory diagram showing an example of data flow during learning in the case of a first time phase, and Fig. 7(B) is an explanatory diagram showing an example of data flow during learning in the case of a second time phase.

[0082] 7(A) will be described. The necessity information generating function 172 sequentially updates the parameter data P21 by learning by inputting a large number of contrast image data generated in the first time phase as training data. The training data is made up of a combination of contrast image data U1, U2, U3, ... generated in the first time phase as training input data and storage necessity information V1, V2, V3, .... The contrast image data U1, U2, U3, ... generated in the first time phase constitute a training input data group U. The storage necessity information V1, V2, V3, ... constitute a training output data group V.

[0083] The necessity information generating function 172 performs so-called learning, updating the parameter data P21 each time training data is input, so that the result of processing the contrast image data U1, U2, U3, ... generated in the first time phase band by the neural network N21 approaches the necessity information V1, V2, V3, ... regarding storage. In general, when the rate of change of the parameter data P21 converges within a threshold, learning is determined to be completed. Hereinafter, the parameter data P21 after learning is particularly referred to as learned parameter data P21' (shown in FIG. 8(A)).

[0084] Note that the type of training input data should match the type of input data during operation shown in Fig. 8(A). For example, when the input data during operation is contrast image data of a subject, the training input data group U during learning is also contrast image data.

[0085] Furthermore, the "image data" includes raw data generated by a medical image generating device such as an ultrasonic diagnostic device. In other words, the input data of the neural network N21 may be raw data before scan conversion.

[0086] On the other hand, Fig. 7(B) will be described. As in the case of Fig. 7(A), the necessity information generating function 172 sequentially updates the parameter data P22 by learning using a large number of input contrast image data generated in the second time phase as training data.

[0087] Fig. 8(A) is an explanatory diagram showing an example of data flow during operation in the case of a first time phase, and Fig. 8(B) is an explanatory diagram showing an example of data flow during operation in the case of a second time phase.

[0088] 8(A) will be described. During operation, the necessity information generating function 172 obtains a time phase band corresponding to a target frame (e.g., the latest frame) of a subject to be diagnosed. Then, when the necessity information generating function 172 determines that the time phase band is a first time phase band, it inputs contrast image data Ua to the trained model D21, and outputs necessity information Va regarding storage of the subject's contrast image data using trained parameter data P21'.

[0089] The neural network N21 and the trained parameter data P21' configure a trained model D21. The neural network N21 is stored in the main memory 18 in the form of a program. The trained parameter data P21' may be stored in the main memory 18, or in a storage medium connected to the ultrasound diagnostic device 10 via the network N. In this case, the necessity information generating function 172 realized by the processor of the processing circuitry 17 reads out the trained model D21 from the main memory 18 and executes it to generate necessity information regarding the storage of the contrast image data based on the contrast image data. The trained model D21 may be constructed by an integrated circuit such as an ASIC or an FPGA.

[0090] In addition, the necessity information Va regarding the storage of contrast image data of a subject output by the trained model D21 including the necessity information generation function 172 may be data indicating, for example, either "1" indicating necessity or "0" indicating no necessity.

[0091] In order to improve the accuracy of judgment by the necessity information generating function 172, in addition to the contrast image data, identification information including at least one of the height, weight, medical history, and medical history of blood relatives of the subject may be used as input data.

[0092] On the other hand, Fig. 8(B) will be described. As in the case of Fig. 8(A), during operation, the necessity information generating function 172 obtains a time phase band corresponding to a target frame (e.g., the latest frame) of a subject to be diagnosed. Then, when the necessity information generating function 172 determines that the time phase band is a second time phase band, it inputs contrast image data Wa to the trained model D22, and outputs necessity information Xa regarding the storage of the subject's contrast image data using trained parameter data P22'.

[0093] The contrast image data is not limited to one frame image data. The contrast image data may be video data for each time phase band, which is composed of a plurality of consecutive frames. In this case, the video data for the first time phase band is used as training input data for the first time phase band, and the video data Ua of the subject for the first time phase band is input to the trained model D21. The video data for the second time phase band is used as training input data for the second time phase band, and the video data Wa of the subject for the second time phase band is input to the trained model D22. This makes it possible to learn the dynamic movement of bubbles contained in the contrast agent.

[0094] The contrast image data may be one piece of video data consisting of a plurality of consecutive frames and divided into a plurality of time phase bands by indexes. In this case, partial data of the video data relating to a first time phase band divided by the index is used as training input data relating to the first time phase band, and partial data Ua of the video data of the subject relating to the first time phase band is input to the trained model D21.

[0095] According to the medical image processing device M2, by constructing and operating one trained model for each time phase, in addition to the effects of the medical image processing device M1, it is possible to obtain higher storage accuracy.

[0096] 3. Medical image processing apparatus according to the third embodiment In the above-mentioned first embodiment, one neural network is configured regardless of the time phase, and in the second embodiment, one neural network is configured for each time phase, but the present invention is not limited to this case.

[0097] For example, regardless of the tumor category classification, one neural network may be configured, in other words, the necessity information generation function 172 may generate necessity information regarding the storage of contrast image data of a subject related to each tumor category classification by inputting contrast image data of a subject related to each tumor category classification into one trained model.

[0098] Also, for example, one neural network may be configured for each tumor category classification, that is, the necessity information generating function 172 may generate necessity information regarding the storage of the subject's contrast image data by inputting the subject's contrast image data to a trained model corresponding to the category classification among a plurality of trained models corresponding to a plurality of tumor category classifications. This case will be described as a medical image processing device M3 according to a third embodiment (shown in FIG. 1 and FIG. 2).

[0099] The configuration and functions of the medical image processing device M3 are the same as those shown in FIGS. 1 and 2, and the operation of the medical image processing device M3 is the same as that shown in FIG. 3, so a description thereof will be omitted.

[0100] In this case, the necessity information generating function 172 generates necessity information regarding storage of contrast image data of a subject by inputting contrast image data of the subject related to the first category classification into a first trained model that generates necessity information regarding storage based on contrast image data related to a first category classification of tumors. Meanwhile, the necessity information generating function 172 generates necessity information regarding storage of contrast image data of a subject by inputting contrast image data of the subject related to the second category classification into a second trained model that generates necessity information regarding storage based on contrast image data related to a second category classification of tumors.

[0101] The information regarding the category classification of the tumor is, for example, HCC (hepatocellular carcinoma), Meta (metastatic liver tumor), Hema (hepatic hemangioma), and FNH (focal nodule).

[0102] According to the medical image processing device M3 of the third embodiment, by constructing and operating one trained model D3 for each tumor category classification, in addition to the effects of the medical image processing device M1, it is possible to obtain more accurate storage.

[0103] 4. First Modification In the medical image processing devices M1 to M3 according to the first to third embodiments described above, the training input data and the input to the trained model are contrast image data, or contrast image data and additional information such as identification information, but the present invention is not limited to these. At least one of B-mode image data, color Doppler image data, and power Doppler image data may be added to the training input data and the input to the trained model.

[0104] In addition, B-mode image data includes so-called B-mode image data generated by imaging separate from the contrast image data by CHI, and fundamental image data generated by the same scan as the contrast image, which is generated from reflected wave data (received signal) of the fundamental wave component.

[0105] According to the first modified example of the medical image processing apparatuses M1 to M3 according to the first to third embodiments, in addition to the effects of the medical image processing apparatuses M1 to M3, it is possible to obtain more accurate diagnosis results.

[0106] 5. Second Variation In the first embodiment described above, the training output data is storage necessity information, but the present invention is not limited to this case. In addition to the necessity information, at least one of time phase band information and definitive diagnosis result data may be added to the training output data. An example of time phase band information will be described in the fourth embodiment described later.

[0107] The definitive diagnosis result data may be, for example, data indicating whether the tumor is benign or malignant, data indicating which of the categories such as HCC, Meta, Hema, and FNH the tumor belongs to, or data indicating the possibility of the category classification in percentage (e.g., x% benign, y% malignant).Furthermore, the category classification may be a further subdivision of HCC, etc. into a differentiation degree (high differentiation, low differentiation, etc.).

[0108] According to the medical image processing devices M1 to M3 (including the modified examples), a definitive diagnosis of a tumor can be made independent of the experience or subjectivity of an operator such as a doctor, and the diagnosis time can be shortened.

[0109] 6. Medical image processing apparatus according to the fourth embodiment In the above first to third embodiments, the image acquiring function 171 acquires contrast image data live, and the necessity information generating function 172 generates necessity information for each contrast image data in approximately real time. That is, this is a case where the time phase band information of each contrast image data is known. Here, a case where the time phase band information of each contrast image data is unknown will be described as the medical image processing device M4 (shown in FIGS. 1 and 2) according to the fourth embodiment. When the time phase band information of each contrast image data is unknown, the image acquiring function 171 acquires multiple contrast image data from the image memory 15 after imaging is completed, and the necessity information generating function 172 generates necessity information in a post-process based on the multiple contrast image data, and this may also include a case where the necessity information generating function 172 generates necessity information in approximately real time.

[0110] The necessity information generation function 172 generates time phase band information by inputting the contrast image data acquired by the image acquisition function 171 into a trained model for generating time phase band information classified according to the staining state of the contrast agent in the lesion area contained in the contrast image data.

[0111] FIG. 9A is an explanatory diagram showing an example of data flow during operation.

[0112] 9(A) will be described. The necessity information generating function 172 sequentially updates the parameter data P4 by learning by inputting a large number of contrast image data relating to predetermined frames as training data. The training data is made up of a combination of contrast image data S1, S2, S3, ... of each frame as training input data and time band information Y1, Y2, Y3, .... The contrast image data S1, S2, S3, ... constitute a training input data group S. The time band information Y1, Y2, Y3, ... constitute a training output data group Y.

[0113] The necessity information generating function 172 performs so-called learning, updating the parameter data P4 each time training data is input, so that the result of processing the contrast image data S1, S2, S3, ... by the neural network N4 approaches the time-phase band information Y1, Y2, Y3, ... related to storage. In general, when the rate of change of the parameter data P4 converges within a threshold, learning is determined to be completed. Hereinafter, the parameter data P4 after learning is specifically referred to as learned parameter data P4'.

[0114] Note that the type of training input data should match the type of input data during operation shown in Fig. 9(B). For example, if the input data during operation is contrast image data of a subject, the training input data group S during learning is also contrast image data.

[0115] In addition, the "image data" includes raw data generated by a medical image generating device such as an ultrasonic diagnostic device. In other words, the input data of the neural network N4 may be raw data before scan conversion.

[0116] 9B will be described. As in the case of FIG. 9A, the necessity information generating function 172 sequentially updates the parameter data P4 by learning from a large number of input contrast image data as training data.

[0117] FIG. 9B is an explanatory diagram showing an example of data flow during operation in the first time phase.

[0118] 9(B) will be described. During operation, the necessity information generating function 172 inputs contrast image data Sa corresponding to a target frame of a subject to be diagnosed, and outputs time phase band information Ya related to the storage of the contrast image data of the subject by using the learned parameter data P4'.

[0119] The neural network N4 and the trained parameter data P4' configure a trained model D4. The neural network N4 is stored in the main memory 18 in the form of a program. The trained parameter data P4' may be stored in the main memory 18, or in a storage medium connected to the ultrasound diagnostic device 10 via the network N. In this case, the necessity information generating function 172 realized by the processor of the processing circuitry 17 reads out the trained model D4 from the main memory 18 and executes it to generate time phase band information of the contrast image data based on the contrast image data. The trained model D4 may be constructed by an integrated circuit such as an ASIC or an FPGA.

[0120] In order to improve the accuracy of judgment by the necessity information generating function 172, in addition to the contrast image data, identification information including at least one of the height, weight, medical history, and medical history of blood relatives of the subject may be used as input data.

[0121] As a result, the time phase band information of each contrast image data is known. As described in the first to third embodiments, the necessity information generating function 172 can generate necessity information for storing the contrast image data of the subject by inputting the contrast image data of the subject to a trained model for generating necessity information based on a plurality of contrast image data and their time phase band information. In other words, the fourth embodiment can be applied to any of the first to third embodiments.

[0122] According to the medical image processing device M4, not only the elapsed time from the start of contrast agent injection but also the time phase band information can be estimated from the contrast image data. In addition, by applying the estimated time phase band information to the first to third embodiments, in the post-process after the imaging is completed, it is possible to make a definitive diagnosis of a tumor that does not depend on the experience or subjectivity of an operator such as a doctor, and to reduce the diagnosis time.

[0123] 7. Medical image processing apparatus according to the fifth embodiment The first to fourth embodiments described above are characterized by setting time phase band information of each contrast image data and setting necessity information of each contrast image data. This embodiment generates diagnosis result data of a subject regarding a tumor based on the time phase band information or necessity information of each contrast image data described above. This case will be described as a medical image processing device M5 (shown in FIG. 1) according to a fifth embodiment.

[0124] The configuration of a medical image processing apparatus M5 according to the fifth embodiment is the same as that shown in FIG. 1, and therefore a description thereof will be omitted.

[0125] FIG. 10 is a block diagram showing the functions of the medical image processing device M5.

[0126] The processing circuitry 17 realizes an image acquisition function 171, a diagnosis result generation function 174, and an output control function 175 by reading and executing a program stored in the main memory 18 or directly incorporated in the processing circuitry 17. The following description will be given taking as an example a case where the functions 171, 174, and 175 function as software, but all or a part of the functions 171, 174, and 175 may be provided in the ultrasound diagnostic device 10 as a circuit such as an ASIC.

[0127] In FIG. 10, the same members as those shown in FIG. 2 are denoted by the same reference numerals and their description will be omitted.

[0128] The diagnosis result generating function 174 includes a function of generating information related to the type of lesion site (e.g., diagnosis result data of a subject related to a tumor) by inputting a plurality of contrast image data of a subject related to a plurality of time phases to a trained model for generating diagnosis result data related to a tumor based on a plurality of contrast image data related to a plurality of different time phases. The diagnosis result generating function 174 is an example of a diagnosis result generating unit.

[0129] The output control function 175 includes a function of displaying the diagnostic result data generated by the diagnostic result generating function 174 on the display 40, or outputting the diagnostic result data as audio via a speaker (not shown) to present the diagnostic result data to an operator such as a doctor. The output control function 175 is an example of an output control unit.

[0130] The functions 171, 174, and 175 will be described in detail with reference to FIGS.

[0131] Next, the operation of the ultrasound diagnostic device 10 will be described. Here, a combination of two (or three) different time phase zones will be described as an example of n (n=2, 3, 4, ...) different time phase zones, but the present invention is not limited to this case. The different multiple time phase zones may be, for example, a combination of four or more different time phase zones.

[0132] Fig. 11 is a flowchart showing the operation of the ultrasound diagnostic device 10. In Fig. 11, the letters "ST" followed by numbers indicate each step of the flowchart.

[0133] In FIG. 11, the same steps as those in FIG. 3 are denoted by the same reference numerals and the description thereof will be omitted.

[0134] Following step ST4, the diagnosis result generating function 174 judges whether the time phase of the contrast image data generated in step ST3 corresponds to the second or subsequent time phase band after the injection of the contrast agent (step ST15). After the injection of the contrast agent, the time phase bands are classified according to the degree of tumor staining by the contrast agent. For example, when the imaging site is the liver, the time phase bands are classified according to the elapsed time from the start of the injection of the contrast agent, such as an early vascular phase, an arterial dominant phase, a portal vein dominant phase, and a posterior vascular phase.

[0135] If the judgment in step ST15 is NO, i.e., if it is judged that the time phase of the contrast image data generated in step ST3 corresponds to the first time phase band after injection of the contrast agent (e.g., the early vascular phase), the image acquisition function 171 controls the image generation circuit 14, etc. to generate contrast image data for the next frame and store it in the image memory 15 (step ST3).

[0136] On the other hand, if the determination in step ST15 is YES, that is, if it is determined that the time phase of the contrast image data generated in step ST3 is related to the second or later time phase band (for example, arterial dominant phase) after the injection of the contrast agent, the diagnosis result generating function 174 generates diagnosis result data on the tumor based on the contrast image data of the latest frame generated in step ST3 and the contrast image data of the past frame written in the image memory 15 (step ST16). Here, the contrast image data of the past frame is image data of a frame related to a time phase band different from the time phase band of the latest frame.

[0137] Here, the diagnosis result generating function 174 performs a process of generating diagnosis result data on a tumor based on a combination of two contrast image data generated in two different time phases. For example, a lookup table (LUT) that associates the combination of two contrast image data generated in two time phases with the diagnosis result data on a tumor may be used for this process. Machine learning may also be used for this process. As the machine learning, deep learning using a multi-layered neural network such as CNN or convolutional deep belief network may also be used.

[0138] In the explanation of the fifth embodiment, an example is shown in which the diagnosis result generating function 174 includes a neural network N5, and uses deep learning to generate diagnosis result data regarding a tumor based on a combination of two contrast image data (one contrast image data relating to a first time phase band and one contrast image data relating to a second time phase band) generated respectively in two time phase bands. Fig. 12(A) is an explanatory diagram showing an example of data flow during learning, and Fig. 12(B) is a diagram showing an example of training data in the form of a table.

[0139] The diagnosis result generating function 174 sequentially updates the parameter data P5 by learning from a large amount of input training data. The training data is composed of combinations F1, F2, F3, ... of two contrast image data generated in two time phases as training input data, and ideal diagnosis results G1, G2, G3, .... The combinations F1, F2, F3, ... of two contrast image data generated in two time phases constitute a training input data group F. The ideal diagnosis results G1, G2, G3, ... constitute a training output data group G. The ideal diagnosis results G1, G2, G3, ... may be definitive diagnosis results regarding tumors determined by extracting tissues from the subjects of the imaging subjects of the corresponding combinations F1, F2, F3, ....

[0140] The diagnosis result generating function 174 performs so-called learning, updating the parameter data P5 each time training data is input, so that the result of processing the combinations F1, F2, F3, ... of two contrast image data respectively generated in two time phases by the neural network N5 approaches the definitive diagnosis results G1, G2, G3, ... of the tumor. In general, when the rate of change of the parameter data P5 converges within a threshold, the learning is determined to be completed. Hereinafter, the parameter data P5 after learning is particularly referred to as the learned parameter data P5' (shown in FIG. 13).

[0141] Note that the type of training input data should match the type of input data during operation shown in Fig. 13. For example, when the input data during operation is contrast image data of a subject, the training input data group F during learning is also contrast image data.

[0142] In addition, the "image data" includes raw data generated by a medical image generating device such as an ultrasonic diagnostic device. In other words, the input data of the neural network N5 may be raw data before scan conversion.

[0143] FIG. 13 is an explanatory diagram showing an example of data flow during operation.

[0144] During operation, the diagnostic result generation function 174 inputs a combination Fa of two contrast image data generated in two time phases of a subject to be diagnosed into the learned model D5, and outputs diagnostic result data Ga regarding the subject's tumor using the learned parameter data P5'.

[0145] The neural network N5 and the trained parameter data P5' configure a trained model D5. The neural network N5 is stored in the main memory 18 in the form of a program. The trained parameter data P5' may be stored in the main memory 18, or in a storage medium connected to the ultrasound diagnostic device 10 via the network N. In this case, the diagnosis result generating function 174 realized by the processor of the processing circuitry 17 reads out the trained model D5 from the main memory 18 and executes it to generate diagnosis result data on a tumor based on a combination of two sets of contrast image data generated in two time phases. The trained model D5 may be constructed by an integrated circuit such as an ASIC or an FPGA.

[0146] The diagnosis result data Ga on the subject's tumor output by the trained model D5 including the diagnosis result generating function 174 may be, for example, data indicating whether the tumor is benign or malignant, data indicating which of category classifications such as HCC (hepatocellular carcinoma), Meta (metastatic liver tumor), Hema (hepatic hemangioma), and FNH (focal nodule) the tumor belongs to, or data indicating the possibility of category classification in percentage (for example, x% benign, y% malignant). Furthermore, the category classification may be a differentiation degree (high differentiation, low differentiation, etc.) that further subdivides HCC, etc.

[0147] In addition, in order to improve the accuracy of tumor diagnosis by the diagnostic result generating function 174, in addition to the combination of two contrast image data, identification information including at least one of the height, weight, medical history, and medical history of blood relatives of the subject may be used as input data.

[0148] In this case, during learning, the identification information of the subject of each of the combinations F1, F2, F3, ... of two contrast image data generated in two time phases as training input data is also input to the neural network N5 as training input data. During operation, the diagnosis result generating function 174 outputs diagnosis result data Ga regarding the tumor of the subject by inputting the identification information of the subject together with the combination Fa of two contrast image data generated in two time phases of the subject to be diagnosed to the learned model D5 read from the main memory 18. By using the combination of two contrast image data generated in two time phases and the identification information of the subject as input data, learned parameter data P5' that has been learned according to the type of the subject can be generated, and thus the accuracy of diagnosis can be improved compared to the case where only the combination of two contrast image data generated in two time phases is used as input data.

[0149] Returning to the explanation of Fig. 11, the output control function 175 displays the diagnostic result data generated in step ST16 on the display 40 (step ST17). Alternatively, the output control function 175 presents the diagnostic result data to an operator such as a doctor by outputting it as sound via a speaker (not shown).

[0150] FIG. 14 is a diagram showing an example of displaying the diagnosis result.

[0151] 14, the display screen includes an image based on the contrast image data of the latest frame, an image based on the contrast image data of a past frame combined with the image based on the latest frame, and diagnostic result data which is the output of the diagnostic result generating function 174. As the diagnostic result, data showing the possibility of category classification in percentage is included.

[0152] Returning to the explanation of FIG. 11, the image acquiring function 171 determines whether or not to end the ultrasonic imaging started in step ST2 (step ST18).

[0153] If the determination in step ST18 is NO, that is, if it is determined that the ultrasonic imaging started in step ST2 is not to be ended, the image acquisition function 171 controls the image generation circuit 14 and the like to generate contrast image data for the next frame and store it in the image memory 15 (step ST3). That is, the ultrasonic imaging is continued.

[0154] On the other hand, if the judgment in step ST18 is YES, i.e., if it is judged that the ultrasound imaging started in step ST2 is to be terminated, the image acquisition function 171 judges that the operator has given an instruction to terminate via the input interface 30, and terminates the ultrasound imaging.

[0155] That is, according to the flowchart shown in FIG. 11, the ultrasound diagnostic apparatus 10 repeats the set of steps ST3, ST4, and ST15 to ST18 in the second and subsequent time phases to generate and display diagnosis result data regarding the tumor for each frame.

[0156] The contrast image data is not limited to one frame image data. The contrast image data may be video data consisting of a plurality of consecutive frames. In this case, a combination of two video data sets is used as training input data, and a combination Fa of two video data sets of a subject to be diagnosed is used as an input of the trained model D5. This makes it possible to learn the dynamic movement of bubbles contained in the contrast agent.

[0157] 11, the diagnosis result generating function 174 generates diagnosis result data every time the contrast image data of each successive frame is generated, but this is not limited to the case. For example, the diagnosis result generating function 174 may generate diagnosis result data every time the contrast image data of each frame generated at a certain frame interval is generated, or may generate diagnosis result data every time the first frame of contrast image data of each time phase is generated.

[0158] The medical image processing device M5 can assist in the definitive diagnosis of a tumor based on multiple contrast image data obtained in multiple different time phases, thereby enabling a definitive diagnosis of a tumor in real time that is not dependent on the experience or subjectivity of an operator such as a doctor.

[0159] 8. Medical image processing apparatus according to the sixth embodiment In the above-mentioned fifth embodiment, a single neural network is configured regardless of the time phase, that is, the diagnosis result generating function 174 is described as inputting contrast image data of a subject related to each time phase to one trained model to generate diagnosis result data of the subject related to each time phase. However, the present invention is not limited to this case.

[0160] For example, one neural network may be configured based on a combination of multiple time phases related to a predetermined time phase band or earlier, and another neural network may be configured based on a combination of multiple time phases related to another time phase band or earlier. That is, the diagnosis result generating function 174 generates diagnosis result data of a subject by inputting a combination of multiple contrast image data of a subject to a trained model corresponding to the combination among multiple trained models corresponding to the combination of multiple contrast image data related to multiple time phase bands. This case will be described as a medical image processing device M6 according to a sixth embodiment (shown in FIG. 1 and FIG. 10).

[0161] The configuration and functions of the medical image processing device M6 are equivalent to those shown in FIGS. 1 and 10, and the operation of the medical image processing device M6 is equivalent to that shown in FIG. 11, so a description thereof will be omitted.

[0162] In step ST16 shown in FIG. 11, the diagnostic result generating function 174 shown in FIG. 10 generates diagnostic result data regarding a tumor based on a combination of the contrast image data of the latest frame generated in step ST3 and the contrast image data of the past frame stored in the image memory 15.

[0163] Here, the diagnosis result generating function 174 performs processing by distinguishing between the case where the target frame is in the first time phase band and the case where the target frame is in the next second time phase band. When the target frame is in the first time phase band, the diagnosis result generating function 174 performs processing to generate diagnosis result data on the tumor based on a combination of n pieces of contrast image data corresponding to n time phase bands relating to the first time phase band or earlier, and when the target frame is in the second time phase band, the diagnosis result generating function 174 performs processing to generate diagnosis result data on the tumor based on a combination of n pieces of contrast image data corresponding to n+1 time phase bands relating to the second time phase band or earlier.

[0164] For example, a lookup table that associates a combination of two contrast image data generated in two time phases before the first time phase with diagnosis result data on a tumor, and a lookup table that associates a combination of three contrast image data generated in three time phases before the second time phase with diagnosis result data on a tumor may be used in this process. Machine learning may also be used in this process. As the machine learning, deep learning using a multi-layer neural network such as CNN or convolutional deep belief network may be used.

[0165] In the explanation of the sixth embodiment, an example is shown in which the diagnosis result generating function 174 includes a plurality of neural networks N6 (for example, “N61” and “N62” shown in FIGS. 15(A) and 15(B) and the like) and generates diagnosis result data regarding a tumor based on a combination of n pieces of contrast image data generated in n time phase bands using deep learning.

[0166] Fig. 15(A) is an explanatory diagram showing an example of data flow during learning when combining multiple time bands corresponding to the first time band and earlier. Fig. 15(B) is an explanatory diagram showing an example of data flow during learning when combining multiple time bands corresponding to the second time band and earlier.

[0167] 15A will be described. The diagnosis result generating function 174 sequentially updates the parameter data P61 by learning by inputting a large amount of training data relating to the first time phase or earlier. The training data is composed of combinations H1, H2, H3, ... of two contrast image data generated in two early vascular phases and an arterial dominant phase before the arterial dominant phase as training input data, and ideal diagnosis results I1, I2, I3, .... The combinations H1, H2, H3, ... of two contrast image data generated in two early vascular phases and an arterial dominant phase before the arterial dominant phase constitute a training input data group H. The ideal diagnosis results I1, I2, I3, ... constitute a training output data group I. The ideal diagnosis results I1, I2, I3, ... may be definitive diagnosis results regarding tumors determined by extracting tissues from the subjects of the imaging of the corresponding combinations H1, H2, H3, ....

[0168] The diagnosis result generating function 174 performs so-called learning, updating the parameter data P61 each time training data is input, so that the result of processing the combinations H1, H2, H3, ... of two contrast image data generated in two early vascular phases before the arterial dominant phase and the arterial dominant phase, respectively, by the neural network N61, approaches the definitive diagnosis results I1, I2, I3, ... of the tumor. In general, when the rate of change of the parameter data P61 converges within a threshold, learning is determined to be completed. Hereinafter, the parameter data P61 after learning is particularly referred to as learned parameter data P61' (shown in FIG. 16(A)).

[0169] Note that the type of training input data should match the type of input data during operation shown in Fig. 16(A). For example, if the input data during operation is contrast image data of a subject, the training input data group H during learning is also contrast image data.

[0170] Furthermore, the "image data" includes raw data generated by a medical image generating device such as an ultrasonic diagnostic device. In other words, the input data to the neural network N61 may be raw data before scan conversion.

[0171] On the other hand, Fig. 15(B) will be described. As in the case of Fig. 15(A), the diagnosis result generating function 174 sequentially updates the parameter data P62 by learning by inputting a large amount of training data relating to the second time phase or earlier. Each time training data is input, the diagnosis result generating function 174 updates the parameter data P62 so that the result of processing the combinations J1, J2, J3, ... of three contrast image data generated in three early vascular phases, the arterial dominant phase, and the portal vein dominant phase, respectively, in the neural network N62 approaches the definitive diagnosis results K1, K2, K3, ... of the tumor, thereby performing so-called learning.

[0172] Fig. 16(A) is an explanatory diagram showing an example of data flow during operation when combining multiple time zones corresponding to the first time zone and earlier, and Fig. 16(B) is an explanatory diagram showing an example of data flow during operation when combining multiple time zones corresponding to the second time zone and earlier.

[0173] 16(A) will be described. During operation, the diagnosis result generating function 174 obtains a time phase band corresponding to a target frame (e.g., the latest frame) of a subject to be diagnosed. Then, when the diagnosis result generating function 174 determines that the time phase band is a first time phase band, for example, an arterial dominant phase, it inputs a combination Ha of two contrast image data generated in two early vascular phases and the arterial dominant phase before the arterial dominant phase to the trained model D61, and outputs the diagnosis result data Ia regarding the tumor of the subject using the trained parameter data P61'.

[0174] The neural network N61 and the trained parameter data P61' configure a trained model D61. The neural network N61 is stored in the main memory 18 in the form of a program. The trained parameter data P61' may be stored in the main memory 18, or in a storage medium connected to the ultrasound diagnostic device 10 via the network N. In this case, the diagnosis result generating function 174 realized by the processor of the processing circuitry 17 reads out the trained model D61 from the main memory 18 and executes it to generate diagnosis result data on a tumor based on a combination of two sets of contrast image data generated in two early vascular phases and an arterial dominant phase, respectively. The trained model D61 may be constructed by an integrated circuit such as an ASIC or an FPGA.

[0175] The diagnosis result data Ia on the subject's tumor output by the trained model D61 including the diagnosis result generating function 174 may be, for example, data indicating whether the tumor is benign or malignant, data indicating which of the category classifications such as HCC, Meta, Hema, and FNH the tumor belongs to, or data indicating the possibility of the category classification in percentage (for example, x% benign, y% malignant). Furthermore, the category classification may be a differentiation degree (highly differentiated, poorly differentiated, etc.) that further subdivides HCC, etc.

[0176] In addition, in order to improve the accuracy of tumor diagnosis by the diagnosis result generating function 174, in addition to the contrast image data, identification information including at least one of the height, weight, medical history, and medical history of blood relatives of the subject may be used as input data.

[0177] On the other hand, Fig. 16(B) will be described. As in the case of Fig. 16(A), during operation, the diagnosis result generating function 174 obtains a time phase band corresponding to a target frame (e.g., the latest frame) of a subject to be diagnosed. Then, when the diagnosis result generating function 174 determines that the time phase band is a second time phase band, for example, a portal vein dominant phase, it inputs a combination Ja of three contrast image data generated respectively for three early vascular phases, an arterial dominant phase, and a portal vein dominant phase to the trained model D62, and outputs the diagnosis result data Ka regarding the subject's tumor using the trained parameter data P62'.

[0178] In the medical image processing device M6, the combinations of multiple time phases are classified into two or three, and a neural network is configured for each of them. However, the present invention is not limited to this. For example, a different neural network may be configured for each combination of four or more time phases. Alternatively, one neural network may be configured based only on the combination of characteristic time phases.

[0179] Regarding the former, for example, one neural network is configured for a combination of two early vascular phases and an arterial dominant phase, one neural network is configured for a combination of three early vascular phases, an arterial dominant phase, and a portal vein dominant phase, and one neural network is configured for a combination of four early vascular phases, an arterial dominant phase, a portal vein dominant phase, and a posterior vascular phase. When a time phase band corresponding to a target frame (e.g., the latest frame) of a subject to be diagnosed is an arterial dominant phase, a neural network based on a combination of two early vascular phases and an arterial dominant phase is used, when the time phase advances and the time phase band corresponding to the target frame becomes the portal vein dominant phase, a neural network based on a combination of three early vascular phases, an arterial dominant phase, and a portal vein dominant phase is used, and when the time phase advances and the time phase band corresponding to the target frame becomes the posterior vascular phase, a neural network based on a combination of four early vascular phases, an arterial dominant phase, a portal vein dominant phase, and a posterior vascular phase is used.

[0180] As a result, as the time phase progresses from the start of contrast agent injection, the number of input images increases, and the neural network to be applied changes in real time.

[0181] Moreover, it goes without saying that the diagnostic result data of the subject generated in the sixth embodiment may be configured to be output together with the diagnostic result data of the subject generated in the fifth embodiment.

[0182] Furthermore, the contrast image data is not limited to one frame image data. The contrast image data may be video data for each time phase band, which is composed of a plurality of consecutive frames. In this case, a combination of video data relating to the early vascular phase and video data relating to the arterial dominant phase is set as the first training input data, and a combination Ha of the video data relating to the early vascular phase and video data relating to the arterial dominant phase of the subject is input to the trained model D61. A combination of video data relating to the early vascular phase, video data relating to the arterial dominant phase, and video data of the portal vein dominant phase is set as the second training input data, and a combination Ja of the video data relating to the early vascular phase, video data relating to the arterial dominant phase, and video data of the portal vein dominant phase of the subject is input to the trained model D62. With these, the dynamic movement of bubbles contained in the contrast agent can be learned.

[0183] The contrast image data may be one video data set that is divided into a plurality of time phase bands by indexes and is composed of a plurality of consecutive frames. In this case, a combination of partial data of a video image related to an early vascular phase divided by an index and partial data of a video image related to an arterial-dominant phase divided by an index is used as first training input data, and a combination Ha of partial data of a video image related to an early vascular phase divided by an index and partial data of a video image related to an arterial-dominant phase divided by an index of a subject is input to the trained model D61.

[0184] According to the medical image-processing device M6, in addition to the effects achieved by the medical image-processing device M5, it is possible to obtain more accurate diagnosis results.

[0185] 9. Medical image processing apparatus according to the seventh embodiment In the fifth and sixth embodiments described above, the image acquisition function 171 acquires live contrast image data, and the diagnosis result generating function 174 generates the definitive diagnosis result data of the subject in approximately real time every time the contrast image data is acquired. However, the present invention is not limited to this case.

[0186] For example, the operator may perform a process of generating data on a definitive diagnosis result of a subject by operating a screen displayed on the display 40. This case will be described as a medical image processing device M7 according to a seventh embodiment (shown in FIGS. 1 and 10).

[0187] The configuration and functions of the medical image processing device M7 relating to the seventh embodiment are equivalent to those shown in Figures 1 and 10, and the operation of the medical image processing device M7 relating to the seventh embodiment is equivalent to that shown in Figure 11, so the explanation will be omitted.

[0188] 11, a certain phase in the arterial-dominant phase is set in addition to a certain phase in the early vascular phase by an operation via the input interface 30. When a certain phase in the arterial-dominant phase is set in addition to a certain phase in the early vascular phase, the diagnosis result generating function 174 generates definitive diagnosis result data for the subject by inputting a combination of contrast image data of the set phases into the trained model D5 (shown in FIG. 13) or the trained model D61 (shown in FIG. 16(A)).

[0189] FIG. 17 is a diagram showing an example of a phase setting screen.

[0190] The upper part of Fig. 17(A) shows live-displayed contrast image data, and the lower part of Fig. 17(A) shows a time phase band that is predetermined based on the elapsed time from the start of contrast agent injection, the actual elapsed time from the start of contrast agent injection (time slider), and a "Set" button.

[0191] While checking the time slider, the operator clicks the "Set" button via the input interface 30 when contrast image data useful for diagnosis appears as a live image in each time phase. When the "Set" button is clicked in the early vascular phase, a certain time phase in the early vascular phase is set. Then, to make the set time phase visible, a mark is displayed at the corresponding time phase on the time slider, and the contrast image data for that phase is displayed as a still image in the upper right.

[0192] When the imaging time has elapsed and the "Set" button is clicked at a certain time phase of the arterial dominant phase, a certain time phase in the arterial dominant phase is set. By setting the contrast image data via this input interface 30, the diagnosis result generating function 174 can set the time phase band information corresponding to the contrast image data as the time phase band information to be input to the trained model. FIG. 17(B) shows live-displayed contrast image data, a time phase band, a time slider, and a "Set" button, similar to FIG. 17(A).

[0193] While checking the time slider, the operator clicks the "Set" button via the input interface 30 when contrast image data effective for diagnosis appears as a live image in each time phase. In addition, when the contrast image data effective for diagnosis disappears, the operator clicks the "Set" button via the input interface 30. When the "Set" button is clicked twice in the early vascular phase, a certain phase width in the early vascular phase is set. Then, to make the set phase width visible, marks are displayed on the corresponding two phases on the time slider, and multiple contrast image data between the two phases are displayed as a video in the upper right.

[0194] When the imaging time has elapsed and the "Set" button is clicked twice at a certain phase in the arterial dominant phase, a certain phase width in the arterial dominant phase is set. By setting the contrast image data via this input interface 30, the diagnosis result generating function 174 can set the phase band information corresponding to the contrast image data as the phase band information to be input to the trained model.

[0195] The setting screen may be configured to display each time a definitive diagnosis result data is generated according to the set combination of contrast image data. An example of this case is shown in FIG.

[0196] FIG. 18 is a diagram showing an example of display of definitive diagnosis result data.

[0197] Fig. 18 shows definitive diagnosis result data generated according to a combination of contrast image data related to the time phases set on the time phase setting screen of Fig. 17(A). Similarly, definitive diagnosis result data can be displayed in Fig. 17(B).

[0198] When a certain phase in the arterial dominant phase is set, the diagnosis generating function 174 generates the definitive diagnosis data "HCC: 30%" and "Meta: 10%" based on a combination of the contrast image data of the phase set in the early vascular phase and the contrast image data of the phase set in the arterial dominant phase. The output control function 175 causes the definitive diagnosis data "HCC: 30%" and "Meta: 10%" to be displayed on the display 40 as shown in the lower part of FIG. 18.

[0199] According to the medical image processing device M7, it is possible to carefully select the data to be input into the trained model, so in addition to the effects of the medical image processing device M5, it is possible to obtain diagnostic results more quickly and also to save hardware resources.

[0200] 10. Medical image processing apparatus according to the eighth embodiment In the fifth to seventh embodiments described above, the image acquiring function 171 acquires live contrast image data, and the diagnosis generating function 174 generates definitive diagnosis data for the subject in approximately real time. That is, this is a case where the time band information of each contrast image data is known. Here, a case where the time band information of each contrast image data is unknown will be described as a medical image processing device M8 (shown in FIGS. 1 and 10) according to the eighth embodiment. When the time band information of each contrast image data is unknown, the image acquiring function 171 acquires multiple contrast image data from the image memory 15 after imaging is completed, and the diagnosis generating function 174 generates definitive diagnosis data in a post-process based on the multiple contrast image data, and the diagnosis generating function 174 may also generate definitive diagnosis data in approximately real time.

[0201] The diagnosis result generation function 174 generates time phase band information by inputting the contrast image data acquired by the image acquisition function 171 into a trained model for generating time phase band information classified according to the staining state of the lesion site contained in the contrast image data by the contrast agent.

[0202] The method of generating time-phase band information in the medical image processing device M7 is the same as that explained using Figures 9(A) and (B), so the explanation will be omitted. The method of generating time-phase band information in the medical image processing device M7 is simply a replacement of the "necessity information generating function 172" shown in Figures 9(A) and (B) with the "diagnosis result generating function 174".

[0203] According to the medical image processing device M8, not only the elapsed time from the start of contrast agent injection but also time phase band information can be estimated from contrast image data. In addition, by applying the estimated time phase band information to the fifth to seventh embodiments, a definitive diagnosis of a tumor can be made in the post-processing after the end of imaging, without relying on the experience or subjectivity of an operator such as a doctor.

[0204] 11. First Modification In the fifth to eighth embodiments described above, the training input data and the input to the trained model are a combination of n contrast image data generated in n time phase bands, respectively, or a combination of n contrast image data generated in n time phase bands and additional information such as identification information, but the present invention is not limited to this. At least one of B-mode image data, color Doppler image data, and power Doppler image data may be added to the training input data and the input to the trained model.

[0205] The added image data may be image data relating to one corresponding time phase band, or two image data generated in two corresponding time phase bands. Note that the B-mode image data includes so-called B-mode image data generated by imaging different from the contrast image data by CHI, and fundamental image data generated from reflected wave data (received signal) of the fundamental wave component and generated by the same scan as the contrast image, etc.

[0206] According to the first modification of the medical image-processing devices M5 to M8, in addition to the effects of the medical image-processing devices M5 to M8, it is possible to obtain more accurate diagnosis results.

[0207] 12. Second Variation In the medical image processing devices M5 to M8 according to the fifth to eighth embodiments described above, the training input data and the input to the trained model are a combination of two contrast image data generated in two time phases, respectively, or a combination of two contrast image data generated in two time phases, respectively, and additional information such as identification information, but the present invention is not limited to these cases. BTI (Bubble Tracking Imaging) image data, which is image data showing a tracking trajectory of a bubble, may be added to the training input data and the input to the trained model.

[0208] The added BTI image data is image data relating to a corresponding one temporal band and showing the trajectory of the bubble up to the frame.

[0209] Although the case where BTI image data is added to the combination of two contrast image data generated in two time phases as the training input data or the input to the trained model has been described, the present invention is not limited to this case. Instead of the combination of two contrast image data generated in two time phases as the training input data or the input to the trained model, BTI image data may be adopted.

[0210] According to the second modification of the medical image processing devices M5 to M8 according to the fifth to eighth embodiments, in addition to the effects of the medical image processing devices M5 to M8, it is possible to obtain more accurate diagnosis results.

[0211] 13.Third Variation In the above fifth to eighth embodiments, the definitive diagnosis result data is generated every time a combination occurs live, but the present invention is not limited to this case.

[0212] 17 and 18, a case has been described in which the image acquiring function 171 acquires live contrast image data, and the diagnosis result generating function 174 generates definitive diagnosis result data in approximately real time. However, the present invention is not limited to this case, and the image acquiring function 171 may acquire multiple contrast image data from the image memory 15 after imaging is completed, and the diagnosis result generating function 174 may generate definitive diagnosis result data in a post-process based on a combination of the multiple contrast image data.

[0213] For example, diagnosis result data of a subject regarding a tumor may be generated based on the necessity information generated by the medical image processing apparatuses M1 to M4 according to the above-described first to fourth embodiments.

[0214] 14. Medical image processing apparatus according to the ninth embodiment FIG. 19 is a schematic diagram showing the configuration of a medical image display device including a medical image processing device according to the ninth embodiment.

[0215] 19 shows a medical image display device 70 including a medical image processing device M9 according to the ninth embodiment. The medical image display device 70 is a medical image management device (image server), a workstation, an image interpretation terminal, or the like, and is provided on a medical image system connected via a network. The medical image display device 70 may be an offline device.

[0216] The medical image display device 70 includes a processing circuitry 71, a memory 72, an input interface 73, a display 74, and a network interface 75. The processing circuitry 71 and the memory 72 constitute a medical image processing device M9. Alternatively, the medical image processing device M9 may be constituted by adding at least one of the input interface 73 and the display 74 to the processing circuitry 71 and the memory 72.

[0217] It should be noted that the processing circuit 71, memory 72, input interface 73, and display 74 have the same configuration as the processing circuit 17, main memory 18, input interface 30, and display 40 shown in Figures 1 and 10, and therefore their explanations are omitted.

[0218] The network interface 75 is configured with a connector conforming to a parallel connection specification or a serial connection specification. When the medical image display device 70 is provided on a medical image system, the network interface 75 transmits and receives information to and from an external device on the network. For example, under the control of the processing circuitry 71, the network interface receives medical image data such as contrast image data from an external device.

[0219] Next, the functions of the medical image display device 70 equipped with the medical image processing device M9 will be described.

[0220] FIG. 20 is a block diagram showing the functions of a medical image display device 70 equipped with a medical image processing device M9.

[0221] The processing circuitry 71 executes a program stored in the memory 72 to realize an image acquisition function 711, a necessity information generation function 712, a storage control function 713, a diagnosis result generation function 714, and an output control function 715. Note that all or a part of the functions 711 to 715 are not limited to being realized by execution of a program of the medical image processing device M9, and may be provided in the medical image display device 70 as a circuit such as an ASIC.

[0222] The image acquiring function 711 includes a function of acquiring contrast image data from the medical image management device 60 or the ultrasound diagnostic device 10 via the network interface 75. The image acquiring function 711 is an example of an image acquiring unit.

[0223] The necessity information generating function 712 includes a function equivalent to the necessity information generating function 172 shown in Fig. 2. The necessity information generating function 712 is an example of a processing unit.

[0224] The storage control function 713 includes a function equivalent to the storage control function 173 shown in Fig. 2. The storage control function 713 is an example of a storage control unit.

[0225] The diagnostic result generating function 714 includes a function equivalent to the diagnostic result generating function 174 shown in Fig. 10. The diagnostic result generating function 714 is an example of a processing unit.

[0226] The output control function 715 includes a function equivalent to the output control function 175 shown in Fig. 10. The output control function 715 is an example of an output control unit.

[0227] When the medical image processing device M9 remotely controls the ultrasonic diagnostic device 10 via the network N, it can obtain a live image from the ultrasonic diagnostic device 10 and generate necessity information and definitive diagnosis result data using contrast image data as the live image. The medical image processing device M9 can also obtain contrast image data after imaging is completed from the ultrasonic diagnostic device 10 or the medical image management device 60 via the network N and generate necessity information and definitive diagnosis result data in post-process. Note that the operation of the medical image display device 70 is the same as the operation of the ultrasonic diagnostic device 10 shown in FIG. 3 or FIG. 11, so a description thereof will be omitted.

[0228] The medical image processing device M9 enables a definitive diagnosis of a tumor that is not dependent on the experience or subjectivity of an operator such as a doctor, using a device other than a medical image generating device. In addition, the medical image processing device M9 can support a definitive diagnosis of a tumor based on multiple contrast image data obtained in multiple different time phases, enabling a definitive diagnosis of a tumor that is not dependent on the experience or subjectivity of an operator such as a doctor.

[0229] According to at least one of the embodiments described above, a definitive diagnosis of a tumor can be made without depending on the experience or subjectivity of an operator such as a doctor.

[0230] The image acquisition functions 171 and 711 are an example of an image acquisition unit. The necessity information generation functions 172 and 712 are an example of a processing unit. The storage control functions 173 and 713 are an example of a storage control unit. The diagnosis result generation functions 174 and 714 are an example of a processing unit. The output control functions 175 and 715 are an example of an output control unit.

[0231] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0232] M1~M9 Medical image processing equipment 17,71 Processing circuit 171,711 Image Acquisition Function 172,712 Necessity information generation function 173,713 Storage Control Function 174,714 Diagnostic result generation function 175,715 Output control function

Claims

1. An image acquisition unit that acquires a combination of contrast image data consisting of a predetermined number of frames corresponding to different time phases according to the elapsed time of the imaging, from contrast image data consisting of a plurality of consecutive frames generated by imaging a subject into which a contrast agent has been injected; a processing unit that acquires information for identifying a type of a lesion site in the subject by inputting a combination of a plurality of contrast image data acquired by the image acquisition unit to a trained model that identifies a type of a lesion site by inputting the combination of the plurality of contrast image data corresponding to different time phases and configured with a predetermined number of frames; A medical image processing device having the following:

2. The trained model identifies a type of a lesion site by inputting a combination of the plurality of contrast image data acquired for each time phase and time phase band information corresponding to each of the plurality of contrast image data, the processing unit inputs, to the trained model, a combination of the plurality of contrast image data acquired for each of the time phases by the image acquisition unit and time phase band information corresponding to each of the plurality of contrast image data, thereby acquiring information for identifying a type of a lesion site related to the subject. The medical image processing device according to claim 1 .

3. The trained model is further based on at least one additional image data of B-mode image data, color Doppler image data, and power Doppler image data; The processing unit further inputs additional image data of the subject to the trained model, thereby acquiring information for identifying a type of a lesion site related to the subject. The medical image processing device according to claim 1 .

4. The trained model is further based on tracking image data representing a tracking trajectory of a bubble, The processing unit further inputs tracking image data of the subject to the trained model to obtain information for identifying a type of a lesion site of the subject. The medical image processing device according to claim 1 .

5. The imaging of the subject is ultrasound imaging of a liver. the image acquisition unit acquires a combination of the plurality of contrast image data from the contrast image data generated by ultrasonic imaging of the liver. The medical image processing device according to claim 1 .

Images