Medical image diagnostic device, ultrasound diagnostic device, medical image system, and imaging control method

Abstract

To identify an object area that has not been imaged in medical image data.SOLUTION: A medical image diagnostic apparatus includes an acquisition part and an identification part. The acquisition part acquires a medical image that is acquired by imaging an object of a subject and positional information corresponding to the medical image. The identification part identifies a non-imaged region of the object that is not included in the medical image using the acquired medical image and positional information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The embodiments disclosed in the specification and the like relate to a medical image diagnostic apparatus, an ultrasound diagnostic apparatus, a medical image system, and an imaging control method. [Background technology]

[0002] There are medical imaging diagnostic devices that generate medical image data that visualize the internal tissues of a subject. Examples of medical imaging diagnostic devices include ultrasound diagnostic devices, X-ray CT (Computed Tomography) devices, and MRI (Magnetic Resonance Imaging) devices. Ultrasound diagnostic devices transmit ultrasound waves from an ultrasound probe into a subject, generate echo signals based on the reflected waves, and obtain desired ultrasound images through image processing. X-ray CT devices irradiate the subject with X-rays and generate CT images such as axial tomograms of the subject based on electrical signals based on the X-rays detected by an X-ray detector. MRI devices place the subject in a static magnetic field and generate MRI images of the subject's internal information based on radio-frequency pulses applied to the subject.

[0003] When acquiring images using a medical image diagnostic device, it is possible that only a portion of the entire area of an object (e.g., an organ) to be imaged within a subject is visualized, resulting in a loss of data for a portion of the entire area to be imaged from the image. This is not a major problem if the data-missing area does not appear within the object area, but there are cases in which the data-missing area appears within the object area. When the data-missing area appears within the object area, the portion of the object area where the data-missing area appears becomes an unimaged object area. In such cases, one method is to obtain the unimaged object area by complementing the data-missing area with another image, but there may be cases in which no other image without the data-missing area exists, or where the data-missing area is large and difficult to complement. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2017-225544 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems that the embodiments disclosed in this specification and elsewhere aim to solve is to identify unimaged object regions in medical image data. However, the problems solved by the embodiments disclosed in this specification are not limited to the above problems. Problems corresponding to the effects of the configurations described in the embodiments below can also be considered as other problems solved by the embodiments disclosed in this specification. [Means for solving the problem]

[0006] A medical image diagnostic apparatus according to an embodiment includes an acquisition unit and an identification unit. The acquisition unit acquires medical images obtained by imaging an object of a subject and position information corresponding to the medical images. The identification unit uses the acquired medical images and position information to identify unimaged areas of the object that are not included in the medical images. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 2] FIG. 2 is a diagram for explaining position information of an ultrasound probe in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 3] FIG. 3 is a block diagram showing functions of an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 4] FIG. 4 is a diagram for explaining the cause of a data missing region occurring when continuous imaging is performed using an ultrasound probe in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 5]FIG. 5 is a diagram for explaining the cause of a data missing region occurring when intermittent imaging is performed by an ultrasound probe in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 6] FIG. 6 is a diagram showing the concept of an object in a subject and data regions included in ultrasound image data of a plurality of cross sections in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 7] FIG. 7 is an explanatory diagram showing an example of a data flow during learning in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing an example of a data flow during operation in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 9] FIG. 9 is a flowchart showing a first operation example of an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 10] FIG. 10 is a diagram for explaining a method for determining the body surface position and posture of an ultrasound probe for imaging an object region that has not been imaged due to missing data in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment. [Figure 11] FIG. 11 is a diagram showing an example of display of information about an object region that has not been imaged due to missing data, in an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to the embodiment. [Figure 12] FIG. 12 is a flowchart showing a second operation example of the ultrasound diagnostic apparatus as an example of the medical image diagnostic apparatus according to the embodiment. [Figure 13] FIG. 13 is a schematic diagram showing the configuration of a medical image system including an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to a second modification. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of a medical image diagnostic apparatus, an ultrasound diagnostic apparatus, a medical image system, and an imaging control method will be described in detail with reference to the drawings.

[0009] Examples of medical image diagnostic devices according to the embodiments include an ultrasound diagnostic device, an X-ray CT device, and an MRI device. The X-ray CT device includes an imaging device and a medical image processing device. The imaging device irradiates a subject with X-rays, detects the X-rays with an X-ray detector, and generates an electrical signal. The medical image processing device generates a CT image such as an axial tomogram of the subject based on the received electrical signal. The MRI device includes an imaging device and a medical image processing device. The subject is placed in a static magnetic field generated by the imaging device. The imaging device applies radio frequency pulses to the subject to receive MR signals. The medical image processing device generates an MRI image of the subject based on the MR signals.

[0010] The following description will be given taking an ultrasonic diagnostic apparatus as an example of a medical image diagnostic apparatus, but the present invention is not limited to this case.

[0011] 1. Ultrasound diagnostic equipment FIG. 1 is a schematic diagram showing the configuration of an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to an embodiment.

[0012] FIG. 1 shows an ultrasound diagnostic apparatus 1 as an example of a medical image diagnostic apparatus according to an embodiment. The ultrasound diagnostic apparatus 1 includes an apparatus main body 10 as a medical image processing device, an ultrasound probe 30 as an imaging device, and a position sensor 40. Note that only the apparatus main body 10 may be referred to as the "ultrasound diagnostic apparatus," and the apparatus main body 10 and at least one of the ultrasound probe 30 and the position sensor 40 may be referred to as the "ultrasound diagnostic apparatus." In the following description, a case will be described in which the ultrasound diagnostic apparatus 1 includes both the ultrasound probe 30 and the position sensor 40 in addition to the apparatus main body 10.

[0013] The main body 10 of the ultrasound diagnostic device 1 includes a transmission / reception circuit 11, a B-mode processing circuit 12, a Doppler processing circuit 13, an image generation circuit 14, an image memory 15, a network interface 16, a processing circuit 17, a main memory 18, an input interface 19, and a display 20. The input interface 19 and the display 20 may be provided outside the main body 10, or may be provided outside the ultrasound diagnostic device 1. The circuits 11 to 14 are configured using an application specific integrated circuit (ASIC) or the like. However, this is not a limitation, and all or part of the functions of the circuits 11 to 14 may be realized by the processing circuit 17 executing a computer program.

[0014] The transmission / reception circuit 11 has a transmission circuit and a reception circuit (not shown). The transmission / reception circuit 11 controls the transmission directivity and reception directivity in transmitting and receiving ultrasound under the control of the processing circuit 17. Note that, although a case where the transmission / reception circuit 11 is provided in the device main body 10 will be described, the transmission / reception circuit 11 may be provided in the ultrasound probe 30, or may be provided in both the device main body 10 and the ultrasound probe 30. Note that the transmission / reception circuit 11 is an example of a transmission / reception unit.

[0015] The transmission circuit includes a pulse generation circuit, a transmission delay circuit, a pulser circuit, etc., and supplies a drive signal to the ultrasonic transducer. The pulse generation circuit repeatedly generates rate pulses at a predetermined rate frequency to form transmitted ultrasonic waves. The transmission delay circuit provides each rate pulse generated by the pulse generation circuit with a delay time required to focus the ultrasonic waves generated from the ultrasonic transducer of the ultrasonic probe 30 into a beam and determine the transmission directivity. The pulser circuit also applies drive pulses to the ultrasonic transducer at a timing based on the rate pulse. The transmission delay circuit adjusts the delay time provided to each rate pulse to arbitrarily adjust the transmission direction of the ultrasonic beam transmitted from the piezoelectric transducer surface.

[0016] The receiving circuit, which includes an amplifier circuit, an A / D (Analog to Digital) converter, an adder, etc., receives echo signals received by the ultrasonic transducer and performs various processing on these echo signals to generate echo data. The amplifier circuit amplifies the echo signals for each channel and performs gain correction processing. The A / D converter A / D converts the gain-corrected echo signals and gives the digital data a delay time required to determine the reception directivity. The adder performs addition processing on the echo signals processed by the A / D converter to generate echo data. The addition processing by the adder emphasizes the reflection components from the direction corresponding to the reception directivity of the echo signal.

[0017] Under the control of the processing circuitry 17, the B-mode processing circuitry 12 receives echo data from the receiving circuitry, and performs logarithmic amplification, envelope detection, and other processes to generate data (two-dimensional or three-dimensional data) in which signal strength is expressed as brightness. This data is generally called B-mode data. The B-mode processing circuitry 12 is an example of a B-mode processing unit.

[0018] The B-mode processing circuit 12 can change the frequency band to be visualized by changing the detection frequency through filter processing. Using the filter processing function of the B-mode processing circuit 12, harmonic imaging such as contrast harmonic imaging (CHI) and tissue harmonic imaging (THI) can be performed. That is, the B-mode processing circuit 12 can separate reflected wave data (harmonic data or sub-harmonic data) of harmonic components reflected from the contrast agent (microbubbles, bubbles) and reflected wave data (fundamental wave data) of fundamental components reflected from tissues within the subject from reflected wave data of a subject injected with a contrast agent. The B-mode processing circuit 12 can also generate B-mode data for generating contrast image data from the reflected wave data (received signal) of the harmonic components, and can also generate B-mode data for generating fundamental image data from the reflected wave data (received signal) of the fundamental component.

[0019] Furthermore, in THI, harmonic data or sub-harmonic data, which is reflected wave data (received signals) of harmonic components, can be separated from reflected wave data of the subject using the filter processing function of the B-mode processing circuitry 12. Then, the B-mode processing circuitry 12 can generate B-mode data for generating tissue image data from the reflected wave data (received signals) of harmonic components, with noise components removed.

[0020] Furthermore, when performing harmonic imaging of CHI or THI, the B-mode processing circuit 12 can extract harmonic components using a method different from the above-mentioned filter processing method. Harmonic imaging employs an imaging method called the Amplitude Modulation (AM) method, the Phase Modulation (PM) method, or an AMPM method that combines the AM and PM methods. In the AM, PM, and AMPM methods, ultrasonic waves with different amplitudes and phases are transmitted multiple times along the same scanning line. This causes the transmission / reception circuit 11 to generate and output multiple pieces of reflected wave data (received signals) for each scanning line. The B-mode processing circuit 12 then extracts harmonic components by performing addition and subtraction processing on the multiple pieces of reflected wave data (received signals) for each scanning line according to the modulation method. The B-mode processing circuit 12 then performs envelope detection processing or the like on the reflected wave data (received signals) of the harmonic components to generate B-mode data.

[0021] For example, when the PM method is performed, the transmission / reception circuitry 11 transmits ultrasonic waves of the same amplitude with inverted phase polarity, such as (-1, 1), twice on each scan line according to the scan sequence set by the processing circuitry 17. The transmission / reception circuitry 11 then generates a reception signal resulting from the transmission of "-1" and a reception signal resulting from the transmission of "1," and the B-mode processing circuitry 12 adds these two reception signals together. This removes the fundamental wave component, generating a signal in which the second-order harmonic component remains as the main component. The B-mode processing circuitry 12 then performs envelope detection processing and the like on this signal to generate THI B-mode data and CHI B-mode data.

[0022] Alternatively, for example, in THI, a method of imaging using a second harmonic component and a difference frequency component contained in a received signal has been put to practical use. In an imaging method using a difference frequency component, for example, a transmitted ultrasonic wave having a composite waveform obtained by combining a first fundamental wave having a center frequency of "f1" and a second fundamental wave having a center frequency of "f2" greater than "f1" is transmitted from the ultrasonic probe 30. This composite waveform is a waveform obtained by combining the waveform of the first fundamental wave and the waveform of the second fundamental wave, the phases of which are adjusted to generate a difference frequency component having the same polarity as the second harmonic component. The transmission / reception circuitry 11 transmits the transmitted ultrasonic wave having the composite waveform, for example, twice, while inverting the phase. In such a case, for example, the B-mode processing circuitry 12 adds two received signals to remove the fundamental component and extract harmonic components, mainly consisting of the difference frequency component and the second harmonic component, and then performs envelope detection processing, etc.

[0023] Under the control of the processing circuitry 17, the Doppler processing circuitry 13 performs frequency analysis on velocity information from the echo data from the receiving circuitry, and generates data (two-dimensional or three-dimensional data) that extracts moving object information such as average velocity, variance, and power for multiple points. This data is generally called Doppler data. Here, the moving object refers to, for example, blood flow, tissue such as the heart wall, or a contrast agent. The Doppler processing circuitry 13 is an example of a Doppler processing unit.

[0024] Under the control of the processing circuitry 17, the image generation circuitry 14 generates, as image data, an ultrasound image expressed in a predetermined brightness range based on the echo signals received by the ultrasound probe 30. For example, the image generation circuitry 14 generates, as an ultrasound image, a B-mode image in which the intensity of the reflected wave is expressed as brightness from the two-dimensional B-mode data generated by the B-mode processing circuitry 12. The image generation circuitry 14 also generates, as an ultrasound image, a mean velocity image, a variance image, a power image, or a color Doppler image as a combination of these images, which represent locomotion information from the two-dimensional Doppler data generated by the Doppler processing circuitry 13. The image generation circuitry 14 is an example of an image generation unit.

[0025] Here, the image generation circuit 14 generally converts (scan converts) a scan line signal sequence of an ultrasonic scan into a scan line signal sequence of a video format, such as that of a television, to generate ultrasound image data for display. Specifically, the image generation circuit 14 generates ultrasound image data for display by performing coordinate conversion according to the ultrasound scanning format of the ultrasound probe 30. In addition to scan conversion, the image generation circuit 14 also performs various image processing, such as image processing (smoothing processing) that regenerates an average brightness image using multiple image frames after scan conversion, and image processing (edge enhancement processing) that uses a differential filter within the image. The image generation circuit 14 also combines text information of various parameters, scales, body marks, etc. with the ultrasound image data.

[0026] That is, the B-mode data and Doppler data are ultrasound image data before scan conversion processing, and the data generated by the image generation circuit 14 is ultrasound image data for display after scan conversion processing. Note that the B-mode data and Doppler data are also called raw data. The image generation circuit 14 generates two-dimensional ultrasound image data for display from the two-dimensional ultrasound image data before scan conversion processing.

[0027] The image memory 15 includes a two-dimensional memory that has a plurality of memory cells in two axial directions per frame, the memory having a number of such memory cells for a plurality of frames. The two-dimensional memory as the image memory 15 stores an ultrasound image relating to one frame or multiple frames generated by the image generating circuit 14 as two-dimensional image data under the control of the processing circuit 17. The image memory 15 is an example of a storage unit.

[0028] Furthermore, the image generation circuitry 14 generates three-dimensional B-mode image data by performing coordinate transformation on the three-dimensional B-mode data generated by the B-mode processing circuitry 12. The image generation circuitry 14 also generates three-dimensional Doppler image data by performing coordinate transformation on the three-dimensional Doppler data generated by the Doppler processing circuitry 13. The image generation circuitry 14 generates "three-dimensional B-mode image data or three-dimensional Doppler image data" as "three-dimensional ultrasound image data (volume data)."

[0029] The image memory 15 may include a three-dimensional memory that is a memory having a plurality of memory cells in three axial directions (X-axis, Y-axis, and Z-axis directions). The three-dimensional memory as the image memory 15 stores a plurality of ultrasound images generated by the image generating circuitry 14 as volume data under the control of the processing circuitry 17.

[0030] The image generation circuit 14 then processes the volume data to display the volume data on a two-dimensional display and processes the three-dimensional data in a three-dimensional manner, in order to generate various two-dimensional image data for displaying the volume data stored in the three-dimensional memory on the display 20. The image generation circuit 14 performs such processes as volume rendering (VR), surface rendering (SR), maximum intensity projection (MIP), and multi-planar reconstruction (MPR).

[0031] The network interface 16 implements various information communication protocols according to the type of network. The network interface 16 connects the ultrasound diagnostic apparatus 1 to other devices, such as an external medical image management device or medical image processing device, in accordance with these various protocols. This connection can be an electrical connection via an electronic network. Here, the term "electronic network" refers to any information communication network that utilizes electrical communication technology, including wireless / wired hospital-based local area networks (LANs) and the Internet, as well as telephone communication networks, optical fiber communication networks, cable communication networks, and satellite communication networks.

[0032] The network interface 16 may also implement various protocols for contactless wireless communication. In this case, the device main body 10 can directly transmit and receive data to, for example, the ultrasound probe 30 without going through a network. The network interface 16 is an example of a network connection unit.

[0033] The processing circuit 17 refers to a dedicated or general-purpose CPU (Central Processing Unit), MPU (Microprocessor Unit), or GPU (Graphics Processing Unit), as well as an ASIC, a programmable logic device, etc. Examples of the programmable logic device include a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA).

[0034] Furthermore, the processing circuitry 17 may be configured as a single circuit, or may be configured as a combination of multiple independent circuit elements. In the latter case, the main memory 18 may be provided separately for each circuit element, or a single main memory 18 may store programs corresponding to the functions of multiple circuit elements. The processing circuitry 17 is an example of a processing unit.

[0035] The main memory 18 is composed of semiconductor memory elements such as RAM (Random Access Memory) and flash memory, a hard disk, an optical disk, etc. The main memory 18 may also be composed of portable media such as a USB (Universal Serial Bus) memory and a DVD (Digital Video Disk). The main memory 18 stores various processing programs (including application programs and an OS (Operating System)) used in the processing circuit 17 and data required for executing the programs. The OS may also include a GUI (Graphical User Interface) that makes extensive use of graphics to display information to the operator on the display 20 and allows basic operations to be performed via the input interface 19. The main memory 18 is an example of a storage unit.

[0036] The input interface 19 includes an input device that can be operated by an operator and an input circuit that inputs signals from the input device. The input device can be realized by a trackball, a switch, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that combines a display screen and a touchpad, a non-contact input device that uses an optical sensor, a voice input device, etc. When the operator operates the input device, the input circuit generates a signal corresponding to the operation and outputs it to the processing circuit 17. The input interface 19 is an example of an input unit.

[0037] The display 20 is configured by a general display output device such as a liquid crystal display or an OLED (Organic Light Emitting Diode) display. The display 20 displays various information under the control of the processing circuit 17. The display 20 is an example of a display unit.

[0038] The ultrasound probe 30 of the ultrasound diagnostic device 1 is an imaging device equipped with multiple tiny transducers (piezoelectric elements) on its front surface. The ultrasound probe 30 transmits and receives ultrasound waves to and from an object inside a test subject (e.g., a patient). Each transducer is an electroacoustic conversion element that converts an electrical pulse into an ultrasonic pulse during transmission and converts a reflected wave into an electrical signal (received signal) during reception. The ultrasound probe 30 is small and lightweight, and is connected to the device main body 10 via a cable (or wireless communication).

[0039] The ultrasonic probe 30 is classified into types such as linear type, convex type, and sector type depending on the scanning method. Furthermore, the ultrasonic probe 30 is classified into types such as 1D array probes in which a plurality of transducers are arranged one-dimensionally (1D) in the azimuth direction and 2D array probes in which a plurality of transducers are arranged two-dimensionally (2D) in both the azimuth direction and the elevation direction depending on the array arrangement dimension. The 1D array probe includes a probe in which a small number of transducers are arranged in the elevation direction.

[0040] Here, when a 3D scan, that is, a volume scan, is performed, a 2D array probe equipped with a scanning method such as a linear type, a convex type, or a sector type is used as the ultrasonic probe 30. Alternatively, when a volume scan is performed, a 1D probe equipped with a scanning method such as a linear type, a convex type, or a sector type and equipped with a mechanism for mechanically oscillating in the elevation direction is used as the ultrasonic probe 30. The latter probe is also called a mechanical 4D probe.

[0041] The position sensor 40 detects multiple pieces of position information of the ultrasonic probe 30 in time series and outputs the information to the device main body 10. The position sensor 40 is classified into a type that is attached to the ultrasonic probe 30 and a type that is provided separately from the ultrasonic probe 30. The latter type of sensor is an optical sensor that captures images of characteristic points of the ultrasonic probe 30, which is the measurement target, from multiple positions and detects each position of the ultrasonic probe 30 using the principle of triangulation.

[0042] Furthermore, the position sensor 40 is attached to the ultrasonic probe 30, detects its own position information, and outputs it to the device main body 10. The position information of the position sensor 40 can also be considered as the position information of the ultrasonic probe 30. The position information of the ultrasonic probe 30 includes the position (X, Y, Z) of the ultrasonic probe 30 and the tilt angle (posture) from each axis. For example, the posture of the ultrasonic probe 30 can be detected by a magnetic field transmitter (not shown) sequentially transmitting magnetic fields on three axes and the position sensor 40 sequentially receiving the magnetic fields.

[0043] Furthermore, the position sensor 40 may be a so-called nine-axis sensor that includes at least one of a three-axis gyro sensor that detects angular velocity on three axes in three-dimensional space, a three-axis acceleration sensor that detects acceleration on three axes in three-dimensional space, and a three-axis geomagnetic sensor that detects geomagnetism on three axes in three-dimensional space. Note that the position sensor 40 is not an essential component.

[0044] FIG. 2 is a diagram for explaining the position information of the ultrasonic probe 30. As shown in FIG.

[0045] 2 shows three orthogonal axial directions, i.e., U-axis direction, V-axis direction, and W-axis direction, based on the ultrasonic probe 30. The U-axis direction is defined as the array direction of the transducers, i.e., the azimuth direction, the V-axis direction is defined as the depth direction, i.e., the direction perpendicular to the U-axis and W-axis directions, and the W-axis direction is defined as the elevation direction. The ultrasonic probe 30, whose position and posture are defined by the U-axis direction, V-axis direction, and W-axis direction, is arbitrarily placed in the XYZ space where the subject is placed, and is arbitrarily moved and operated.

[0046] Next, the functions of the ultrasound diagnostic device 1 will be described.

[0047] FIG. 3 is a block diagram showing the functions of the ultrasound diagnostic device 1.

[0048] The processing circuitry 17 reads and executes a computer program stored in the main memory 18 or directly incorporated in the processing circuitry 17, thereby realizing an acquisition function 171, an estimation function 172, an identification function 173, an output control function 174, and a movement control function 175. The following description will be given taking as an example a case where the functions 171 to 175 function as software, but all or part of the functions 171 to 175 may be provided as functions of circuits such as ASICs in the ultrasound diagnostic device 1. Furthermore, all or part of the estimation function 172 and the identification function 173 may be realized by an external device connected via the network N.

[0049] The acquisition function 171 includes a function of controlling the transmission / reception circuit 11, the B-mode processing circuit 12, the Doppler processing circuit 13, the image generation circuit 14, etc. to perform imaging of the object of the subject using the ultrasound probe 30, and acquire ultrasound image data as medical image data. Specifically, the acquisition function 171 collects M-mode image data, B-mode image data, Doppler image data, etc. as the ultrasound image data. The acquisition function 171 also includes a function of acquiring position information corresponding to the ultrasound image data (for example, position information of the ultrasound probe 30 and position information of a cross section based on the position of the ultrasound probe 30).

[0050] For example, the acquisition function 171 acquires ultrasound image data of a plurality of cross sections as medical image data of a plurality of cross sections. The acquisition function 171 also acquires position information corresponding to each of the ultrasound image data of the plurality of cross sections. Hereinafter, a case where the acquisition function 171 acquires ultrasound image data of a plurality of cross sections will be described. The acquisition function 171 is an example of an acquisition unit.

[0051] Here, the ultrasound image data of each cross section acquired by the acquisition function 171 includes a data region to be visualized and a missing region that is not visualized and is difficult to complement. The data region means the range of an ultrasound beam (raster) having a length that can be reached by a transmitted wave.

[0052] 4(A) to 4(G) are diagrams for explaining the cause of the occurrence of a missing region when continuous imaging is performed using the ultrasonic probe 30. FIG.

[0053] 4(A) shows ideal data regions P when the ultrasound probe 30 is moved in the positive direction of the W axis. In FIG. 4(A), the data regions P are distributed almost evenly throughout the object region R, and no data-missing regions where no data regions P exist appear. For example, the object is an organ, and the object region is an organ region corresponding to the organ. Here, an organ is a unit that constitutes the body of a multicellular animal organism, and refers to a collection of tissues that work together to perform a certain function, and includes, for example, the brain, heart, lungs, liver, pancreas, stomach, intestines, blood vessels, nerves, etc.

[0054] 4(B) shows each data region P when the operation speed in the positive direction of the W axis of the ultrasonic probe 30 is not constant, in contrast to FIG. 4(A). In FIG. 4(B), data-deficient regions where no data region P exists appear in locations in the object region R where there is a relatively large gap between adjacent data regions P.

[0055] Figure 4(C) shows each data region P when tissue with a large difference in acoustic impedance is included, in contrast to Figure 4(A). In Figure 4(C), a data-missing region where no data region P exists appears at the center of the object region R on the negative V-axis side. In tissue with a large difference in acoustic impedance, the ultrasound beam in data region P is greatly attenuated, and the length of data region P passing through the tissue becomes shorter.

[0056] Figure 4(D) shows each data region P when imaging of an object region R corresponding to an object to be imaged is stopped midway through imaging, in contrast to Figure 4(A). In Figure 4(D), a data-deficient region where no data region P exists appears in the object region R on the positive side of the W axis.

[0057] 4(E) shows each data region P when the posture of the ultrasonic probe 30 during operation is not constant (the VW axis rotates within the VW plane), in contrast to FIG. 4(A). In FIG. 4(E), data-deficient regions where no data region P exists appear in locations within the object region R where there is a relatively large gap between adjacent data regions P.

[0058] 4(F) shows each data region P when the operation of the ultrasonic probe 30 is not linear (the W axis changes in the U axis direction) in contrast to FIG. 4(A). In FIG. 4(F), a data missing region where no data region P exists appears in the object region R on the positive side of the U axis and the negative side of the W axis.

[0059] 4(G) shows each data region P when the rotation angle during operation of the ultrasonic probe 30 is not constant (the UW axis rotates within the UW plane), in contrast to FIG. 4(A). In FIG. 4(G), data-deficient regions where no data region P exists appear in the object region R where there is a relatively large gap between adjacent data regions P.

[0060] In each of the cases shown in FIGS. 4(B) to (G), compared to the case of FIG. 4(A), a data-deficient area where no data area P exists appears in the object area R.

[0061] 5A and 5B are diagrams for explaining the cause of data loss areas occurring when intermittent imaging is performed using the ultrasound probe 30. FIG.

[0062] Figure 5(A) shows each data region P when it does not include tissue with a large difference in acoustic impedance. Figure 5(B) shows each data region P when it includes tissue with a large difference in acoustic impedance, in contrast to Figure 5(A). In tissue with a large difference in acoustic impedance, the ultrasonic beam in the data region P is greatly attenuated, and the length of the data region P passing through the tissue becomes shorter.

[0063] In the left and right cases shown in FIG. 5(B), compared to the case of FIG. 5(A), a data-deficient area where no data area P exists appears in the object area R due to the shortened data area P. As a result, an unimaged object area Q appears in the object area R. Note that the area remaining after excluding the unimaged object area Q from the object area R is the imaged object area. In other words, the object area R is made up of the unimaged object area Q and the imaged object area.

[0064] 4B to 4G and 5B, complex data missing regions appear. As a result, complex unimaged object regions appear within the object region R.

[0065] Returning to the explanation of Fig. 3, the estimation function 172 includes a function of estimating the shape of an object in the subject and an imaged object region using the ultrasound image data and position information acquired by the acquisition function 171. For example, the estimation function 172 includes a function of estimating the shape of an organ in the subject and an imaged organ region using the ultrasound image data of multiple cross sections and their position information. The estimation function 172 is an example of an estimation unit.

[0066] Furthermore, the estimation function 172 arranges the ultrasound image data acquired by the acquisition function 171 in the image memory 15, which is a three-dimensional memory, based on the position information acquired by the position sensor 40.

[0067] 6A and 6B are diagrams showing the concept of an object in a subject and a data region P included in ultrasound image data of a plurality of cross sections.

[0068] Fig. 6(A) is a diagram showing a liver region R1, which is an object region R, obtained by transmitting and receiving ultrasonic waves to and from the liver, which is an object within a subject. Fig. 6(B) shows a data region P when an ultrasonic probe 30 is placed at multiple positions on the body surface of the subject and ultrasonic waves are transmitted and received from multiple positions to the liver in Fig. 6(A).

[0069] As conceptually shown in Figure 6(B), even if ultrasound is transmitted from every position and angle of the liver R1, it is not possible to acquire the entire liver region R1 as an imageable data region P. In Figure 6(B), the gray areas in Figure 6(A) that are missing are unimaged object regions that appear due to missing data. Missing data occurs due to the reasons explained using Figures 4(A) to (G) and Figures 5(A) and (B).

[0070] Therefore, the estimation function 172 performs a process of estimating the shape of organs such as the liver in the subject based on the ultrasound image data of multiple cross sections arranged three-dimensionally and their position information, and estimates the organ shape and the imaged organ region. Then, the identification function 173, which will be described later, identifies the region remaining after excluding the estimated imaged organ region from the entire estimated liver region R1 as an unimaged object region. Note that the unimaged object region is a portion of the entire object region that cannot be visualized due to a data-deficient region where the data region P is insufficient, and refers to an area that should continue to be imaged within the examination.

[0071] The estimation function 172 may use, for example, a database associating ultrasound image data with organ shapes to estimate the shapes of organs within the subject. The organ shapes may further include shape information indicating the general overall shape of the organ and the organ name, as well as trajectory information indicating how the ultrasound probe 30 should be moved to image the entire organ, various images obtained when the target organ was previously imaged, and position information for those images. The estimation function 172 may also use machine learning to estimate the shapes of organs within the subject. Deep learning using a multilayer neural network, such as a convolutional neural network (CNN) or a convolutional deep belief network (CDBN), may also be used as the machine learning method.

[0072] Below, an example will be shown in which the estimation function 172 includes a neural network Na, and uses deep learning to estimate the overall shape of an organ contained in ultrasound image data from the shape of a portion of the organ contained in the ultrasound image data.

[0073] FIG. 7 is an explanatory diagram showing an example of data flow during learning.

[0074] The estimation function 172 sequentially updates the parameter data Pa by learning from a large amount of input training data. The training data is made up of a combination of ultrasound image data (for example, arbitrary cross-sectional data constituting volume data) S1, S2, S3, ... as training input data and organ shapes T1, T2, T3, ... corresponding to each arbitrary cross-sectional data. The ultrasound image data S1, S2, S3, ... constitute a training input data group Ba. The organ shapes T1, T2, T3, ... constitute a training output data group Ca.

[0075] The estimation function 172 performs so-called learning, updating the parameter data Pa each time training data is input so that the result of processing the ultrasound image data S1, S2, S3, ... by the neural network Na approaches the organ shapes T1, T2, T3, .... Generally, when the rate of change of the parameter data Pa converges within a threshold, learning is determined to be complete. Hereinafter, the parameter data Pa after learning will be referred to specifically as learned parameter data Pa'.

[0076] It should be noted that the type of training input data should match the type of input data during operation shown in Fig. 8. For example, if the input data during operation is ultrasound image data, the training input data group Ba during learning should also be ultrasound image data.

[0077] Furthermore, the "ultrasound image data" includes raw data generated by the ultrasound diagnostic device 1. That is, the input data to the neural network Na may be raw data before scan conversion.

[0078] FIG. 8 is an explanatory diagram showing an example of data flow during operation.

[0079] During operation, the estimation function 172 inputs ultrasound image data Sa of the object to be diagnosed and outputs the organ shape Ta included in the ultrasound image data using the learned parameter data Pa'. When outputting the organ shape Ta, any of the following information may be output along with the overall organ shape and organ name: the current position and angle of the ultrasound probe, the movement of the ultrasound probe 30 to image the unimaged object region, i.e., until the entire organ is imaged, and the position of each captured image within the entire organ.

[0080] The neural network Na and the trained parameter data Pa' constitute a trained model 19a. The neural network Na is stored in the main memory 18 in the form of a program. The trained parameter data Pa' may be stored in the main memory 18, or may be stored in a storage medium connected to the ultrasound diagnostic apparatus 1 via the network N. In this case, the estimation function 172 realized by the processor of the processing circuitry 17 reads and executes the trained model 19a from the main memory 18 to generate organ shape information included in the ultrasound image data. The trained model 19a may be constructed using an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0081] In addition, in order to improve the accuracy of judgment by the estimation function 172, supplementary information including at least one of the height and weight of the subject, image data including other modalities already imaged, and representative model data of the gadget may be used as input data in addition to the ultrasound image data.

[0082] In this case, during learning, the supplementary information of the subject for each of the ultrasound image data S1, S2, S3, ... as training input data is also input as training input data to the neural network Na. During operation, the estimation function 172 inputs the ultrasound image data Ba of the object to be diagnosed along with the supplementary information of the subject having the object to the trained model 19a read from the main memory 18, thereby outputting the organ shape Ta included in the ultrasound image data. By using the ultrasound image data and the supplementary information of the subject as input data, trained parameter data Pa' that has been trained according to the type of subject can be generated, thereby improving estimation accuracy compared to when only ultrasound image data is used as input data.

[0083] Returning to the explanation of FIG. 3, the identification function 173 includes a function of identifying an unimaged object region of the object that is not included in the ultrasound image data of multiple cross sections, using the ultrasound image data of multiple cross sections and position information acquired by the acquisition function 171. Alternatively, the identification function 173 includes a function of identifying an unimaged object region (e.g., an unimaged organ region) based on the object shape (e.g., an organ shape) estimated by the estimation function 172 and an imaged object region (e.g., an organ region). In other words, the identification function 173 three-dimensionally compares the imaged organ shape included in the ultrasound image data with the estimated organ shape, and identifies the unimaged organ region of the organ region that is due to missing data. Note that the identification function 173 is an example of an identification unit.

[0084] The output control function 174 includes a function of outputting, to the outside of the processing circuitry 17, three-dimensional information of the unimaged object region identified by the identification function 173 and / or information for imaging the unimaged object region. For example, the output control function 174 includes a display control function 71, a storage control function 72, and a transmission control function 73. It is sufficient that the output control function 174 includes at least one of the display control function 71, the storage control function 72, and the transmission control function 73.

[0085] The display control function 71 displays the ultrasound image data acquired by the acquisition function 171 on the display 20, and also includes a function to display information about the unimaged object region on the display 20 when the identification function 173 identifies the unimaged object region. For example, the information about the unimaged object region is a display of the unimaged organ region (position and size) relative to the entire organ, and the position (shown as a marker) of the ultrasound probe 30 on the body surface, as well as the angle and pressure during imaging required to image the unimaged organ region. An alert can be displayed together with the display about the unimaged organ region. The position of the ultrasound probe 30 on the body surface may also be projected directly onto the body surface. The display control function 71 is an example of a display control unit.

[0086] The memory control function 72 includes a function of storing, when an unimaged object region is identified by the identification function 173, information for imaging together with three-dimensional information of the unimaged object region in a storage medium such as the image memory 15. The memory control function 72 is an example of a memory control unit.

[0087] The transmission control function 73 includes a function of transmitting, when an unimaged object region is identified by the identification function 173, information for imaging together with three-dimensional information of the unimaged object region to an external device of the medical image processing apparatus 10 (for example, the medical image display device 80 shown in FIG. 13) via the network interface 16. The transmission control function 73 is an example of a transmission control unit.

[0088] The movement control function 175 includes a function to control an external device such as a robot arm to move (including sliding movement, rotational movement, angle of probe attitude, and change of probe pressure) the ultrasound probe 30, which is an imaging device, when an unimaged object region is identified by the identification function 173. The movement operation of the ultrasound probe 30 may be performed by an operator holding the ultrasound probe 30, but may also be performed by an auto-scan that is performed to correct movements such as breathing of the subject, or by a robot arm scan that is performed to reduce the operator's operation of the ultrasound probe 30. The movement control function 175 is an example of a movement control unit.

[0089] Next, the operation of the ultrasound diagnostic device 1 will be described with reference to FIGS.

[0090] Fig. 9 is a flowchart showing a first operation example of the ultrasound diagnostic device 1. In Fig. 9, the reference numerals "ST" followed by numbers indicate the steps of the flowchart. Note that Fig. 9 explains the case where the object to be imaged is an organ.

[0091] The acquisition function 171 controls the transmission / reception circuit 11, the B-mode processing circuit 12, the Doppler processing circuit 13, the image generation circuit 14, etc. to perform ultrasonic imaging of the object of the subject using the ultrasonic probe 30 and acquire ultrasonic image data (step ST1). The display control function 71 displays the ultrasonic image data acquired in step ST1 on the display 20 as an ultrasonic image (step ST2).

[0092] Furthermore, the acquisition function 171 stores the ultrasound image data acquired in step ST1 in a three-dimensional arrangement in the three-dimensional memory of the image memory 15 based on the position information of the ultrasound image data (step ST3). Here, the position sensor 40 is not an essential component because the relative changes in the position and posture of the ultrasound probe 30 can be detected by matching the data area where the data to be visualized exists. The display control function 71 may display the three-dimensional arrangement of the ultrasound image data on the display 20. This allows the operator performing the imaging to visually confirm the extent to which the image has been captured inside the body, particularly in the trunk.

[0093] The acquiring function 171 determines whether or not to end the ultrasonic imaging (step ST4). The acquiring function 171 may determine to end the ultrasonic imaging based on an end instruction input by the operator via the input interface 19, or may determine to end the ultrasonic imaging after a certain time has elapsed since the ultrasonic probe 30 was removed from the body surface of the subject and left in the air. For example, leaving the ultrasonic probe 30 in the air may be determined based on the position information of the ultrasonic probe 30.

[0094] If the determination in step ST4 is NO, that is, if it is determined that ultrasonic imaging for the next slice will be performed without terminating the ultrasonic imaging, the acquisition function 171 controls the transmission / reception circuitry 11, the B-mode processing circuitry 12, the Doppler processing circuitry 13, the image generation circuitry 14, etc. to perform ultrasonic imaging using the ultrasonic probe 30 and acquire ultrasonic image data for the next slice (step ST1). By repeating the set of steps ST1 to ST3 due to NO in step ST4, ultrasonic image data for multiple slices is arranged in the three-dimensional memory of the image memory 15 (shown in FIG. 6(B)).

[0095] On the other hand, if the determination in step ST4 is YES, i.e., if it is determined that ultrasound imaging should be terminated, the estimation function 172 estimates the organ shape of the entire target organ in the subject and the imaged organ region (step ST5) using ultrasound image data of one or more cross sections arranged in the three-dimensional memory of the image memory 15. The identification function 173 identifies the three-dimensional organ region that has not yet been imaged, based on the partial organ shape included in the ultrasound image data of one or more cross sections arranged three-dimensionally in step ST3 and the entire organ shape estimated in step ST5 (step ST6).

[0096] The identification function 173 extracts organ contours from ultrasound image data of multiple cross sections arranged three-dimensionally, arranges the extracted organ contours three-dimensionally, and compares them with a 3D model of the entire organ.The identification function 173 then identifies, as an unimaged organ region, an area obtained by excluding organ contours included in existing data regions from the organ contours of the 3D model.The 3D model may be generated by obtaining organ contours from volume data such as 3D-CT image data previously captured from the same subject (same patient), or a 3D model showing a general organ shape may be used.

[0097] The identification function 173 determines whether or not an unimaged organ region exists (step ST7). If the determination in step ST7 is YES, that is, if it is determined that an unimaged organ region exists, the display control function 71 displays information about the unimaged organ region on the display 20 for the operator (step ST8). The information about the unimaged organ region may be the range of an ultrasound beam determined to fill the unimaged organ region (display example (1) described below), or may include the body surface position and posture of the ultrasound probe 30 for imaging the unimaged organ region (display examples (2) to (4) described below). Furthermore, the display control function 71 displays information for imaging the unimaged object region, thereby displaying information about the unimaged object region and / or displaying the unimaged object region in three dimensions.

[0098] FIG. 10 is a diagram for explaining a method for determining the body surface position and posture of the ultrasound probe 30 for imaging an organ region that has not been imaged due to missing data.

[0099] FIG. 10 shows the unimaged object region Q shown in FIG. 5(B). In the case of ultrasound imaging, the display control function 71 may select a body surface position and posture that allows the unimaged object region Q to be imaged as shallowly as possible and that has an imaging range, imaging position, and imaging angle similar to those of the existing data region P (for example, the two probe positions at both ends in FIG. 10). The display control function 71 may select the body surface position and posture of the ultrasound probe 30 displayed to image the unimaged object region. The selected body surface position and posture may be one or more. Furthermore, if the positions of parts (e.g., bones) within the subject with large differences in acoustic impedance are known, it is possible to select a body surface position and posture that do not include parts with large differences in acoustic impedance on a cross section determined from the body surface position and posture, or to adjust the body surface position and posture by shifting the body surface position and posture.

[0100] Display examples (1) to (4) of information relating to the unimaged object region are shown below.

[0101] (1) For the unimaged object area on the 3D model (shown in Figure 6(B)), display the range of the ultrasound beam determined to fill the unimaged object area. (2) Project and display the probe mark of the ultrasound probe 30 for imaging the unimaged object region onto the subject. (3) Displaying a probe mark of the ultrasound probe 30 for imaging the unimaged object region on the 3D model of the subject. (4) An LED lamp or the like is attached to the ultrasonic probe 30 itself, and the direction of movement and rotation is indicated by the lamp.

[0102] 11A to 11C are diagrams showing examples of displaying information about object regions that have not been imaged due to missing data.

[0103] FIG. 11(A) shows the above display example (1). FIG. 11(B) shows the above display example (2). FIG. 11(C) shows the above display example (3). In FIG. 11(A), the upper base (the shorter of the two base sides) of the trapezoid formed by the thick lines corresponds to the body surface position. In FIG. 11(B), mark M1 is a probe mark projected onto the body surface of the subject, and mark M2 is a mark of a marker attached to one side of the ultrasound probe 30, which is projected onto the body surface of the subject. In FIG. 11(C), mark M1 is a probe mark on the 3D model displayed on the display 20, and mark M2 is a mark of a marker attached to one side of the ultrasound probe 30, which is on the 3D model displayed on the display 20.

[0104] Note that an unimaged object region may be projected onto the body surface of the subject together with the marks M1 and M2. The position of the unimaged object region is determined from a plurality of ultrasound image data sets including the imaged object region and their position information. This is because the position of the entire object region can be determined from a plurality of ultrasound image data sets including the imaged object region and their position information.

[0105] The operator checks the information about the unimaged object region displayed in step ST8 and continues the examination as necessary. Specifically, the operator presses the tip of the ultrasonic probe 30 against the position on the body surface of the subject that corresponds to the position on the body surface of the subject (the upper base of the trapezoid) displayed on the display 20. Alternatively, the operator presses the tip of the ultrasonic probe 30 against the mark M1 (shown in FIG. 11(B)) so that the marker coincides with the mark M2 (shown in FIG. 11(B)) projected on the body surface of the subject. Alternatively, the operator presses the tip of the ultrasonic probe 30 against the mark M1 (shown in FIG. 11(C)) so that the marker coincides with the mark M2 (shown in FIG. 11(C)) at the position on the body surface of the subject displayed on the display 20. By pressing the ultrasonic probe 30 so that the markers attached to the ultrasonic probe 30 align, the operator can align the position of the ultrasonic probe 30 in the rotational direction.

[0106] Furthermore, a probe model for imaging an unimaged object region can be displayed on the 3D model of the subject, and arrows can be used to indicate the directions in which the ultrasound probe 30 should be moved and rotated relative to the probe model. The unimaged object region may also be displayed on the probe model together with the arrows.

[0107] Returning to the explanation of FIG. 9, if the determination in step ST7 is NO, that is, if it is determined that there is no organ region that has not yet been imaged, the examination is terminated.

[0108] As described above, according to the first operation example of the ultrasound diagnostic device 1, when data loss occurs due to incomplete imaging, the operator can be prompted after imaging is completed to perform additional imaging of the unimaged object region due to the data loss. Information about the unimaged object region can also be presented to the operator. For example, the unimaged object region itself can be displayed as information about the unimaged object region, and the position and angle at which the ultrasound probe 30 should be placed on the subject's body surface can also be displayed. Furthermore, the unimaged object region does not need to be supplemented with other images to fill in the unimaged object region. This avoids a situation where it is determined that additional imaging of the unimaged object region is necessary after the subject has left the examination.

[0109] FIG. 12 is a flowchart illustrating a second operation example of the ultrasound diagnostic device 1. In FIG. 12, the symbols "ST" followed by numbers indicate the steps of the flowchart. Unlike the first operation example of the ultrasound diagnostic device 1, the second operation example of the ultrasound diagnostic device 1 sequentially identifies object regions that have not yet been imaged. Note that FIG. 12 will be described using a case where the object to be imaged is an organ.

[0110] In FIG. 12, the same steps as those shown in FIG. 9 are denoted by the same reference numerals and the description thereof will be omitted.

[0111] Similar to step ST5 (shown in FIG. 9), the estimation function 172 estimates the organ shape of the entire target organ in the subject and the imaged organ region (step ST15) using ultrasound image data of one or more cross sections arranged in the three-dimensional memory of the image memory 15. Similar to step ST6 (shown in FIG. 9), the identification function 173 identifies the three-dimensional unimaged organ region based on the partial organ shape included in the ultrasound image data of one or more cross sections arranged three-dimensionally in step ST3 and the entire organ shape estimated in step ST5 (step ST16).

[0112] The display control function 71 determines whether the ratio of the volume and contour of the unimaged organ region to the volume and contour calculated from the overall organ shape estimated in step ST15 is equal to or less than a threshold value (e.g., 20%) (step ST17). If the determination in step ST17 is YES, i.e., if it is determined that the ratio of the volume and contour of the unimaged organ region is equal to or less than the threshold value, the display control function 71 displays on the display 20 that there is little data loss and that the data is sufficient (step ST18).

[0113] On the other hand, if the judgment in step ST17 is NO, i.e., if it is determined that the proportion of the volume of the unimaged organ area exceeds the threshold, the display control function 71 causes the operator to display information about the unimaged organ area on the display 20 (step ST19), as in step ST8 (shown in Figure 9).

[0114] As described above, according to the second operation example of the ultrasound diagnostic device 1, the ratio of the volume and contour of the unimaged object region to the volume and contour calculated from the estimated overall object shape, as well as its position and range, can be presented to the operator in real time during imaging. This allows the operator to proceed with imaging while checking the ratio of the unimaged object region, thereby avoiding a situation in which the operator has to determine that additional imaging should be performed on the unimaged object region after the subject has left after the examination.

[0115] Although the above description deals with data loss due to incomplete imaging, the present invention is not limited to this case. For example, the present invention can be applied to cases where there is no data loss but an artifact exists on the ultrasound image data, making it impossible to visually identify a portion of the tissue that should be visualized. In this case, the artifact portion on the ultrasound image data can be detected and designated as a data loss area.

[0116] Furthermore, after an unimaged object region is identified, virtual ultrasound image data that would result if the unimaged object region were imaged can be displayed as a model. The virtual ultrasound image data may be generated by combining pixel values of existing data regions that are close to the unimaged object region, or, if volume data of a CT image or the like already exists, may be an MPR image generated from that volume data.

[0117] 2. First Modification The object of the subject imaged by the ultrasound probe 30 may be an abnormal object. For example, the object of the subject imaged may be an object containing a tumor. In this case, the estimation function 172 estimates the object shape and the imaged object region within the same subject based on past medical image data (e.g., ultrasound image data) of the same subject using ultrasound image data of the multiple cross sections to be imaged and their position information. Then, the identification function 173 identifies the unimaged object region based on the object shape estimated by the estimation function 172 and the imaged object region. The identification function 173 can extract the position of the tumor based on the past medical image data and can identify the position of the data region based on the position information of the ultrasound image data of the multiple cross sections to be imaged.

[0118] As a result, when the tumor portion has not been imaged, the output control function 174 can output, to the outside of the processing circuit 17, three-dimensional information of the unimaged object area including the tumor identified by the identification function 173, or information for imaging the unimaged object area including the tumor.

[0119] On the other hand, if the tumor portion has already been imaged, the identification function 173 determines the degree of data loss in the tumor in the imaged object region. If it is determined that the degree of data loss in the tumor is equal to or greater than the threshold, the output control function 174 outputs information for imaging the tumor in the imaged object region to the outside of the processing circuit 17. Display examples of information for imaging the tumor are the same as the above-mentioned "Display examples (1) to (4) of information regarding unimaged object regions." This allows the operator to be prompted to reimage the tumor at short intervals or from an appropriate direction. On the other hand, if it is determined that the degree of data loss in the tumor is less than the threshold and the data related to the tumor is sufficient, the output control function 174 allows the end of ultrasound imaging.

[0120] 3. Second Modification The following describes a case where the transmission control function 73 of the output control function 174 transmits three-dimensional information about the unimaged object area identified by the identification function 173 and / or information for imaging the unimaged object area to an external device of the ultrasound diagnostic device 1.

[0121] FIG. 13 is a schematic diagram showing the configuration of a medical image system including an ultrasound diagnostic apparatus as an example of a medical image diagnostic apparatus according to the second modification.

[0122] 13 shows a medical image system S including an ultrasound diagnostic apparatus 1 as a medical image diagnostic apparatus. The medical image system S includes the ultrasound diagnostic apparatus 1 described above and a medical image display device 80 as a medical image processing device. The medical image display device 80 is a workstation that performs various image processing operations on medical image data, a portable information processing terminal such as a tablet terminal, or the like, and is connected to the ultrasound diagnostic apparatus 1 so as to be able to communicate with each other via a network N.

[0123] The medical image display device 80 includes a network interface 86, a processing circuitry 87, a memory 88, an input interface 89, and a display 90. The network interface 86, the processing circuitry 87, the memory 88, the input interface 89, and the display 90 have the same configurations as the network interface 16, the processing circuitry 17, the main memory 18, the input interface 19, and the display 20 shown in Fig. 1, respectively, and therefore descriptions thereof will be omitted.

[0124] The processing circuitry 87 realizes the display control function 71A by reading and executing a computer program stored in the memory 88 or directly incorporated in the processing circuitry 87. The following description will be given taking as an example a case where the function 71A functions as software, but all or part of the function 71A may be provided in the medical image display device 80 as a function of a circuit such as an ASIC.

[0125] The display control function 71A receives ultrasound image data acquired by the acquisition function 171 of the ultrasound diagnostic device 1 and displays it on the display 90, and also includes a function to receive information about an unimaged object region when the unimaged object region is identified by the identification function 173 of the ultrasound diagnostic device 1 and display it on the display 90. The display control function 71A can also generate 3D hologram image data (image data that can be viewed stereoscopically), display it on a 3D hologram display as the display 90, and project it onto the body surface of the subject.

[0126] With the configuration shown in FIG. 13, the medical image display device 80, which is an external device of the ultrasound diagnostic device 1, can display three-dimensional information of an unimaged object region and / or information for imaging the unimaged object region.

[0127] According to at least one of the embodiments described above, it is possible to identify unimaged object regions in medical image data. Furthermore, by reducing the number of unimaged regions, it is possible to reduce differences between operators and improve efficiency.

[0128] The acquisition function 171 is an example of an acquisition unit. The estimation function 172 is an example of an estimation unit. The identification function 173 is an example of an identification unit. The output control function 174 is an example of an output control unit. The movement control function 175 is an example of a movement control unit. The display control functions 71 and 71A are examples of a display control unit. The memory control function 72 is an example of a memory control unit. The transmission control function 73 is an example of a transmission control unit.

[0129] In the above-described embodiment, the case where imaged organ regions and unimaged organ regions are identified is mainly focused on organs within a subject. However, organs are merely an example of an object, and similar identification may be performed on objects within the subject other than organs. For example, instead of organs, the object may be a surgical device such as a stent inserted into the subject, a lesion such as a tumor or lymphoma, or a muscle layer. The shape of these scanned objects may be collectively referred to as the "object shape," the imaged region may be referred to as the "imaged object region," and the unimaged region may be referred to as the "unimaged object region."

[0130] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0131] 1. Medical imaging diagnostic equipment (ultrasound diagnostic equipment) 10 Medical image processing device (device body) 17 Processing circuit 30 Imaging device (ultrasound probe) 40 Position Sensor 171 Acquisition Function 172 Guessing Function 173 Specific Functions 174 Output control function 175 Movement control function 71,71A Display control function 72 Memory control function 73 Transmission Control Function 80 Medical image processing device (medical image display device) S Medical Imaging System

Claims

1. an acquisition unit that acquires a plurality of medical cross-sectional images obtained by imaging an object of a subject and information on the position and orientation of a probe corresponding to each of the plurality of medical cross-sectional images; an identifying unit that identifies an unimaged region of the object that is not included in the plurality of medical tomographic images, using the plurality of acquired medical tomographic images and information related to the position and orientation of the probe; a display control unit that displays, on a display unit, position information of the unimaged region and information about the position and orientation of a probe for imaging the unimaged region at a shallow part; A medical imaging diagnostic device having:

2. an acquisition unit that acquires a plurality of medical cross-sectional images obtained by imaging an object of a subject and information on the position and orientation of a probe corresponding to each of the plurality of medical cross-sectional images; an identifying unit that identifies an unimaged region of the object that is not included in the plurality of medical tomographic images, using the plurality of acquired medical tomographic images and information related to the position and orientation of the probe; a display control unit that causes a display unit to display position information of the unimaged region and information about the position and orientation of a probe for imaging the unimaged region; and the display control unit causes the display unit to display a range of the beam determined to fill the unimaged region. Medical imaging diagnostic equipment.

3. the display control unit further causes the display unit to display information regarding the pressure of the probe for imaging the unimaged region.

3. The medical image diagnostic apparatus according to claim 1.

4. the display control unit further displays, on the display unit, information about the position and posture of the probe having an imaging position and imaging angle similar to those of the already existing data area. The medical image diagnostic apparatus according to claim 1 .

5. the display control unit causes the display unit to display information about the position and posture of the probe when there is no region on the medical tomographic image where the difference in acoustic impedance is greater than a predetermined value.

3. The medical image diagnostic apparatus according to claim 1.

6. further comprising an estimation unit; the estimation unit estimates an organ shape and an imaged organ region in the subject based on the acquired medical cross-sectional images and information related to the position and orientation of the probe; the identifying unit identifies an area, obtained by excluding an imaged organ area from the entire estimated organ area, as the unimaged area.

3. The medical image diagnostic apparatus according to claim 1.

7. an acquisition unit that acquires a plurality of ultrasonic cross-sectional images obtained by imaging an object of a subject by scanning with an ultrasonic probe, and information about the position and orientation of the ultrasonic probe corresponding to each of the plurality of ultrasonic cross-sectional images; an identification unit that identifies an unimaged region of the object that is not included in the plurality of ultrasonic cross-sectional images, using the plurality of acquired ultrasonic cross-sectional images and information regarding the position and orientation of the ultrasonic probe; a display control unit that displays, on a display unit, position information of the unimaged region and information about the position and orientation of an ultrasound probe for imaging the unimaged region at a shallow depth; An ultrasound diagnostic device having:

8. an acquisition unit that acquires a plurality of ultrasonic cross-sectional images obtained by imaging an object of a subject by scanning with an ultrasonic probe, and information about the position and orientation of the ultrasonic probe corresponding to each of the plurality of ultrasonic cross-sectional images; an identification unit that identifies an unimaged region of the object that is not included in the plurality of ultrasonic cross-sectional images, using the plurality of acquired ultrasonic cross-sectional images and information regarding the position and orientation of the ultrasonic probe; a display control unit that displays, on a display unit, position information of the unimaged region and information about the position and orientation of an ultrasound probe for imaging the unimaged region; and the display control unit causes the display unit to display a range of the beam determined to fill the unimaged region. Ultrasound diagnostic equipment.

9. the display control unit further causes the display unit to display information regarding the pressure of the ultrasonic probe for imaging the unimaged region.

9. The ultrasonic diagnostic apparatus according to claim 7 or 8.

10. the display control unit further displays, on the display unit, information about the position and posture of the ultrasound probe that is similar to the imaging position and imaging angle of the already existing data area. The ultrasonic diagnostic apparatus according to claim 7.

11. the display control unit causes the display unit to display information about the position and posture of the ultrasonic probe, where no region exists on the ultrasonic cross-sectional image where the difference in acoustic impedance is greater than a predetermined value.

9. The ultrasonic diagnostic apparatus according to claim 7 or 8.

12. further comprising an estimation unit; the estimation unit estimates an organ shape and an imaged organ region in the subject based on the acquired multiple ultrasound cross-sectional images and information related to the position and orientation of the ultrasound probe; the identifying unit identifies an area, obtained by excluding an imaged organ area from the entire estimated organ area, as the unimaged area.

9. The ultrasonic diagnostic apparatus according to claim 7 or 8.

13. A medical image system including both a medical image diagnostic device and a medical image processing device, the medical image diagnostic device and the medical image processing device being connected to each other so as to be able to communicate with each other via a network, the medical image diagnostic apparatus has an acquisition unit that acquires a plurality of medical cross-sectional images obtained by imaging an object of a subject and information on the position and orientation of a probe corresponding to each of the plurality of medical cross-sectional images; The medical image processing device, an identifying unit that identifies an unimaged region of the object that is not included in the plurality of medical tomographic images, using the plurality of acquired medical tomographic images and information related to the position and orientation of the probe; a display control unit that causes a display unit to display position information of the unimaged region and information about the position and orientation of a probe for imaging the unimaged region; having Medical imaging systems.

14. acquiring a plurality of medical cross-sectional images obtained by imaging an object of a subject and information on the position and orientation of a probe corresponding to each of the plurality of medical cross-sectional images; using the acquired plurality of medical cross-sectional images and information on the position and orientation of the probe, to identify an unimaged region of the object that is not included in the plurality of medical cross-sectional images; displaying, on a display unit, position information of the unimaged region and information on the position and orientation of a probe for imaging the unimaged region at a shallow depth; Imaging control method.

15. acquiring a plurality of medical cross-sectional images obtained by imaging an object of a subject and information on the position and orientation of a probe corresponding to each of the plurality of medical cross-sectional images; using the acquired plurality of medical cross-sectional images and information on the position and orientation of the probe, to identify an unimaged region of the object that is not included in the plurality of medical cross-sectional images; displaying, on a display unit, position information of the unimaged region and information on the position and orientation of a probe for imaging the unimaged region; displaying the range of the beam determined to fill the unimaged region on the display unit; Imaging control method.

Images