Ultrasonic diagnostic apparatus and control method of ultrasonic diagnostic apparatus

Abstract

To provide an ultrasonic diagnostic apparatus and a control method of an ultrasonic diagnostic apparatus that allow a user to perform comprehensive scanning of an organ reliably.SOLUTION: An ultrasonic diagnostic apparatus includes: an ultrasonic probe (1); a position posture sensor for acquiring position posture information on the ultrasonic probe (1); an image acquisition unit (32) for acquiring an ultrasonic image of a subject; a contour detection unit (25) for detecting the contour of an organ captured in the ultrasonic image; a region detection unit (27) for detecting a region of the organ captured in the ultrasonic image; a contour data generation unit (26) for generating three-dimensional image data on the contour on the basis of the detected contour and the position posture information; a region data generation unit (28) for generating three-dimensional image data on the region on the basis of the detected region and the position posture information; and a monitor (23) for sequentially displaying images showing the contour and the region on the basis of the three-dimensional image data on the contour and the region.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an ultrasonic diagnostic apparatus that comprehensively scans the inside of a subject, and a method for controlling the ultrasonic diagnostic apparatus. [Background technology]

[0002] Conventionally, examinations have been performed by capturing ultrasound images showing cross-sectional images of a subject using a so-called ultrasound diagnostic device. In such examinations, a user such as a doctor usually captures ultrasound images while changing the position of a so-called ultrasound probe in contact with the body surface of the subject and moving the ultrasound probe.

[0003] In this case, the user usually takes ultrasound images sequentially while checking the captured ultrasound images to determine the scanned portion of the subject, but a user who is not very skilled in examinations using an ultrasound diagnostic device may have difficulty in determining whether the subject's target site has been comprehensively scanned. Therefore, as disclosed in Patent Document 1, for example, a technology has been developed that extracts the contours of the organ to be examined from multiple frames of ultrasound images captured while moving an ultrasound probe, and displays areas where the interval between the extracted contours is equal to or greater than a certain value as areas that have not yet been scanned. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-104137 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when trying to determine whether comprehensive scanning has been performed by extracting the contour of an organ as in Patent Document 1, it may be determined that comprehensive scanning has been completed even if there is an unscanned area inside the contour, for example. In this case, a user with low skill level may complete the examination even if the organ has not been comprehensively scanned.

[0006] The present invention has been made to solve such conventional problems, and an object of the present invention is to provide an ultrasound diagnostic apparatus and a method for controlling an ultrasound diagnostic apparatus that enable a user to reliably perform comprehensive scanning of organs. [Means for solving the problem]

[0007] The above object can be achieved by the following configuration. [1] An ultrasonic probe; a position and orientation sensor for acquiring position and orientation information of an ultrasonic probe; an image acquisition unit that acquires an ultrasonic image of a subject by transmitting and receiving an ultrasonic beam using an ultrasonic probe; a contour detection unit that detects a contour of an organ imaged in the ultrasound image based on the ultrasound image; an area detection unit that detects an area of ​​an organ imaged in the ultrasound image based on the ultrasound image; a contour data generating unit that generates three-dimensional image data of the contour of the organ based on the contour detected by the contour detecting unit and the position and orientation information acquired by the position and orientation sensor; an area data generating unit that generates three-dimensional image data of the organ area based on the area detected by the area detecting unit and the position and orientation information acquired by the position and orientation sensor; a monitor for sequentially displaying an image representing the contour and an image representing the region based on the three-dimensional image data of the contour and the three-dimensional image data of the region; An ultrasound diagnostic device comprising: [2] The contour detection unit detects contours using a contour detection model that has been trained to detect contours of organs in ultrasound images of the organs; The ultrasound diagnostic device of claim 1, wherein the region detection unit detects regions using a region detection model that has been trained to detect organ regions in ultrasound images of the organ. [3] an unimaged portion determining unit that determines a portion whose contour is not detected by the contour detection unit or a portion whose region is not detected by the region detection unit as an unimaged portion; a notification unit that notifies a user when the unimaged portion determination unit determines that there is an unimaged portion; The ultrasound diagnostic apparatus according to claim 1 or 2, [4] An ultrasound diagnostic device as described in [3], wherein the unimaged portion determination unit determines that a discontinuous portion is an unimaged portion when the contour detected by the contour detection unit or the area detected by the area detection unit has a discontinuous portion having a predetermined interval or more. [5] A non-contact detection unit is provided to detect a non-contact state in which the ultrasonic probe is separated from the body surface of the subject, The ultrasound diagnostic device according to claim 3, wherein the non-imaged portion determination unit determines the presence of an unimaged portion when the non-contact detection unit detects a non-contact state. [6] The ultrasound diagnostic device according to [3], wherein the unimaged portion determination unit determines the presence of an unimaged portion when a predetermined time has elapsed since the start of scanning by the ultrasound probe. [7] The ultrasound diagnostic device according to [3], wherein the unimaged portion determination unit determines the presence of an unimaged portion when a user gives an instruction to end the examination. [8] An ultrasound diagnostic device according to [3], wherein the unimaged portion determination unit determines the presence of an unimaged portion whose area has not been detected by the area detection unit when the contour detected by the contour detection unit is closed. [9] An ultrasound diagnostic device according to any one of [3] to [8], wherein the notification unit notifies a user that scanning is complete when the contour detected by the contour detection unit is closed and there is no area within the interior of the contour that has not been detected by the area detection unit.

[10] The monitor displays a three-dimensional schematic image; The ultrasound diagnostic device according to any one of [1] to [9], further comprising an emphasis display unit that highlights the contours detected by the contour detection unit and the regions detected by the region detection unit on a three-dimensional schematic image.

[11] A reference contour memory in which a reference contour to be rendered is stored, The ultrasound diagnostic device according to any one of [3] to [8], wherein the notification unit notifies the user to move the ultrasound probe when the contour detection unit does not detect a contour corresponding to the reference contour stored in the reference contour memory.

[12] The ultrasound diagnostic device according to any one of [1] to

[11] , wherein the monitor sequentially displays a three-dimensional image of the contour and a three-dimensional image of the region as an image representing the contour and an image representing the region.

[13] A tomographic image extraction unit extracts two-dimensional tomographic images of the contour and two-dimensional tomographic images of the region on the same cross section from the three-dimensional image data of the contour generated by the contour data generation unit and the three-dimensional image data of the region generated by the region data generation unit, The ultrasound diagnostic device according to any one of claims [1] to

[12] , wherein the monitor sequentially displays a two-dimensional tomographic image of the contour and a two-dimensional tomographic image of the region extracted by the tomographic image extraction unit as an image representing the contour and an image representing the region.

[14] The ultrasonic diagnostic device according to any one of [1] to

[13] , wherein the monitor displays an image representing the contour and an image representing the region in a superimposed manner.

[15] The ultrasonic diagnostic device according to any one of [1] to

[13] , wherein the monitor displays an image representing a contour and an image representing a region side by side.

[16] The ultrasonic diagnostic device according to any one of [1] to

[15] , wherein the position and orientation sensor includes an inertial sensor, a magnetic sensor, or an optical sensor.

[17] Acquire position and orientation information of an ultrasound probe; obtaining an ultrasound image of the subject by transmitting and receiving an ultrasound beam using an ultrasound probe; Detecting a contour of an organ imaged in the ultrasound image based on the ultrasound image; Detecting an area of ​​an organ imaged in the ultrasound image based on the ultrasound image; generating three-dimensional image data of the organ contour based on the detected contour and the acquired position and orientation information; generating three-dimensional image data of the organ region based on the detected region and the acquired position and orientation information; An image representing the contour and an image representing the region are sequentially displayed on a monitor based on the three-dimensional image data of the contour and the three-dimensional image data of the region. A method for controlling an ultrasonic diagnostic device. Effect of the Invention

[0008] In the present invention, the ultrasound diagnostic apparatus includes an ultrasound probe, a position and orientation sensor for acquiring position and orientation information of the ultrasound probe, an image acquisition unit for acquiring an ultrasound image of a subject by transmitting and receiving an ultrasound beam using the ultrasound probe, a contour detection unit for detecting a contour of an organ imaged in the ultrasound image based on the ultrasound image, a region detection unit for detecting a region of the organ imaged in the ultrasound image based on the ultrasound image, a contour data generation unit for generating three-dimensional image data of the contour of the organ based on the contour detected by the contour detection unit and the position and orientation information acquired by the position and orientation sensor, a region data generation unit for generating three-dimensional image data of the region of the organ based on the region detected by the region detection unit and the position and orientation information acquired by the position and orientation sensor, and a monitor for sequentially displaying an image representing the contour and an image representing the region based on the three-dimensional image data of the contour and the three-dimensional image data of the region, thereby enabling a user to reliably perform comprehensive scanning of the organ. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an ultrasound diagnostic apparatus according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a configuration of a transmission / reception circuit in a first embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram showing a configuration of an image generating unit according to the first embodiment of the present invention. [Figure 4] 1 is a schematic diagram of an ultrasound probe on a subject's body surface and the organ being scanned. [Diagram 5] 4 is a display example of the contour and area of ​​an organ in the first embodiment. [Figure 6] 5 is a flowchart showing the operation of the ultrasound diagnostic apparatus according to the first embodiment of the present invention. [Figure 7] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a second embodiment of the present invention. [Figure 8] FIG. 13 is a diagram showing an example of a three-dimensional grid set in three-dimensional image data of an organ. [Figure 9] FIG. 2 is a partially enlarged view of a three-dimensional grid. [Figure 10] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a third embodiment of the present invention. [Figure 11] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a fourth embodiment of the present invention. [Figure 12] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a fifth embodiment of the present invention. [Figure 13] FIG. 13 is a diagram showing an example of a three-dimensional schematic image according to the fifth embodiment of the present invention. [Figure 14] FIG. 13 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a sixth embodiment of the present invention. [Figure 15] 23 is another display example of the contour and area of ​​an organ in the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The following description of the components will be given based on a representative embodiment of the present invention, but the present invention is not limited to such an embodiment. In this specification, a numerical range expressed using "to" means a range that includes the numerical values ​​before and after "to" as the lower and upper limits. In this specification, the terms "same" and "identical" include a generally acceptable margin of error in the technical field.

[0011] First embodiment 1 shows the configuration of an ultrasonic diagnostic device according to a first embodiment of the present invention. The ultrasonic diagnostic device includes an ultrasonic probe 1 and a device main body 2 connected to the ultrasonic probe 1. The ultrasonic probe 1 and the device main body 2 are connected to each other by so-called wired communication or so-called wireless communication.

[0012] The ultrasonic probe 1 has a transducer array 11. A transmission / reception circuit 12 is connected to the transducer array 11. The ultrasonic probe 1 also has a position and orientation sensor 13. The position and orientation sensor 13 may be built into the ultrasonic probe 1, or may be attached to a housing of the ultrasonic probe 1. Furthermore, when a sensor device that measures the ultrasonic probe 1 from the outside, such as a so-called optical sensor, is used as the position and orientation sensor 13, the position and orientation sensor 13 may be disposed on the outside of the ultrasonic probe 1.

[0013] The device main body 2 has an image generating unit 21 connected to the transmitting / receiving circuit 12 of the ultrasound probe 1. A display control unit 22 and a monitor 23 are connected to the image generating unit 21, in that order. An image memory 24 is connected to the image generating unit 21. A contour detection unit 25 is connected to the image memory 24. A contour data generating unit 26 is connected to the position and orientation sensor 13 and the contour detection unit 25. The contour data generating unit 26 is connected to the display control unit 22. In addition, a region detection unit 27 is connected to the image memory 24. A region data generating unit 28 is connected to the position and orientation sensor 13 and the region detection unit 27. The region data generating unit 28 is connected to the display control unit 22.

[0014] A main body control unit 29 is connected to the image generating unit 21, the display control unit 22, the image memory 24, the contour detecting unit 25, the contour data generating unit 26, the area detecting unit 27, and the area data generating unit 28. An input device 30 is connected to the main body control unit 29. A processor 31 for the device main body 2 is configured by the image generating unit 21, the display control unit 22, the contour detecting unit 25, the contour data generating unit 26, the area detecting unit 27, the area data generating unit 28, and the main body control unit 29.

[0015] The transducer array 11 of the ultrasonic probe 1 has a plurality of ultrasonic transducers arranged one-dimensionally or two-dimensionally. Each of these ultrasonic transducers transmits ultrasonic waves according to a drive signal supplied from the transmission / reception circuit 12, receives ultrasonic echoes from the subject, and outputs a signal based on the ultrasonic echoes. Each ultrasonic transducer is configured by forming electrodes on both ends of a piezoelectric body made of, for example, a piezoelectric ceramic represented by PZT (Lead Zirconate Titanate), a polymeric piezoelectric element represented by PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal represented by PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0016] The transmission / reception circuit 12, under the control of the main body control unit 29, transmits ultrasonic waves from the transducer array 11 and generates sound ray signals based on reception signals acquired by the transducer array 11. As shown in Fig. 2, the transmission / reception circuit 12 has a pulser 41 connected to the transducer array 11, and an amplifier 42, an AD (Analog to Digital) converter 43, and a beamformer 44, which are connected in series from the transducer array 11 in this order.

[0017] The pulser 41 includes, for example, a plurality of pulse generators, and adjusts the delay amount of each drive signal and supplies it to the plurality of ultrasonic transducers of the transducer array 11 so that ultrasonic waves transmitted from the plurality of ultrasonic transducers form ultrasonic beams based on a transmission delay pattern selected in response to a control signal from the main body control unit 29. In this way, when a pulsed or continuous wave voltage is applied to the electrodes of the ultrasonic transducers of the transducer array 11, the piezoelectric body expands and contracts, and pulsed or continuous wave ultrasonic waves are generated from each ultrasonic transducer, and an ultrasonic beam is formed from the composite wave of those ultrasonic waves.

[0018] The transmitted ultrasonic beam is reflected by an object such as a part of the subject, and propagates toward the transducer array 11 of the ultrasonic probe 1. The ultrasonic echo propagating toward the transducer array 11 in this manner is received by each ultrasonic transducer constituting the transducer array 11. At this time, each ultrasonic transducer constituting the transducer array 11 expands and contracts upon receiving the propagating ultrasonic echo, generating a received signal which is an electrical signal, and outputs this received signal to the amplifier 42.

[0019] The amplifier 42 amplifies the signals input from the respective ultrasonic transducers constituting the transducer array 11, and transmits the amplified signals to the AD converter 43. The AD converter 43 converts the signals transmitted from the amplifier 42 into digital reception data. The beamformer 44 performs so-called reception focusing processing by adding each piece of reception data received from the AD converter 43 with a respective delay. Through this reception focusing processing, each piece of reception data converted by the AD converter 43 is phased and added, and a sound ray signal in which the focus of the ultrasonic echo is narrowed is acquired.

[0020] As shown in FIG. 3, the image generating unit 21 has a configuration in which a signal processing unit 45, a DSC (Digital Scan Converter) 46, and an image processing unit 47 are connected in series.

[0021] The signal processing unit 45 performs correction for attenuation due to distance on the sound ray signals received from the transmission / reception circuit 12 according to the depth of the ultrasonic reflection position using the sound velocity value set by the main body control unit 29, and then performs envelope detection processing to generate a B-mode image signal, which is tomographic image information on the tissue in the subject.

[0022] The DSC 46 converts (raster converts) the B-mode image signal generated by the signal processor 45 into an image signal conforming to a normal television signal scanning method. The image processing unit 47 performs various necessary image processing such as gradation processing on the B-mode image signal input from the DSC 46, and then sends the B-mode image signal to the display control unit 22 and the image memory 24. Hereinafter, the B-mode image signal that has been subjected to image processing by the image processing unit 47 will be referred to as an ultrasound image.

[0023] The image memory 24 is a memory that stores an ultrasonic image acquired by the image acquisition unit 32. As the image memory 24, for example, a recording medium such as a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an FD (Flexible Disk), an MO disk (Magneto-Optical disk), an MT (Magnetic Tape), a RAM (Random Access Memory), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SD card (Secure Digital card), or a USB memory (Universal Serial Bus memory) can be used.

[0024] The position and orientation sensor 13 of the ultrasonic probe 1 is a sensor device that acquires the position and orientation information of the ultrasonic probe 1. Generally, when a user uses an ultrasonic diagnostic device to inspect a subject, the user often performs the inspection while changing the orientation, i.e., angle, of the ultrasonic probe 1 in contact with the body surface of the subject and moving the position of the ultrasonic probe 1. The position and orientation information of the ultrasonic probe 1 acquired by the position and orientation sensor 13 includes information on the orientation and position of the ultrasonic probe 1. The position and orientation sensor 13 may include at least one of a so-called inertial sensor, a magnetic sensor, or an optical sensor, for example. The inertial sensor may include at least one of a so-called acceleration sensor and a gyro sensor, for example.

[0025] The contour detection unit 25 detects the contour of an organ captured in an ultrasound image based on the ultrasound image acquired by the image acquisition unit 32. The contour detection unit 25 can detect the contour of an organ, for example, by using a contour detection model that has previously learned the contour of the organ in an ultrasound image of the organ.

[0026] Here, the contour detection model is a so-called machine learning model, and as the contour detection model, a so-called deep learning model, a so-called SVM (Support Vector Machine), a so-called decision tree model, etc. can be used. The contour detection model learns in advance the relationship between a large number of ultrasound images in which an organ is captured and the contours of the organ in the ultrasound images, and when an ultrasound image in which an organ is captured is input, the contour detection model outputs the contour of the organ in the ultrasound image.

[0027] In addition to the method using the contour detection model, the contour detection unit 25 can also detect the contours of organs using, for example, the so-called SIFT (Scale-Invariant Feature Transform) method, an edge detection method based on changes in brightness in ultrasound images, etc.

[0028] The contour data generating unit 26 generates three-dimensional image data of the contour of the organ based on the contour detected by the contour detecting unit 25 and the position and orientation information acquired by the position and orientation sensor 13. At this time, the contour data generating unit 26 can generate three-dimensional image data of the contour of the organ by, for example, associating the contour of the organ detected by the contour detecting unit 25 with the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13 at the timing when the ultrasound image in which the contour of the organ is detected is acquired, and sequentially arranging the contour of the organ in three-dimensional space according to the corresponding position and orientation information.

[0029] The contour data generating unit 26 can, for example, associate the position and orientation information acquired by the position and orientation sensor 13 with the contour that was being detected by the contour detecting unit 25 when the position and orientation information was acquired. In addition, when the position and orientation sensor 13 assigns a time stamp representing the acquisition time to the position and orientation information and the image acquiring unit 32 assigns a time stamp representing the acquisition time to the ultrasound image, the contour data generating unit 26 can associate the position and orientation information with the contour of the organ detected from the ultrasound image when the difference between the time stamp assigned to the position and orientation information and the time stamp assigned to the ultrasound image is within a certain range. For example, when the generation rate of the ultrasound image by the image acquiring unit 32 is slower than the acquisition rate of the position and orientation information by the position and orientation sensor 13, the contour data generating unit 26 can associate the position and orientation information having a time stamp representing a time close to the time stamp of the ultrasound image with the contour detected from the ultrasound image, based on the time stamp of the ultrasound image.

[0030] The region detection unit 27 detects the region of the organ captured in the ultrasound image based on the ultrasound image acquired by the image acquisition unit 32. Here, the region of the organ refers to a portion constituted by the tissue of the organ. The region detection unit 27 can detect the region of the organ, for example, by using a region detection model that has been trained on the region of the organ in the ultrasound image in which the organ is captured.

[0031] Here, the region detection model is a machine learning model that performs so-called segmentation processing on ultrasound images, and the region detection model can be a deep learning model, a support vector machine (SVM), a decision tree model, etc. The region detection model learns in advance the relationship between a large number of ultrasound images in which organs are captured and the regions of the organs in those ultrasound images, and when an ultrasound image in which an organ is captured is input, the region of the organ in that ultrasound image is output for each pixel.

[0032] The region data generating unit 28 generates three-dimensional image data of the region of the organ based on the region of the organ detected by the region detecting unit 27 and the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13. At this time, the region data generating unit 28 can generate three-dimensional image data of the region of the organ by, for example, associating the region of the organ detected by the region detecting unit 27 with the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13 at the timing when the ultrasound image in which the region of the organ is detected is acquired, and sequentially arranging the regions of the organ in three-dimensional space according to the corresponding position and orientation information.

[0033] The region data generating unit 28 can, for example, associate the position and orientation information input from the position and orientation sensor 13 with the region of the organ input at the time when the time required for the region detection by the region detection unit 27 to detect the region has elapsed since the position and orientation information was input. In addition, when the position and orientation sensor 13 assigns a timestamp representing the acquisition time to the position and orientation information and the image acquiring unit 32 assigns a timestamp representing the acquisition time to the ultrasound image, the region data generating unit 28 can associate the position and orientation information with the region of the organ detected from the ultrasound image when, for example, the difference between the timestamp assigned to the position and orientation information and the timestamp assigned to the ultrasound image is within a certain range.

[0034] Under the control of the main body control unit 29, the display control unit 22 performs predetermined processing on the ultrasound image sent from the image acquisition unit 32, the image based on the three-dimensional image data of the organ contour generated by the contour data generation unit 26, and the image based on the three-dimensional image data of the organ region generated by the region data generation unit 28, and displays them on the monitor 23.

[0035] Here, for example, when ultrasound images of an organ Q in a subject are sequentially captured while changing the posture of the ultrasound probe 1 as shown in FIG. 4, multiple frames of ultrasound images corresponding to different postures of the ultrasound probe 1 are acquired, and the contour data generating unit 26 sequentially generates three-dimensional image data of the contour of the organ Q in the scanning range of the ultrasound probe 1, and the region data generating unit 28 sequentially generates three-dimensional image data of the region of the organ Q in the scanning range of the ultrasound probe 1.

[0036] At this time, the display control unit 22 can cumulatively display on the monitor 23 an image representing the three-dimensional shape of the contour C and an image representing the three-dimensional shape of the region R1 based on three-dimensional image data of the contour C of the organ Q and three-dimensional image data of the region R1 of the organ Q, which are respectively generated sequentially, as shown in, for example, Fig. 5. For example, the display control unit 22 can impart transparency and different colors to the image representing the three-dimensional shape of the contour C and the image representing the three-dimensional shape of the region R1 so that the user can confirm both the image representing the three-dimensional shape of the contour C and the image representing the three-dimensional shape of the region R1. Here, in the example of Fig. 5, the organ Q is shown as an ellipsoid for the sake of simplicity of explanation.

[0037] When only a portion of the organ Q is scanned, an image showing the contour C of the portion of the organ Q and an image showing the region R1 of the portion of the organ Q are displayed on the monitor 23, but when the organ Q is comprehensively scanned, an image showing the contour C covering the entire surface of the organ Q and an image showing the region R1 filling in all parts surrounded by the surface of the organ Q are displayed on the monitor 23. Therefore, by performing scanning with the ultrasound probe 1 while checking the image showing the three-dimensional shape of the contour C and the image showing the three-dimensional shape of the region R1 displayed on the monitor 23, the user can easily grasp the progress of the scanning and ensure comprehensive scanning of the organ Q.

[0038] In addition, instead of displaying the image representing the three-dimensional shape of contour C and the image representing the three-dimensional shape of region R1 superimposed on each other on monitor 23, the display control unit 22 can also display the image representing the three-dimensional shape of contour C and the image representing the three-dimensional shape of region R1 adjacent to each other on monitor 23.

[0039] The monitor 23 displays ultrasound images, images representing the contour C of the organ Q and the region R1, etc. under the control of the display control unit 22, and has a display device such as an LCD (Liquid Crystal Display) or an organic EL display (Organic Electroluminescence Display).

[0040] The main body control unit 29 controls each part of the device main body 2 and the transmission / reception circuit 12 of the ultrasonic probe 1 based on a control program stored in advance.

[0041] The input device 30 is used by the user to perform input operations, and is composed of devices such as a keyboard, a mouse, a trackball, a touch pad, and a touch sensor disposed over the monitor 23, for example.

[0042] The processor 31 having the image generation unit 21, the display control unit 22, the contour detection unit 25, the contour data generation unit 26, the area detection unit 27, the area data generation unit 28 and the main body control unit 29 is composed of a CPU (Central Processing Unit) and a control program for causing the CPU to perform various processes, but may also be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit) or other ICs (Integrated Circuits), or may be composed of a combination of these.

[0043] In addition, the image generation unit 21, display control unit 22, contour detection unit 25, contour data generation unit 26, area detection unit 27, area data generation unit 28 and main body control unit 29 of the processor 31 can be configured as a single CPU or the like, either partially or entirely integrated.

[0044] Next, the operation of the ultrasound diagnostic apparatus according to the first embodiment will be described with reference to the flowchart shown in Fig. 6. In the following description, it is assumed that a user scans the ultrasound probe 1 while changing the position and posture of the ultrasound probe 1 in a state where the ultrasound probe 1 is in contact with the body surface of a subject, in order to comprehensively scan an organ Q.

[0045] In step S1, the position and orientation sensor 13 acquires the position and orientation information of the ultrasonic probe 1. The position and orientation information of the ultrasonic probe 1 acquired in step S1 is sent to the contour data generating unit 26 and the region data generating unit .

[0046] In step S2, the image acquisition unit 32 acquires an ultrasonic image of the subject. At this time, under the control of the main body control unit 29, transmission and reception of ultrasonic waves is started from the multiple transducers of the transducer array 11 in accordance with a drive signal from the pulsar 41 of the transmission and reception circuit 12 of the ultrasonic probe 1, ultrasonic echoes from within the subject are received by the multiple transducers of the transducer array 11, and the received signals, which are analog signals, are output to the amplifier unit 42 and amplified, and are AD converted by the AD conversion unit 43 to obtain received data.

[0047] The beamformer 44 performs reception focusing processing on this reception data, and the sound ray signal generated thereby is sent to the image generating unit 21 of the device body 2, which generates an ultrasound image showing tomographic image information of the subject. At this time, the signal processing unit 45 of the image generating unit 21 performs attenuation correction and envelope detection processing according to the depth of the reflection position of the ultrasound on the sound ray signal, the DSC 46 converts it into an image signal according to the scanning method of a normal television signal, and the image processing unit 47 performs various necessary image processing such as gradation processing. The ultrasound image thus acquired in step S2 is displayed on the monitor 23 via the display control unit 22, and is also sent to the contour detection unit 25 and the area detection unit 27 via the image memory 24.

[0048] In step S3, the contour detection unit 25 detects the contour C of the organ Q imaged in the ultrasound image acquired in step S2, and the contour data generation unit 26 generates three-dimensional image data of the contour C of the organ Q based on the contour C of the organ Q detected by the contour detection unit 25 and the position and posture information of the ultrasound probe 1 acquired in step S1.

[0049] The contour detection unit 25 can detect the contour C of the organ Q captured in the ultrasound image, for example, by a method using a contour detection model that has previously learned the contour C of the organ Q in the ultrasound image of the organ Q.

[0050] In addition, the contour data generation unit 26 corresponds the contour C of the organ Q detected by the contour detection unit 25 to the position and orientation information of the ultrasound probe 1, and generates three-dimensional image data of the contour C of the organ Q by placing the contour C of the organ Q in three-dimensional space based on the position and orientation information of the ultrasound probe 1.

[0051] The contour data generation unit 26 can, for example, associate the position and orientation information input from the position and orientation sensor 13 with the contour C of the organ Q input at a point in time when the time required for the contour C to be detected by the contour detection unit 25 has elapsed after the position and orientation information was input.

[0052] In step S4, the region detection unit 27 detects a region R1 of the organ Q imaged in the ultrasound image acquired in step S2, and the region data generation unit 28 generates three-dimensional image data of the region R1 of the organ Q based on the region R1 of the organ Q detected by the region detection unit 27 and the position and posture information of the ultrasound probe 1 acquired in step S1.

[0053] The area detection unit 27 can detect the area R1 of the organ Q captured in the ultrasound image, for example, by a method using an area detection model that has been previously trained on the area R1 of the organ Q in the ultrasound image in which the organ Q is captured.

[0054] In addition, the region data generation unit 28 corresponds the region R1 of the organ Q detected by the region detection unit 27 to the position and orientation information of the ultrasound probe 1, and generates three-dimensional image data of the region R1 of the organ Q by placing the region R1 of the organ Q in three-dimensional space based on the position and orientation information of the ultrasound probe 1.

[0055] In step S5, the display control unit 22 displays on the monitor 23 an image representing the contour C of the organ Q and an image representing the region R1 of the organ Q, based on the three-dimensional image data of the contour C of the organ Q generated in step S3 and the three-dimensional image data of the region R1 of the organ Q generated in step S4. At this time, the display control unit 22 can display on the monitor 23 an image representing the three-dimensional shape of the contour C of the organ Q and its inner region R1, for example, as shown in Fig. 5.

[0056] In step S6, the main body control unit 29 judges whether or not to complete the examination of the subject using the ultrasound diagnostic apparatus. For example, the main body control unit 29 judges that the examination of the subject is completed when the user judges that the organ Q has been sufficiently scanned and inputs an instruction to complete the examination via the input device 30, and can determine to continue the examination when the user does not particularly input an instruction to complete the examination via the input device 30.

[0057] If it is determined in step S6 that the examination is to be continued, the process returns to step S1, and the position and orientation information of the ultrasonic probe 1 is acquired by the position and orientation sensor 13. Since the user scans the ultrasonic probe 1 while changing the position and orientation of the ultrasonic probe 1, the position and orientation information acquired in this step S1 represents a position and orientation different from the position and orientation of the ultrasonic probe 1 represented by the position and orientation information acquired in the previous step S1.

[0058] Next, in step S2, a new ultrasound image is acquired. In the next step S3, three-dimensional image data is generated of the contour C of the organ Q captured in the ultrasound image acquired in step S2. At this time, the contour data generating unit 26 generates three-dimensional image data of the contour C by arranging in three-dimensional space the contour C of the organ Q detected by the contour detecting unit 25 in the previous step S3 and the contour of the organ Q detected by the contour detecting unit 25 in the current step S3 according to the corresponding position and orientation information.

[0059] In step S4, three-dimensional image data of region R1 of organ Q captured in the ultrasound image acquired in step S2 is generated. At this time, the region data generating unit 28 generates the three-dimensional image data of region R1 by arranging in three-dimensional space region R1 of organ Q detected by region detecting unit 27 in the previous step S4 and region R1 of organ Q detected by region detecting unit 27 in the current step S4 according to the position and orientation information corresponding to each region R1.

[0060] In step S5, the display control unit 22 displays an image representing the contour C and region R1 of the organ Q on the monitor 23 based on the three-dimensional image data of the contour C and region R1 of the organ Q generated in the previous steps S3 and S4. When the process of step S5 is completed, in step S6, the main body control unit 29 determines whether or not the examination of the subject is to be completed.

[0061] In this manner, the processes of steps S1 to S6 are repeated as long as it is determined in step S6 that the examination of the subject is to be continued. As a result, the contour C of the organ Q displayed on the monitor 23 gradually expands along the surface of the organ Q, and the region R1 of the organ Q gradually fills in the interior of the organ Q. When the organ Q has finally been comprehensively scanned, an image showing the contour C covering the entire surface of the organ Q and an image showing the region R1 filling in all parts surrounded by the surface of the organ Q are displayed on the monitor 23.

[0062] By scanning the ultrasound probe 1 while checking the image representing the contour C and region R1 of the organ Q displayed on the monitor 23 in the repeated steps S1 to S6, the user can easily grasp the progress of the scanning and ensure comprehensive scanning of the organ Q.

[0063] When it is determined in step S6 that the examination of the subject is to be completed, the operation of the ultrasonic diagnostic apparatus according to the flowchart of FIG. 6 is completed.

[0064] As described above, according to the ultrasound diagnostic apparatus of embodiment 1, the contour detection unit 25 detects the contour C of the organ Q imaged in the ultrasound image, the region detection unit 27 detects the region R1 of the organ Q imaged in the ultrasound image, the contour data generation unit 26 generates three-dimensional image data of the contour C of the organ Q based on the contour C of the organ Q and the position and orientation information, the region data generation unit 28 generates three-dimensional image data of the region R1 of the organ Q based on the region R1 of the organ Q and the position and orientation information, and the display control unit 22 sequentially displays an image representing the contour C and an image representing the region R1 on the monitor 23 based on the three-dimensional image data of the contour C and the three-dimensional image data of the region R1, thereby allowing the user to reliably perform a comprehensive scan of the organ Q.

[0065] Although the transmitting / receiving circuit 12 has been described as being provided in the ultrasonic probe 1, the transmitting / receiving circuit 12 may be provided in the device body 2. Further, although the image generating unit 21 is described as being provided in the device body 2, the image generating unit 21 may be provided in the ultrasound probe 1.

[0066] The device body 2 may be a so-called stationary type, a portable type that is easy to carry, or a so-called handheld type that is configured by, for example, a smartphone or a tablet computer. In this way, the type of device that configures the device body 2 is not particularly limited.

[0067] It has also been described that the display control unit 22 displays an image representing the three-dimensional shape of the contour C and region R1 of the organ Q on the monitor 23. At this time, for example, the display control unit 22 can also display the image representing the three-dimensional shape on the monitor 23 while changing the rotation angle of the three-dimensional shape displayed on the monitor 23 in response to an input operation by the user via the input device 30. If part of the contour C or part of the region R1 cannot be detected for some reason, such as the ultrasound image being unclear, the image representing the contour C of the organ Q or the image representing the region R1 is displayed with a missing part, but even in this case, the user can clearly understand which parts of the organ Q have been scanned and which parts have not been scanned.

[0068] In addition, in the flow chart of FIG. 6, step S2 is performed after step S1, but step S1 may be performed after step S2, or step S1 and step S2 may be performed simultaneously. Furthermore, although step S4 is performed after step S3, step S3 may be performed after step S4, or steps S3 and S4 may be performed simultaneously.

[0069] Embodiment 2 In order to enable the user to more reliably perform a comprehensive scan of organ Q, the ultrasound diagnostic device determines parts of organ Q where contour C is not detected or parts where region R1 is not detected as unimaged parts and notifies the user that an unimaged part exists.

[0070] Fig. 7 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to embodiment 2. The ultrasonic diagnostic apparatus according to embodiment 2 includes a device body 2A instead of the device body 2 in the ultrasonic diagnostic apparatus according to embodiment 1 shown in Fig. 1. The device body 2A further includes an unimaged determination unit 51 and a notification unit 52 in the device body 2 in embodiment 1, and includes a body control unit 29A instead of the body control unit 29.

[0071] In device main body 2A, unrendered determination unit 51 is connected to contour data generation unit 26 and region data generation unit 28. Notification unit 52 is connected to notification unit 51. Notification unit 52 is connected to display control unit 22. Notification unit 51 and notification unit 52 are connected to main body control unit 29A. Moreover, image generation unit 21, display control unit 22, contour detection unit 25, contour data generation unit 26, region detection unit 27, region data generation unit 28, main body control unit 29A, notification unit 51 and notification unit 52 constitute a processor 31A for device main body 2A.

[0072] The unimaging determination unit 51 determines as an unimaging portion a portion where the contour C of organ Q has not been detected by the contour detection unit 25 or the region R1 of organ Q has not been detected by the region detection unit 27 due to some reason unintentional by the user, such as the user failing to operate the ultrasound probe 1 or the ultrasound image being unclear, causing the contour detection unit 25 to fail to detect contour C or the region detection unit 27 to fail to detect region R1.

[0073] For example, when, due to some kind of failure, the contour C of the organ Q detected by the contour detection unit 25 or the region R1 of the organ Q detected by the region detection unit 27 has a discontinuous portion having a gap equal to or greater than a specified interval, the unrepresented portion determination unit 51 can determine that the discontinuous portion is an unrepresented portion.

[0074] At this time, the unrepresented determination unit 51 can set a three-dimensional grid G ​​consisting of a plurality of cells D having a cubic shape of a certain size for the three-dimensional image data of the contour C and region R1 of the organ Q, for example, as shown in Fig. 8. When any of four adjacent cells D2 in contact with the four sides of a cell D1 occupied by the contour C or region R1 in the three-dimensional grid G ​​is not occupied by the contour C or region R1, as shown in Fig. 9, the unrepresented determination unit 51 can determine that the adjacent cell D2 is an unrepresented portion by considering the adjacent cell D2 as an unrepresented portion. In addition, when any of eight adjacent cells D2 located around the cell D1 occupied by the contour C or region R1 is not occupied by the contour C or region R1, the unrepresented determination unit 51 can determine that the adjacent cell D2 is an unrepresented portion by considering the adjacent cell D2 as an unrepresented portion.

[0075] Furthermore, the unimaged portion determination unit 51 can determine the presence of an unimaged portion when a predetermined time has elapsed since the start of scanning by the ultrasound probe 1. In many cases, when a sufficient amount of time has elapsed since the start of scanning, the user has finished scanning most of the organ Q, but the unimaged portion determination unit 51 can determine the presence of an unimaged portion and thus find an omission in scanning.

[0076] Furthermore, the unimaged portion determination unit 51 can also determine the presence of an unimaged portion when the user issues an instruction to end the examination. Even if the user recognizes that the organ Q has been sufficiently scanned, there may be areas that have not yet been scanned, and the unimaged portion determination unit 51 can determine the presence of an unimaged portion to discover any omissions in scanning.

[0077] Furthermore, the unimaged portion determination unit 51 determines whether the contour C detected by the contour detection unit 25 is closed, and when the contour C is closed, it can also determine the presence of an unimaged portion where the region R1 has not been detected by the region detection unit 27. Even when all the contours C of the organ Q are detected, there may be cases where the portion surrounded by the contour C cannot be detected for some reason, such as the ultrasound image having unclear parts, and therefore the presence of an unimaged portion determined by the unimaged portion determination unit 51 can be determined, thereby discovering an omission in scanning.

[0078] Here, the unrepresented portion determining unit 51 prestores, for example, an organ shape model representing the general shape of each organ Q, and can determine whether the contour C is closed or not by comparing the contours C sequentially detected by the contour detecting unit 25 with the organ shape model corresponding to the organ Q in which the contour C was detected. At this time, the unrepresented portion detecting unit 51 can store, for example, portions that are not usually depicted in an ultrasound image, such as the vicinity of the portal vein in the liver, from each organ shape model.

[0079] Generally, since an organ Q is connected to other adjacent organs Q, it may be difficult to accurately determine the boundary between the organ Q and other organs Q. Therefore, when the volume of a portion where the region R1 is detected among the volume surrounded by all the contours C of the organ Q becomes equal to or larger than a certain volume, the unrendered portion determination unit 51 can comprehensively scan the organ Q and determine that there is no unrendered portion. At this time, the unrendered portion detection unit 51 can store a volume threshold value related to the volume of the portion where the region R1 is detected for each type of organ Q, such as 95% for the bladder, 90% for the liver and kidneys, and 95% for the heart.

[0080] When the unimaged portion determination unit 51 determines that an unimaged portion exists, the notification unit 52 notifies the user by, for example, displaying a message indicating that an unimaged portion exists on the monitor 23. This notification enables the user to move the ultrasound probe 1 so as to scan the unimaged portion, thereby ensuring comprehensive scanning of the organ Q.

[0081] The notification unit 52 can also notify the user that the scan is complete when the contour C of the organ Q detected by the contour detection unit 25 is closed and there is no region R1 inside the contour C that has not been detected by the region detection unit 27. The user can continue scanning with the ultrasound probe 1 until this notification is made, thereby enabling the user to perform comprehensive scanning of the organ Q more reliably.

[0082] In addition, as shown in FIG. 9, the unrendered portion determination unit 51 determines the unrendered portion by referring to four adjacent cells D2 two-dimensionally adjacent to the cell D1 occupied by the contour C or the region R1 or eight adjacent cells D2 located around the cell D1. However, the unrendered portion can also be determined by referring to six adjacent cells D2 three-dimensionally adjacent to the six faces of the cell D1 occupied by the contour C or the region R1 or 26 adjacent cells D2 located around the cell D1 in three-dimensional space.

[0083] Third embodiment Generally, when a user finishes imaging the subject, the user often moves the ultrasound probe 1 away from the subject's body surface. Therefore, the ultrasound diagnostic device can also determine whether or not there is an unimaged portion at the timing when the ultrasound probe 1 moves away from the subject's body surface.

[0084] Fig. 10 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to embodiment 3. The ultrasonic diagnostic apparatus according to embodiment 3 includes a device body 2B instead of the device body 2A in the ultrasonic diagnostic apparatus according to embodiment 2 shown in Fig. 7. The device body 2B further includes a non-contact detection unit 53 in the device body 2A in embodiment 2, and includes a body control unit 29B instead of the body control unit 29A.

[0085] In the device main body 2B, a non-contact detection unit 53 is connected to the image memory 24. The non-contact detection unit 53 is connected to the unrendered determination unit 51 and the main body control unit 29B. The image generation unit 21, the display control unit 22, the contour detection unit 25, the contour data generation unit 26, the area detection unit 27, the area data generation unit 28, the main body control unit 29B, the unrendered determination unit 51, the notification unit 52, and the non-contact detection unit 53 configure a processor 31B for the device main body 2B.

[0086] The non-contact detection unit 53 detects a non-contact state in which the ultrasonic probe 1 is separated from the body surface of the subject. When the ultrasonic probe 1 is separated from the body surface of the subject, ultrasonic waves are emitted from the ultrasonic probe 1 into the air. In this case, since no ultrasonic echo is obtained, an aerial emission image in which the entire image is filled with a single color such as black is acquired as the ultrasonic image. Therefore, the non-contact detection unit 53 can detect that the ultrasonic probe 1 is in a non-contact state by, for example, analyzing the ultrasonic image and detecting the aerial emission image.

[0087] When the non-contact state of the ultrasound probe 1 is detected by the non-contact detection unit 53, the non-imaged portion determination unit 51 determines the presence of an unimaged portion. When the unimaged portion determining unit 51 determines that an unimaged portion exists, the notifying unit 52 notifies the user of that effect.

[0088] Generally, when the user finishes imaging the subject, the user often removes the ultrasound probe 1 from the subject's body surface. Therefore, by determining the presence of an unimaged portion at the timing when the non-contact state of the ultrasound probe 1 is detected, the user can find an overscanned portion and reliably perform comprehensive scanning of the organ Q.

[0089] Although it has been described that the non-contact detection unit 53 detects the non-contact state of the ultrasonic probe 1 by analyzing the ultrasonic image and detecting the air radiation image, the method of detecting the non-contact state is not particularly limited to this. For example, when the tip of the ultrasonic probe 1 is provided with a pressure sensor (not shown) that measures the pressure of the ultrasonic probe 1 contacting the body surface of the subject, the non-contact detection unit 53 can detect the non-contact state of the ultrasonic probe 1 based on the measurement value of the pressure sensor. In this case, the non-contact detection unit 53 has, for example, a pressure threshold value set for the measurement value of the pressure sensor, and when the measurement value of the pressure sensor becomes equal to or less than the pressure threshold value, it can determine that the ultrasonic probe 1 is not in contact with the subject and detect the non-contact state.

[0090] Fourth embodiment In order to prevent missed scans, if a contour C that can be depicted in an ultrasound image of an organ Q is not detected, the ultrasound diagnostic device can also notify the user to move the ultrasound probe 1 so as to depict the contour C.

[0091] Fig. 11 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to embodiment 4. The ultrasonic diagnostic apparatus according to embodiment 4 includes a device body 2C instead of the device body 2A in the ultrasonic diagnostic apparatus according to embodiment 2 shown in Fig. 7. The device body 2C in embodiment 4 further includes a reference contour memory 54 in the device body 2A in embodiment 2, and includes a body control unit 29C instead of the body control unit 29A.

[0092] In device main body 2C, reference contour memory 54, contour detection section 25, contour data generation section 26, area detection section 27, and area data generation section 28 are connected to unrendered determination section 51 and main body control section 29C. In addition, image generation section 21, display control section 22, contour detection section 25, contour data generation section 26, area detection section 27, area data generation section 28, main body control section 29C, unrendered determination section 51, and notification section 52 constitute a processor 31C for device main body 2C.

[0093] The reference contour memory 54 is a memory in which a plurality of reference contours that can be rendered in a plurality of organs Q are stored. As the reference contour memory 54, for example, a recording medium such as a flash memory, HDD, SSD, FD, MO disk, MT, RAM, CD, DVD, SD card, or USB memory can be used.

[0094] The unrepresented determination unit 51 recognizes the organ Q based on the contour C of the organ Q detected by the contour detection unit 25, the three-dimensional image data of the contour C generated by the contour data generation unit 26, and the three-dimensional image data of the region R1 of the organ Q detected by the region detection unit 27 or the region R1 generated by the region data generation unit 28, and reads out a plurality of reference contours for the recognized organ Q from the reference contour memory 54. The unrepresented determination unit 51 further compares the plurality of reference contours read out from the reference contour memory 54 with the plurality of contours C of the organ Q detected by the contour detection unit 25 to determine whether or not a contour C corresponding to the reference contour has been detected.

[0095] The unrepresented image determination unit 51 can recognize the organ Q and determine whether or not a contour C corresponding to the reference contour has been detected using, for example, so-called template matching, image analysis techniques using features such as AdaBoost, SVM or SIFT, or a machine learning model trained using machine learning techniques such as deep learning.

[0096] When the undetected determination unit 51 determines that the contour C corresponding to the reference contour stored in the reference contour memory 54 has not been detected by the contour detection unit 25, the notification unit 52 notifies the user to move the ultrasound probe 1 to obtain an ultrasound image depicting the contour C corresponding to the reference contour. The notification unit 52 can notify the user, for example, by displaying a portion corresponding to the undetected contour C in a color different from the surroundings on an image representing the three-dimensional shape of the contour C based on the three-dimensional image data of the contour C generated by the contour data generation unit 26.

[0097] By confirming the notification by the notification unit 52 to move the ultrasound probe 1, the user can reliably scan the unimaged parts and reliably perform comprehensive scanning of the organ Q.

[0098] Fifth embodiment In order to enable the user to clearly grasp the contour C and area R1 of the organ Q that has already been scanned, the ultrasound diagnostic apparatus can also highlight the contour C and area R1 of the organ Q that has already been scanned on the monitor 23 in a three-dimensional schematic diagram that mimics the structure inside the subject.

[0099] Fig. 12 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to embodiment 5. The ultrasonic diagnostic apparatus according to embodiment 5 includes a device body 2D instead of the device body 2 in the ultrasonic diagnostic apparatus according to embodiment 1 shown in Fig. 1. The device body 2D further includes a schema memory 55 and an emphasis display unit 56 in the device body 2 in embodiment 1, and includes a body control unit 29D instead of the body control unit 29.

[0100] In device main body 2D, a schema memory 55 is connected to main body control unit 29D. Further, an emphasis display unit 56 is connected to schema memory 55, contour data generation unit 26, and region data generation unit 28. The emphasis display unit 56 is connected to display control unit 22 and main body control unit 29D. Further, image generation unit 21, display control unit 22, contour detection unit 25, contour data generation unit 26, region detection unit 27, region data generation unit 28, main body control unit 29D, and emphasis display unit 56 constitute a processor 31D for device main body 2D.

[0101] The schema memory 55 is a memory that stores in advance a plurality of three-dimensional schema images A, which are three-dimensional schematic diagrams simulating a structure including at least one organ Q in a subject, as shown in Fig. 13. Here, Fig. 13 shows an example of a three-dimensional schema image A including a liver B1, a gallbladder B2, a stomach B3, a spleen B4, a pancreas B5, and a duodenum B6 as the organ Q. As the schema memory 55, for example, a recording medium such as a flash memory, an HDD, an SSD, an FD, an MO disk, an MT, a RAM, a CD, a DVD, an SD card, or a USB memory can be used.

[0102] 13, for example, based on the three-dimensional image data of the contour C and region R1 of the organ Q generated by the contour data generating unit 26 and the region data generating unit 28, the highlighting unit 56 highlights the scanned portion P1 formed by the contour C detected by the contour detecting unit 25 and the region R1 detected by the region detecting unit 27 in the three-dimensional schematic image A. In the example of FIG. 13, a schematic diagram of the ultrasound probe 1 is shown to show that the scanned portion P1 corresponds to a plurality of frames of ultrasound images generated while tilting the ultrasound probe 1 within a certain angle range, but the schematic diagram of the ultrasound probe 1 does not need to be displayed.

[0103] The highlighting unit 56 can, for example, read out a three-dimensional schema image A specified by the user via the input device 30 from among the multiple three-dimensional schema images A stored in the schema memory 55, and display it on the monitor 23. In addition, the highlighting unit 56 can highlight the contour C and region R1 that have already been scanned by giving the already-scanned portion P1 a color different from the surrounding color, or by blinking the color, or the like.

[0104] The user can clearly and easily grasp the parts of the subject whose ultrasound images have already been captured by checking the already scanned parts P1 that are highlighted on the monitor 23. This allows the user to reliably perform comprehensive scanning of the required examination parts.

[0105] Although an example has been described in which a three-dimensional schema image A specified by the user via the input device 30 is read out from among multiple three-dimensional schema images A stored in the schema memory 55, for example, the highlighting unit 56 can automatically select a three-dimensional schema image A including the position of the scanned portion P1 from the schema memory 55 and display it on the monitor 23.

[0106] Sixth embodiment In embodiments 1 to 5, it has been described that an image representing the three-dimensional shape of the contour C of the organ Q and the region R1 is displayed on the monitor 23, but the ultrasound diagnostic apparatus can also display two-dimensional tomographic images of the contour C of the organ Q and the region R1 on the monitor 23.

[0107] Fig. 14 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to embodiment 6. The ultrasonic diagnostic apparatus according to embodiment 6 includes a device body 2E instead of the device body 2 in the ultrasonic diagnostic apparatus according to embodiment 1 shown in Fig. 1. The device body 2E further includes a tomographic image extraction unit 57 in the device body 2 in embodiment 1, and includes a body control unit 29E instead of the body control unit 29.

[0108] In the device main body 2E, a tomographic image extraction unit 57 is connected to the contour data generation unit 26 and the area data generation unit 28. The tomographic image extraction unit 57 is connected to the display control unit 22 and the main body control unit 29E. The image generation unit 21, the display control unit 22, the contour detection unit 25, the contour data generation unit 26, the area detection unit 27, the area data generation unit 28, the main body control unit 29E, and the tomographic image extraction unit 57 constitute a processor 31E for the device main body 2E.

[0109] The tomographic image extraction unit 57 extracts two-dimensional tomographic images of the contour C and two-dimensional tomographic images of the region R1 on the same cross section from the three-dimensional image data of the contour C generated by the contour data generation unit 26 and the three-dimensional image data of the region R1 generated by the region data generation unit 28. The tomographic image extraction unit 57 can extract, for example, three two-dimensional tomographic images of the contour C and the region R1 along planes perpendicular to each other in three-dimensional space.

[0110] The display control unit 22 sequentially displays, as an image representing the contour C of the organ Q and an image representing the region R1, the two-dimensional tomographic image of the contour C extracted by the tomographic image extraction unit 57, on the monitor 23, for example, as shown in Figure 15.

[0111] By checking the two-dimensional tomographic image of the contour C of organ Q and region R1 displayed on monitor 23, the user can reliably perform comprehensive scanning of organ Q while clearly understanding which parts of organ Q have already been scanned.

[0112] The tomographic image extraction unit 57 can change the position and inclination angle of the cut plane to be extracted in the organ Q, for example, based on an input operation by the user via the input device 30. In this case, the display control unit 22 can sequentially display, on the monitor 23, two-dimensional tomographic images of the contour C and the region R1 in the cut plane specified by the user via the input device 30. This allows the user to grasp in more detail the parts of the organ Q that have been scanned and the parts that have not been scanned. [Explanation of symbols]

[0113] 1 Ultrasonic probe, 2, 2A, 2B, 2C, 2D, 2E Device body, 11 Transducer array, 12 Transmitting / receiving circuit, 13 Position and orientation sensor, 21 Image generation unit, 22 Display control unit, 23 Monitor, 24 Image memory, 25 Contour detection unit, 26 Contour data generation unit, 27 Area detection unit, 28 Area data generation unit, 29, 29A, 29B, 29C, 29D, 29E Main body control unit, 30 Input device, 31, 31A, 31B, 31C, 31D, 31E Processor, 32 Image acquisition unit, 41 Pulser, 42 Amplification unit, 43 AD conversion unit, 44 Beamformer, 45 Signal processing unit, 46 DSC, 47 Image processing unit, 51 Non-imaging determination unit, 52 Notification unit, 53 Non-contact detection unit, 54 Reference contour memory, 55 Schema memory, 56 Highlighted area, 57 tomographic image extraction area, A 3D schematic image, B1 liver, B2 gallbladder, B3 stomach, B4 spleen, B5 pancreas, B6 duodenum, D, D1 cell, D2 adjacent cell, G 3D grid, P1 scanned area.

Claims

1. An ultrasonic probe; a position and orientation sensor for acquiring position and orientation information of the ultrasonic probe; an image acquisition unit that acquires an ultrasonic image of a subject by transmitting and receiving an ultrasonic beam using the ultrasonic probe; a contour detection unit that detects a contour of an organ imaged in the ultrasound image based on the ultrasound image; a region detection unit that detects a region of the organ imaged in the ultrasound image based on the ultrasound image; a contour data generation unit that generates three-dimensional image data of the contour of the organ based on the contour detected by the contour detection unit and the position and orientation information acquired by the position and orientation sensor; an area data generation unit that generates three-dimensional image data of the area of ​​the organ based on the area detected by the area detection unit and the position and orientation information acquired by the position and orientation sensor; a monitor for sequentially displaying an image representing the contour and an image representing the region based on the three-dimensional image data of the contour and the three-dimensional image data of the region; An ultrasound diagnostic device comprising:

2. the contour detection unit detects the contour using a contour detection model that has been trained on the contour of the organ in an ultrasound image of the organ; The ultrasound diagnostic apparatus according to claim 1 , wherein the region detection unit detects the region using a region detection model that has been trained to learn the region of the organ in an ultrasound image of the organ.

3. an unimaged portion determining unit that determines, as an unimaged portion, a portion where the contour is not detected by the contour detection unit or a portion where the region is not detected by the region detection unit; a notification unit that notifies a user when the unrendered portion determination unit determines that the unrendered portion exists; and The ultrasonic diagnostic apparatus according to claim 1 .

4. 4. The ultrasound diagnostic apparatus according to claim 3, wherein when the contour detected by the contour detection unit or the region detected by the region detection unit has a discontinuous portion having a predetermined interval or more, the unimaged portion determination unit determines the discontinuous portion to be the unimaged portion.

5. a non-contact detection unit that detects a non-contact state in which the ultrasonic probe is separated from a body surface of the subject; The ultrasonic diagnostic apparatus according to claim 3 , wherein the unimaged portion determining unit determines the presence of the unimaged portion when the non-contact state is detected by the non-contact detection unit.

6. The ultrasonic diagnostic apparatus according to claim 3 , wherein the unimaged portion determining unit determines the presence of the unimaged portion when a predetermined time has elapsed since the start of scanning by the ultrasonic probe.

7. The ultrasound diagnostic apparatus according to claim 3 , wherein the unimaged portion determining unit determines the presence of the unimaged portion when the user issues an instruction to end the examination.

8. The ultrasonic diagnostic apparatus according to claim 3 , wherein the unimaged portion determining unit determines the presence of the unimaged portion, the region of which has not been detected by the region detecting unit, when the contour detected by the contour detecting unit is closed.

9. 4. The ultrasound diagnostic device according to claim 3, wherein the notification unit notifies the user that scanning is completed when the contour detected by the contour detection unit is closed and there is no region within the inner range of the contour that has not been detected by the region detection unit.

10. The monitor displays a three-dimensional schematic image; The ultrasound diagnostic apparatus according to claim 1 , further comprising an emphasis display unit that displays, on the three-dimensional schematic image, the contour detected by the contour detection unit and the region detected by the region detection unit in an emphasis manner.

11. a reference contour memory in which reference contours that can be rendered are stored; 4. The ultrasound diagnostic apparatus according to claim 3, wherein the notification unit notifies the user to move the ultrasound probe when the contour corresponding to the reference contour stored in the reference contour memory is not detected by the contour detection unit.

12. 2. The ultrasonic diagnostic apparatus according to claim 1, wherein the monitor sequentially displays a three-dimensional image of the contour and a three-dimensional image of the region as the image representing the contour and the image representing the region.

13. a tomographic image extraction unit that extracts two-dimensional tomographic images of the contour and two-dimensional tomographic images of the region on the same cross section from the three-dimensional image data of the contour generated by the contour data generation unit and the three-dimensional image data of the region generated by the region data generation unit, 2. The ultrasonic diagnostic apparatus according to claim 1, wherein the monitor sequentially displays, as the image representing the contour and the image representing the region, a two-dimensional tomographic image of the contour and a two-dimensional tomographic image of the region extracted by the tomographic image extraction unit.

14. The ultrasonic diagnostic apparatus according to claim 1 , wherein the monitor displays the image representing the contour and the image representing the region in a superimposed manner.

15. The ultrasonic diagnostic apparatus according to claim 1 , wherein the monitor displays the image representing the contour and the image representing the region side by side.

16. The ultrasonic diagnostic apparatus according to claim 1 , wherein the position and orientation sensor includes an inertial sensor, a magnetic sensor, or an optical sensor.

17. Acquire position and orientation information of the ultrasound probe; Acquiring an ultrasonic image of a subject by transmitting and receiving ultrasonic beams using the ultrasonic probe; detecting a contour of an organ imaged in the ultrasound image based on the ultrasound image; Detecting an area of ​​the organ imaged in the ultrasound image based on the ultrasound image; generating three-dimensional image data of the contour of the organ based on the detected contour and the acquired position and orientation information; generating three-dimensional image data of the region of the organ based on the detected region and the acquired position and orientation information; An image representing the contour and an image representing the region are sequentially displayed on a monitor based on the three-dimensional image data of the contour and the three-dimensional image data of the region. A method for controlling an ultrasonic diagnostic device.

Images