Ultrasonic diagnostic apparatus and control method of ultrasonic diagnostic apparatus

Abstract

To provide an ultrasonic diagnostic apparatus and a control method of an ultrasonic diagnostic apparatus that allow a user to grasp an imaging part of a subject easily irrespective of his or her skill level.SOLUTION: An ultrasonic diagnostic apparatus includes: an ultrasonic probe (1); a position posture sensor (13) for acquiring position posture information on the ultrasonic probe (1); an image acquisition unit (32) for acquiring an ultrasonic image of a subject using the ultrasonic probe (1); a three-dimensional image data generation unit (25) for generating three-dimensional ultrasonic image data on the basis of the position posture information and the ultrasonic image; a three-dimensional structure information extraction unit (26) for extracting three-dimensional structure information included in the three-dimensional ultrasonic image data; and a position estimation unit (27) having a position estimation model that is caused to learn the position of the three-dimensional structure in the three-dimensional image data for estimating the position of a region occupied by the three-dimensional ultrasonic image data on the subject by the position estimation model on the basis of the extracted three-dimensional structure information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an ultrasonic diagnostic apparatus for acquiring a three-dimensional structure of a subject and a method for controlling the ultrasonic diagnostic apparatus. [Background technology]

[0002] Conventionally, examinations have been performed by capturing ultrasound images showing cross sections of a subject using a so-called ultrasound diagnostic device. In such examinations, a user such as a doctor usually captures ultrasound images while changing the position of a so-called ultrasound probe in contact with the body surface of the subject and moving the ultrasound probe.

[0003] In this case, the user usually takes ultrasound images sequentially while checking the taken ultrasound images to determine the part of the subject that has been taken, but a user who is not very skilled in examinations using an ultrasound diagnostic device may have difficulty in determining which part of the subject has been taken even when checking the ultrasound images. Therefore, a technology has been developed that estimates the imaging location of the subject and shows it to the user, as disclosed in Patent Document 1, for example. In Patent Document 1, three-dimensional shape data of a general subject is prepared, and three-dimensional image data of the subject is generated from multiple frames of ultrasound images taken during the examination, and the three-dimensional shape data and the three-dimensional image data are compared with each other to estimate the imaging location of the subject. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-104137 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in general, the shapes and arrangements of parts inside a subject vary greatly from subject to subject, and therefore the technology of Patent Document 1, which uses general three-dimensional shape data of the subject to estimate the subject's imaging location, may not be able to correctly estimate the subject's imaging location, and the user may not be able to accurately grasp the subject's imaging location.

[0006] The present invention has been made to solve such conventional problems, and has an object to provide an ultrasound diagnostic apparatus and a control method for an ultrasound diagnostic apparatus that allows a user, regardless of his or her level of skill, to easily grasp the location on a subject to be imaged. [Means for solving the problem]

[0007] The above object can be achieved by the following configuration. [1] An ultrasonic probe; a position and orientation sensor for acquiring position and orientation information of an ultrasonic probe; an image acquisition unit that acquires an ultrasonic image representing a cross section of a subject by transmitting and receiving an ultrasonic beam using an ultrasonic probe; a three-dimensional image data generating unit that generates three-dimensional ultrasound image data of the subject based on the position and orientation information of the ultrasound probe acquired by the position and orientation sensor and the ultrasound image acquired by the image acquiring unit; a three-dimensional structure information extraction unit that extracts three-dimensional structure information related to the three-dimensional structure contained in the three-dimensional ultrasound image data from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit; a position estimation unit having a position estimation model that has learned the position of a three-dimensional structure in three-dimensional image data in which the three-dimensional structure is captured, and estimating the position of an area occupied by the three-dimensional ultrasound image data of the subject using the position estimation model based on the three-dimensional structure information extracted by the three-dimensional structure information extraction unit; An ultrasound diagnostic device comprising: [2] A reconstruction unit extracts two-dimensional tomographic images from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit, and reconstructs three-dimensional ultrasound image data of the subject based on the extracted tomographic images and position and orientation information of the ultrasound probe acquired by the position and orientation sensor; The ultrasound diagnostic device of claim 1, wherein the three-dimensional structural information extraction unit extracts three-dimensional structural information from the three-dimensional ultrasound image data reconstructed by the reconstruction unit. [3] A monitor for displaying a three-dimensional schematic image; An ultrasound diagnostic device as described in [1] or [2], comprising an emphasis display unit that highlights, in a three-dimensional schematic image, an area occupied by three-dimensional ultrasound image data whose position has been estimated by the position estimation unit. [4] The ultrasound diagnostic device according to [3], further comprising a data synthesis unit that sequentially synthesizes three-dimensional ultrasound image data generated by the three-dimensional image data generation unit based on a three-dimensional structure contained in the three-dimensional ultrasound image data whose position of the occupied area has been estimated by the position estimation unit. [5] An ultrasound diagnostic device as described in [3] or [4], further comprising an alarm unit that, when the position estimation unit is unable to estimate the position of the occupied area of ​​the three-dimensional ultrasound image data, prompts a user to move the ultrasound probe so that the position and orientation of the ultrasound probe are the same as when the three-dimensional ultrasound image data whose position of the occupied area has already been estimated by the position estimation unit, based on position and orientation information of the ultrasound probe acquired by the position and orientation sensor. [6] The ultrasound diagnostic device according to any one of [3] to [5], further comprising a notification unit that identifies an area not highlighted by the highlighting unit and notifies a user of the identified area. [7] The ultrasound diagnostic device according to [6], further comprising a reference image presenting unit that presents a reference ultrasound image to be observed in the area identified by the notification unit. [8] The ultrasonic diagnostic device according to any one of [1] to [7], wherein the three-dimensional structure information includes a shape pattern of the three-dimensional structure. [9] The ultrasound diagnostic device according to [8], wherein the three-dimensional structure information includes a brightness pattern of the three-dimensional structure.

[10] The ultrasonic diagnostic device according to any one of [1] to [9], wherein the position and orientation sensor includes an inertial sensor, a magnetic sensor, or an optical sensor.

[11] Acquire position and orientation information of an ultrasound probe; Acquiring an ultrasound image showing a cross section of the subject by transmitting and receiving an ultrasound beam using an ultrasound probe; generating three-dimensional ultrasound image data of the subject based on the acquired position and orientation information of the ultrasound probe and the acquired ultrasound image; extracting three-dimensional structural information relating to a three-dimensional structure contained in the three-dimensional ultrasound image data from the generated three-dimensional ultrasound image data; The extracted three-dimensional structure information is input to a position estimation model that has learned the positions of the three-dimensional structure in the three-dimensional image data in which the three-dimensional structure is captured, thereby estimating the position of the area occupied by the three-dimensional ultrasound image data of the subject. A method for controlling an ultrasonic diagnostic device. Effect of the Invention

[0008] The present invention provides an ultrasound diagnostic apparatus including an ultrasound probe, a position and orientation sensor for acquiring position and orientation information of the ultrasound probe, an image acquisition unit for acquiring an ultrasound image representing a cross section of the subject by transmitting and receiving an ultrasound beam using the ultrasound probe, a three-dimensional image data generation unit for generating three-dimensional ultrasound image data of the subject based on the position and orientation information of the ultrasound probe acquired by the position and orientation sensor and the ultrasound image acquired by the image acquisition unit, a three-dimensional structure information extraction unit for extracting three-dimensional structure information relating to the three-dimensional structure contained in the three-dimensional ultrasound image data from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit, and a position estimation unit having a position estimation model that has learned the position of the three-dimensional structure in the three-dimensional image data in which the three-dimensional structure is captured, and estimating the position of an area occupied by the three-dimensional ultrasound image data of the subject using the position estimation model based on the three-dimensional structure information extracted by the three-dimensional structure information extraction unit, thereby allowing a user to easily grasp the imaging location of the subject regardless of their level of proficiency. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an ultrasound diagnostic apparatus according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a configuration of a transmission / reception circuit in a first embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram showing a configuration of an image generating unit according to the first embodiment of the present invention. [Figure 4] 5 is a flowchart showing the operation of the ultrasound diagnostic apparatus according to the first embodiment of the present invention. [Diagram 5] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a second embodiment of the present invention. [Figure 6] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a third embodiment of the present invention. [Figure 7] FIG. 13 is a diagram showing an example of a three-dimensional schematic image according to the third embodiment of the present invention. [Figure 8] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a fourth embodiment of the present invention. [Figure 9] FIG. 11 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a fifth embodiment of the present invention. [Figure 10] FIG. 13 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a sixth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The following description of the components will be given based on a representative embodiment of the present invention, but the present invention is not limited to such an embodiment. In this specification, a numerical range expressed using "to" means a range that includes the numerical values ​​before and after "to" as the lower and upper limits. In this specification, the terms "same" and "identical" include a generally acceptable margin of error in the technical field.

[0011] First embodiment 1 shows the configuration of an ultrasonic diagnostic device according to a first embodiment of the present invention. The ultrasonic diagnostic device includes an ultrasonic probe 1 and a device main body 2 connected to the ultrasonic probe 1. The ultrasonic probe 1 and the device main body 2 are connected to each other by so-called wired communication or so-called wireless communication.

[0012] The ultrasonic probe 1 has a transducer array 11. A transmission / reception circuit 12 is connected to the transducer array 11. The ultrasonic probe 1 also has a position and orientation sensor 13. The position and orientation sensor 13 may be built into the ultrasonic probe 1, or may be attached to a housing of the ultrasonic probe 1. Furthermore, when a sensor device that measures the ultrasonic probe 1 from the outside, such as a so-called optical sensor, is used as the position and orientation sensor 13, the position and orientation sensor 13 may be disposed on the outside of the ultrasonic probe 1.

[0013] The device body 2 has an image generating unit 21 connected to the transmitting / receiving circuit 12 of the ultrasound probe 1. A display control unit 22 and a monitor 23 are connected to the image generating unit 21, in that order. An image memory 24 is connected to the image generating unit 21. The display control unit 22 is connected to the image memory 24. A three-dimensional image data generating unit 25 is connected to the position and orientation sensor 13 and the image memory 24. A three-dimensional structure information extracting unit 26 is connected to the three-dimensional image data generating unit 25. In addition, a position estimating unit 27 is connected to the three-dimensional image data generating unit 25 and the three-dimensional structure information extracting unit 26. The position estimating unit 27 is connected to the display control unit 22.

[0014] A main body control unit 28 is connected to the transmission / reception circuit 12, the image generation unit 21, the display control unit 22, the image memory 24, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, and the position estimation unit 27. An input device 29 is connected to the main body control unit 28. A processor 31 for the device main body 2 is formed by the image generation unit 21, the display control unit 22, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, the position estimation unit 27, and the main body control unit 28. An image acquisition unit 32 is formed by the transmission / reception circuit 12 and the image generation unit 21.

[0015] The transducer array 11 of the ultrasonic probe 1 has a plurality of ultrasonic transducers arranged one-dimensionally or two-dimensionally. Each of these ultrasonic transducers transmits ultrasonic waves according to a drive signal supplied from the transmission / reception circuit 12, receives ultrasonic echoes from the subject, and outputs a signal based on the ultrasonic echoes. Each ultrasonic transducer is configured by forming electrodes on both ends of a piezoelectric body made of, for example, a piezoelectric ceramic represented by PZT (Lead Zirconate Titanate), a polymeric piezoelectric element represented by PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal represented by PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0016] The transmission / reception circuit 12, under the control of the main body control unit 28, transmits ultrasonic waves from the transducer array 11 and generates sound ray signals based on reception signals acquired by the transducer array 11. As shown in Fig. 2, the transmission / reception circuit 12 has a pulser 41 connected to the transducer array 11, and an amplifier 42, an AD (Analog to Digital) converter 43, and a beamformer 44, which are connected in series from the transducer array 11 in sequence.

[0017] The pulser 41 includes, for example, a plurality of pulse generators, and adjusts the delay amount of each drive signal and supplies it to the plurality of ultrasonic transducers of the transducer array 11 so that ultrasonic waves transmitted from the plurality of ultrasonic transducers form ultrasonic beams based on a transmission delay pattern selected in response to a control signal from the main body control unit 28. In this way, when a pulsed or continuous wave voltage is applied to the electrodes of the ultrasonic transducers of the transducer array 11, the piezoelectric body expands and contracts, and each ultrasonic transducer generates a pulsed or continuous wave ultrasonic wave, and an ultrasonic beam is formed from the composite wave of those ultrasonic waves.

[0018] The transmitted ultrasonic beam is reflected by an object such as a part of the subject, and propagates toward the transducer array 11 of the ultrasonic probe 1. The ultrasonic echo propagating toward the transducer array 11 in this manner is received by each ultrasonic transducer constituting the transducer array 11. At this time, each ultrasonic transducer constituting the transducer array 11 expands and contracts upon receiving the propagating ultrasonic echo, generating a received signal which is an electrical signal, and outputs this received signal to the amplifier 42.

[0019] The amplifier 42 amplifies the signals input from the respective ultrasonic transducers constituting the transducer array 11, and transmits the amplified signals to the AD converter 43. The AD converter 43 converts the signals transmitted from the amplifier 42 into digital reception data. The beamformer 44 performs so-called reception focusing processing by adding each piece of reception data received from the AD converter 43 with a respective delay. Through this reception focusing processing, each piece of reception data converted by the AD converter 43 is phased and added, and a sound ray signal in which the focus of the ultrasonic echo is narrowed is acquired.

[0020] As shown in FIG. 3, the image generating unit 21 has a configuration in which a signal processing unit 45, a DSC (Digital Scan Converter) 46, and an image processing unit 47 are connected in series.

[0021] The signal processing unit 45 performs correction for attenuation due to distance on the sound ray signals received from the transmission / reception circuit 12 using the sound velocity value set by the main body control unit 28 in accordance with the depth of the ultrasonic reflection position, and then performs envelope detection processing to generate a B-mode image signal, which is tomographic image information regarding the tissue in the subject.

[0022] The DSC 46 converts (raster converts) the B-mode image signal generated by the signal processor 45 into an image signal conforming to a normal television signal scanning method. The image processing unit 47 performs various necessary image processing such as gradation processing on the B-mode image signal input from the DSC 46, and then sends the B-mode image signal to the display control unit 22 and the image memory 24. Hereinafter, the B-mode image signal that has been subjected to image processing by the image processing unit 47 will be referred to as an ultrasound image.

[0023] The image memory 24 is a memory for storing ultrasonic images acquired by the image acquisition unit 32 and three-dimensional image data (described later) generated by the three-dimensional image data generation unit 25. As the image memory 24, for example, a recording medium such as a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an FD (Flexible Disk), an MO disk (Magneto-Optical disk), an MT (Magnetic Tape), a RAM (Random Access Memory), a CD (Compact Disc), a DVD (Digital Versatile Disc), an SD card (Secure Digital card), or a USB memory (Universal Serial Bus memory) can be used.

[0024] The position and orientation sensor 13 of the ultrasonic probe 1 is a sensor device that acquires the position and orientation information of the ultrasonic probe 1. Generally, when a user uses an ultrasonic diagnostic device to inspect a subject, the user often performs the inspection while changing the orientation, i.e., angle, of the ultrasonic probe 1 in contact with the body surface of the subject and moving the position of the ultrasonic probe 1. The position and orientation information of the ultrasonic probe 1 acquired by the position and orientation sensor 13 includes information on the orientation and position of the ultrasonic probe 1. The position and orientation sensor 13 may include at least one of a so-called inertial sensor, a magnetic sensor, or an optical sensor, for example. The inertial sensor may include at least one of a so-called acceleration sensor and a gyro sensor, for example.

[0025] The three-dimensional image data generating unit 25 generates three-dimensional ultrasound image data of the subject based on the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13 and the ultrasound images acquired by the image acquiring unit 32. At this time, the three-dimensional image data generating unit 25 can generate three-dimensional image data by, for example, associating the ultrasound images of multiple frames acquired by the image acquiring unit 32 with the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13 at the timing when each of the ultrasound images of the multiple frames is acquired, and arranging the ultrasound images of the multiple frames according to the corresponding position and orientation information.

[0026] The three-dimensional image data generating unit 25 can, for example, associate an ultrasound image acquired by the image acquiring unit 32 with position and orientation information input from the position and orientation sensor 13 within a certain period of time including the time point at which the ultrasound image is input to the three-dimensional image data generating unit 25 via the image memory 24. In addition, when the position and orientation sensor 13 assigns a timestamp representing the acquisition time to the position and orientation information and the image acquiring unit 32 assigns a timestamp representing the acquisition time to the ultrasound image, the three-dimensional image data generating unit 25 can associate the position and orientation information with the ultrasound image when, for example, the difference between the timestamp assigned to the position and orientation information and the timestamp assigned to the ultrasound image is within a certain range.

[0027] The three-dimensional image data thus generated by the three-dimensional image data generating unit 25 is sent to the image memory 24 , the three-dimensional structure information extracting unit 26 and the position estimating unit 27 .

[0028] The three-dimensional structure information extraction unit 26 extracts three-dimensional structure information related to the three-dimensional structure contained in the three-dimensional ultrasound image data from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit 25. The three-dimensional structure information extraction unit 26 can extract, for example, the running pattern of the blood vessels of the subject contained in the three-dimensional ultrasound image data as the three-dimensional structure information. The running patterns of the blood vessels, especially the veins, are relatively small in difference between subjects, and are therefore useful for grasping the three-dimensional structure inside the subject. In addition, the three-dimensional structure information extraction unit 26 can also extract, for example, the running pattern of the blood vessels, as well as the shape pattern and size of the structures present around the blood vessels, such as the diaphragm or the intestine, as the three-dimensional structure information. In addition, the three-dimensional structure information extraction unit 26 can also extract the brightness pattern of these three-dimensional structures as the three-dimensional structure information.

[0029] The position estimation unit 27 has a position estimation model that has learned the position of the three-dimensional structure in the three-dimensional image data in which the three-dimensional structure of the subject is captured, and estimates the position of the area occupied by the three-dimensional ultrasound image data of the subject using the position estimation model based on the three-dimensional structure information extracted by the three-dimensional structure information extraction unit 26.

[0030] The position estimation model is a so-called machine learning model, and a so-called deep learning model, a so-called SVM (Support Vector Machine), a so-called decision tree model, or the like can be used as the position estimation model. The position estimation model learns in advance the relationship between the positions of three-dimensional structures in a large amount of three-dimensional image data and the positions of the areas occupied by the three-dimensional ultrasound image data of the subject, and outputs the positions of the areas occupied by the three-dimensional ultrasound image data of the subject by inputting three-dimensional structure information. The position estimation model can learn the positions of the running patterns of blood vessels in the subject and the positions of the shape patterns of structures existing around the blood vessels, such as the diaphragm or intestines, as the positions of the three-dimensional structures in the three-dimensional image data. The position estimation model can also learn the sizes of the shape patterns of structures existing around the blood vessels.

[0031] The position of the region occupied by the three-dimensional ultrasound image data of the subject refers to the position of the region within the subject that corresponds to the three-dimensional ultrasound image data. In addition, the three-dimensional image data that the position estimation model learns can be three-dimensional ultrasound image data, three-dimensional image data formed from so-called CT (Computed Tomography) images, or three-dimensional image data formed from so-called MRI (Magnetic Resonance Imaging) images.

[0032] The position of the area occupied by the three-dimensional ultrasound image data of the subject, estimated in this manner by position estimation unit 27, is displayed on monitor 23 via display control unit 22. Position estimation unit 27 can display the position of the area occupied by the three-dimensional ultrasound image data of the subject, for example, by superimposing it on a three-dimensional human body model representing the entire subject or a part of the subject, such as the abdomen.

[0033] By the way, a user usually sequentially captures ultrasound images while determining the captured part of the subject by checking the captured ultrasound images, but a user who is not very skilled in examinations using an ultrasound diagnostic device may have difficulty determining which part of the subject is being captured even when checking the ultrasound images. Since the position estimation unit 27 in the first embodiment can accurately estimate the position of the area occupied by the three-dimensional ultrasound image data of the subject by using a position estimation model, the user can easily grasp the imaging location in the subject by checking the position of the area occupied by the estimated three-dimensional ultrasound image data.

[0034] Under the control of the main body control unit 28, the display control unit 22 performs predetermined processing on the ultrasound images sent from the image acquisition unit 32 and the image memory 24, the three-dimensional image data of the subject generated by the three-dimensional image data generation unit 25, and the position of the area occupied by the three-dimensional ultrasound image data of the subject estimated by the position estimation unit 27, and displays the results on the monitor 23.

[0035] The monitor 23, under the control of the display control unit 22, displays the ultrasound image, the three-dimensional image data, and the position of the area occupied by the three-dimensional ultrasound image data of the subject, and has a display device such as an LCD (Liquid Crystal Display) or an organic EL display (Organic Electroluminescence Display).

[0036] The main body control unit 28 controls each part of the device main body 2 and the transmission / reception circuit 12 of the ultrasonic probe 1 based on a control program or the like stored in advance.

[0037] The input device 29 is used by the user to perform input operations, and is composed of devices such as a keyboard, a mouse, a trackball, a touch pad, and a touch sensor disposed over the monitor 23, for example.

[0038] The processor 31 having the image generation unit 21, the display control unit 22, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, the position estimation unit 27 and the main body control unit 28 is composed of a CPU (Central Processing Unit) and a control program for causing the CPU to perform various processes, but may also be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit) or other ICs (Integrated Circuits), or may be composed of a combination of these.

[0039] In addition, the image generation unit 21, display control unit 22, three-dimensional image data generation unit 25, three-dimensional structural information extraction unit 26, position estimation unit 27 and main body control unit 28 of the processor 31 can be configured as a single CPU or the like, either partially or entirely integrated.

[0040] Next, the operation of the ultrasound diagnostic apparatus according to the first embodiment will be described with reference to the flowchart shown in Fig. 4. In the following, it is assumed that the user brings the ultrasound probe 1 into contact with the body surface of the subject.

[0041] In step S1, the position and orientation sensor 13 acquires the position and orientation information of the ultrasonic probe 1. The position and orientation information of the ultrasonic probe 1 acquired in step S1 is sent to the three-dimensional image data generator 25.

[0042] In step S2, the image acquisition unit 32 acquires an ultrasonic image of the subject. At this time, under the control of the main body control unit 28, transmission and reception of ultrasonic waves is started from the multiple transducers of the transducer array 11 in accordance with a drive signal from the pulsar 41 of the transmission and reception circuit 12 of the ultrasonic probe 1, ultrasonic echoes from within the subject are received by the multiple transducers of the transducer array 11, and the received signals, which are analog signals, are output to the amplifier 42 and amplified, and are AD converted by the AD conversion unit 43 to obtain received data.

[0043] The beamformer 44 performs reception focusing processing on this reception data, and the sound ray signal generated thereby is sent to the image generating unit 21 of the device body 2, which generates an ultrasound image showing tomographic image information of the subject. At this time, the signal processing unit 45 of the image generating unit 21 performs attenuation correction and envelope detection processing according to the depth of the reflection position of the ultrasound on the sound ray signal, the DSC 46 converts it into an image signal according to the scanning method of a normal television signal, and the image processing unit 47 performs various necessary image processing such as gradation processing. The ultrasound image thus acquired in step S2 is displayed on the monitor 23 via the display control unit 22, and is also sent to the three-dimensional image data generating unit 25 via the image memory 24.

[0044] The position and orientation information of the ultrasound probe 1 acquired in step S1 and the ultrasound image acquired in step S2 are associated with each other by the three-dimensional image data generating unit 25. The three-dimensional image data generating unit 25 can, for example, associate the ultrasound image acquired in step S1 with position and orientation information input within a certain period of time including the time point at which the ultrasound image was input to the three-dimensional image data generating unit 25.

[0045] In step S3, the main body control unit 28 determines whether or not sufficient ultrasound images have been acquired for generating three-dimensional ultrasound image data. The main body control unit 28 can determine whether or not sufficient ultrasound images have been acquired by, for example, determining whether or not a predetermined time has elapsed since the start of step S1. For example, the main body control unit 28 can determine that sufficient ultrasound images have been acquired when a predetermined time has elapsed since the start of step S1, and can determine that sufficient ultrasound images have not been acquired when the predetermined time has not yet elapsed.

[0046] Furthermore, the main body control unit 28 can also determine whether or not an ultrasound image has been acquired sufficiently by, for example, referring to the position and orientation information acquired in step S1 and determining whether or not the movement of the ultrasound probe 1 has stopped after being moved by the user. For example, the main body control unit 28 can determine that an ultrasound image has been acquired sufficiently when the movement of the ultrasound probe 1 has stopped after being moved by the user, and can determine that an ultrasound image has not been acquired sufficiently when the ultrasound probe 1 continues to move.

[0047] If it is determined in step S3 that sufficient ultrasound images to be used for generating three-dimensional ultrasound image data have not been acquired, the process returns to step S1, where the position and orientation sensor 13 newly acquires position and orientation information of the ultrasound probe 1, and in the following step S2, the image acquisition unit 32 newly acquires an ultrasound image, and in step S3, it is determined whether sufficient ultrasound images have been acquired. In this manner, the processes of steps S1 to S3 are repeated until it is determined in step S3 that sufficient ultrasound images have been acquired.

[0048] If it is determined in step S3 that sufficient ultrasound images have been acquired to generate the three-dimensional ultrasound image data, the process proceeds to step S4. In step S4, the three-dimensional image data generating unit 25 generates three-dimensional ultrasound image data of the subject based on the position and orientation information of the ultrasound probe 1 obtained by repeating step S1 and the ultrasound images of the multiple frames obtained by repeating step S2. At this time, the three-dimensional image data generating unit 25 can generate the three-dimensional ultrasound image data by, for example, arranging the ultrasound images of the multiple frames obtained by repeating step S2 according to the position and orientation information of the ultrasound probe 1 associated with each ultrasound image.

[0049] In step S5, the three-dimensional structure information extraction unit 26 extracts three-dimensional structure information related to the three-dimensional structure contained in the three-dimensional ultrasound image data from the three-dimensional ultrasound image data generated in step S4. The three-dimensional structure information extraction unit 26 can extract, for example, the running pattern of the subject's blood vessels contained in the three-dimensional ultrasound image data, and the shape pattern and size of structures existing around the blood vessels, etc., as the three-dimensional structure information.

[0050] In step S6, position estimation unit 27 estimates the position of the area occupied by the 3D ultrasound image data of the subject using a position estimation model based on the 3D structure information extracted in step S5. The position estimation model has previously learned the relationship between the positions of the 3D structures in a large amount of 3D ultrasound image data and the positions of the area occupied by the 3D ultrasound image data of the subject using a machine learning method, and outputs the position of the area occupied by the 3D ultrasound image data of the subject when the 3D structure information is input.

[0051] The position of the area occupied by the three-dimensional ultrasound image data of the subject, estimated in this manner by position estimation unit 27, is displayed on monitor 23. At this time, position estimation unit 27 can display the position of the area occupied by the three-dimensional ultrasound image data of the subject, for example, by superimposing it on a three-dimensional human body model representing the entire subject or a part of the subject, such as the abdomen.

[0052] Although the internal structure of the subject varies from person to person, the position estimation unit 27 estimates the position of the area occupied by the subject's 3D ultrasound image data using a position estimation model, and therefore, even if the internal structure of the subject differs from, for example, an ideal structure, the position of the area occupied by the subject's 3D ultrasound image data can be accurately estimated. Therefore, by checking the position of the area occupied by the subject's 3D ultrasound image data displayed on the monitor 23, the user can easily grasp the imaging location of the subject, regardless of the user's level of skill.

[0053] This allows the user to easily grasp, for example, whether or not the location to be examined in the subject has been comprehensively imaged. Furthermore, since the position estimation unit 27 can accurately estimate the position of the area occupied by the three-dimensional ultrasound image data of the subject, the user can comprehensively image the location to be examined in the subject, making it easier to find a locally existing lesion, such as a tumor located around a blood vessel. Furthermore, a user with low skill level can easily improve his / her skill level by performing imaging while checking the position of the area occupied by the three-dimensional ultrasound image data of the subject displayed on the monitor 23, and the trouble of an expert checking the imaging results can be eliminated.

[0054] Finally, in step S7, the main body control unit 28 judges whether or not to end the test on the subject. For example, the main body control unit 28 judges to end the test when the user judges that the test on the subject has been sufficiently performed and inputs an instruction to end the test via the input device 29, and can judge to continue the test when the user does not particularly input an instruction to end the test via the input device 29.

[0055] If it is determined in step S7 that the examination should be continued, the process returns to step S1. Thereafter, the processes of steps S1 to S7 are performed again. If it is determined in step S7 that the examination should be ended, the operation of the ultrasonic diagnostic apparatus according to the flowchart in FIG. 4 is completed.

[0056] As described above, according to the ultrasound diagnostic apparatus of the first embodiment, the three-dimensional image data generating unit 25 generates three-dimensional ultrasound image data of the subject based on the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13 and the ultrasound image acquired by the image acquiring unit 32, the three-dimensional structure information extracting unit 26 extracts three-dimensional structure information related to the three-dimensional structure contained in the three-dimensional ultrasound image data from the three-dimensional ultrasound image data generated by the three-dimensional image data generating unit 25, and the position estimating unit 27 has a position estimation model that has learned the position of the three-dimensional structure in the three-dimensional ultrasound image data in which the three-dimensional structure is captured, and estimates the position of the area occupied by the three-dimensional ultrasound image data of the subject using the position estimation model based on the three-dimensional structure information extracted by the three-dimensional structure information extracting unit 26. Therefore, the position of the area occupied by the three-dimensional ultrasound image data of the subject can be accurately estimated, and the user can easily grasp the imaging location of the subject regardless of his / her level of skill.

[0057] Although the transmitting / receiving circuit 12 has been described as being provided in the ultrasonic probe 1, the transmitting / receiving circuit 12 may be provided in the device body 2. Further, although the image generating unit 21 is described as being provided in the device body 2, the image generating unit 21 may be provided in the ultrasound probe 1.

[0058] The device body 2 may be a so-called stationary type, a portable type that is easy to carry, or a so-called handheld type that is configured by, for example, a smartphone or a tablet computer. In this way, the type of device that configures the device body 2 is not particularly limited.

[0059] In addition, in the flow chart of FIG. 4, step S2 is performed after step S1, but step S1 may be performed after step S2, or step S1 and step S2 may be performed simultaneously.

[0060] The ultrasound diagnostic apparatus may also include a coverage determination unit (not shown) that determines whether or not an area to be observed has been comprehensively photographed. This coverage determination unit is connected to, for example, the display control unit 22, the position estimation unit 27, and the main body control unit 28. The coverage determination unit can determine whether or not an area to be observed has been comprehensively photographed by determining whether or not a three-dimensional area of ​​the object to be observed in the subject input by a user via the input device 29 and set by the main body control unit 28 is filled without gaps with an area corresponding to the three-dimensional ultrasound image data whose position has been estimated by the position estimation unit 27.

[0061] In this case, for example, instead of step S7 in the flowchart of Fig. 4, a step of determining whether or not the observation target has been comprehensively photographed by the comprehensive determination unit can be provided. For example, the main body control unit 28 causes each unit of the ultrasound diagnostic device to perform the processes of steps S1 to S6 until the comprehensive determination unit determines that the observation target has been comprehensively photographed, and can complete the operation of the ultrasound diagnostic device when the comprehensive determination unit determines that the observation target has been comprehensively photographed. In this way, the ultrasound diagnostic device determines whether or not the observation target has been comprehensively photographed, so that the user can reliably perform comprehensive photography of the observation target.

[0062] Embodiment 2 Due to various factors when acquiring multiple frames of ultrasound images, deviations may occur in the position and orientation information of the ultrasound probe 1 corresponding to the acquired multiple frames of ultrasound images, and the surface of the three-dimensional structure in the three-dimensional ultrasound image data generated by the three-dimensional image data generating unit 25 may have an uneven shape that does not actually exist. Therefore, the ultrasound diagnostic device can reconstruct the three-dimensional ultrasound image data so that the three-dimensional structure in the three-dimensional ultrasound image data has a smooth shape.

[0063] Fig. 5 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to the embodiment 2. The ultrasonic diagnostic apparatus according to the embodiment includes a device body 2A instead of the device body 2 in the ultrasonic diagnostic apparatus according to the embodiment 1 shown in Fig. 1. The device body 2A further includes a reconstruction unit 51 in the device body 2 in the embodiment 1, and includes a body control unit 28A instead of the body control unit 28.

[0064] In the device body 2A, a reconstruction unit 51 is connected to the position and orientation sensor 13 and the three-dimensional image data generation unit 25. The reconstruction unit 51 is connected to the image memory 24, the three-dimensional structural information extraction unit 26, the position estimation unit 27, and the body control unit 28A. The image generation unit 21, the display control unit 22, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, the position estimation unit 27, the body control unit 28A, and the reconstruction unit 51 configure a processor 31A for the device body 2A.

[0065] The reconstruction unit 51 extracts a two-dimensional tomographic image from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit 25, and reconstructs three-dimensional ultrasound image data of the subject based on the extracted tomographic image and the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13. The reconstruction unit 51 can extract a region of a two-dimensional tomographic image of a structure included in the three-dimensional ultrasound image data by, for example, a pixel value analysis method such as binarization or multi-valueization of pixels in the three-dimensional ultrasound image data, a so-called template matching method, a so-called graph cut method, or a method using a deep learning model. The reconstruction unit 51 can further smooth the surface of a three-dimensional structure in the three-dimensional ultrasound image data by performing a process of smoothing the contour of the structure in the multiple two-dimensional tomographic images and a process of interpolating between the multiple two-dimensional tomographic images.

[0066] By reconstructing the three-dimensional ultrasound image data by the reconstruction unit 51, the three-dimensional structure contained in the three-dimensional ultrasound image data can be made to approximate the shape of the actual three-dimensional structure in the subject.

[0067] The three-dimensional structural information extraction unit 26 extracts three-dimensional structural information from the three-dimensional ultrasound image data reconstructed by the reconstruction unit 51 . The position estimation unit 27 uses the position estimation model to estimate the position of the area occupied by the three-dimensional ultrasound image data of the subject from the three-dimensional structural information extracted by the three-dimensional structural information extraction unit 26 .

[0068] In this way, the position of the area occupied by the subject's three-dimensional ultrasound image data is estimated based on three-dimensional structural information extracted from the three-dimensional ultrasound image data reconstructed by the reconstruction unit 51, thereby improving the accuracy of estimating the position of the area occupied by the subject's three-dimensional ultrasound image data compared to when the position of the area occupied by the subject's three-dimensional ultrasound image data is estimated based on three-dimensional structural information extracted directly from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit 25.

[0069] Third embodiment The ultrasonic diagnostic apparatus can also highlight on the monitor 23 the areas inside the subject that have already been imaged so that the user can clearly understand the areas inside the subject that have already been imaged.

[0070] Fig. 6 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to embodiment 3. The ultrasonic diagnostic apparatus according to embodiment 3 includes a device body 2B instead of the device body 2 in the ultrasonic diagnostic apparatus according to embodiment 1 shown in Fig. 1. The device body 2B further includes a schema memory 52 and an emphasis display unit 53 in the device body 2 in embodiment 1, and includes a body control unit 28B instead of the body control unit 28.

[0071] In the device main body 2B, a schema memory 52 is connected to the main body control unit 28B. Furthermore, a highlighting unit 53 is connected to the position estimation unit 27 and the schema memory 52. ​​The highlighting unit 53 is connected to the display control unit 22 and the main body control unit 28B. Furthermore, the image generation unit 21, the display control unit 22, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, the position estimation unit 27, the main body control unit 28B, and the highlighting unit 53 configure a processor 31B for the device main body 2B.

[0072] The schema memory 52 is a memory that stores in advance a plurality of three-dimensional schema images A, which are three-dimensional schematic diagrams that mimic the structure of organs and the like in a subject, as shown in Fig. 7. Here, Fig. 7 shows an example of a three-dimensional schema image A including a liver B1, a gallbladder B2, a stomach B3, a spleen B4, a pancreas B5, and a duodenum B6. As the schema memory 52, for example, a recording medium such as a flash memory, an HDD, an SSD, an FD, an MO disk, an MT, a RAM, a CD, a DVD, an SD card, or a USB memory can be used.

[0073] 7, for example, the highlighting unit 53 highlights an occupied area R1 of the three-dimensional ultrasound image data whose position has been estimated by the position estimation unit 27 in the three-dimensional schematic image A on the monitor 23. In the example of Fig. 7, a schematic diagram of the ultrasound probe 1 is shown to show that the occupied area R1 corresponds to a plurality of frames of ultrasound images generated while tilting the ultrasound probe 1 within a certain angle range, but the schematic diagram of the ultrasound probe 1 does not have to be displayed.

[0074] The highlighting unit 53 can, for example, read out a three-dimensional schema image A specified by a user via the input device 29 from among the multiple three-dimensional schema images A stored in the schema memory 52, and display it on the monitor 23. In addition, the highlighting unit 53 can highlight the occupied region R1 of the three-dimensional ultrasound image data whose position has been estimated by the position estimation unit 27, by applying a color different from the surrounding color to the occupied region R1, by blinking the occupied region R1, or the like.

[0075] The user can clearly and easily grasp the areas of the subject whose ultrasound images have already been captured by checking the highlighted occupied area R1 on the monitor 23. This allows the user to reliably capture comprehensive images of the required examination areas.

[0076] An example has been described in which a three-dimensional schema image A specified by the user via the input device 29 is read out from among multiple three-dimensional schema images A stored in the schema memory 52. ​​However, for example, the highlighting unit 53 can automatically select a three-dimensional schema image A including the position of the occupied area R1 estimated by the position estimation unit 27 from the schema memory 52 and display it on the monitor 23.

[0077] Fourth embodiment When the ultrasound image acquired by the image acquisition unit 32 is unclear for some reason, the position of the occupying region R1 may not be estimated by the position estimation unit 27. In this case, the ultrasound diagnostic device can synthesize the 3D ultrasound image data, whose position could not be estimated, with the occupying region R1, which is adjacent to the 3D ultrasound image data and whose position has already been estimated.

[0078] 8 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to embodiment 4. The ultrasonic diagnostic apparatus according to embodiment 4 further includes a schema memory 52, an emphasis display unit 53, and a data synthesis unit 54 in the ultrasonic diagnostic apparatus according to embodiment 1 shown in FIG. 1, and includes a main body control unit 28C instead of the main body control unit 28. Here, the schema memory 52 and the emphasis display unit 53 in embodiment 4 are the same as the schema memory 52 and the emphasis display unit 53 in embodiment 3.

[0079] In the device main body 2C, a data synthesis unit 54 is connected to the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, and the position estimation unit 27. The data synthesis unit 54 is connected to the highlighting unit 53 and the main body control unit 28C. In addition, the image generation unit 21, the display control unit 22, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, the position estimation unit 27, the main body control unit 28C, the highlighting unit 53, and the data synthesis unit 54 configure a processor 31C for the device main body 2C.

[0080] The data synthesis unit 54 sequentially synthesizes the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit 25 based on the three-dimensional structure contained in the three-dimensional ultrasound image data in which the position of the occupying area R1 has been estimated by the position estimation unit 27.

[0081] Here, when the ultrasound image acquired by the image acquisition unit 32 is unclear for some reason, the position estimation unit 27 may not be able to estimate the position of the occupation region R1. In this case, the data synthesis unit 54, for example, refers to the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13, and synthesizes the three-dimensional ultrasound image data whose position could not be estimated by the position estimation unit 27 to a three-dimensional structure adjacent to the three-dimensional ultrasound image data and included in the three-dimensional ultrasound image data whose position of the occupation region R1 has already been estimated.

[0082] The data synthesis unit 54 further synthesizes the synthesized 3D ultrasound image data with 3D ultrasound image data at a position adjacent to the synthesized 3D ultrasound image data. In this manner, the data synthesis unit 54 can sequentially synthesize the 3D ultrasound image data.

[0083] The highlighting unit 53 highlights in the three-dimensional schematic image A an occupied area R1 of the three-dimensional ultrasound image data whose position has been estimated by the position estimation unit 27 and an occupied area R2 of the three-dimensional ultrasound image data combined by the data combination unit 54.

[0084] Therefore, even if the position estimation unit 27 is unable to identify the position of the occupied area R1 for some reason, the user can check the occupied areas R1 and R2 that are highlighted on the three-dimensional schematic image A, thereby easily understanding the areas within the subject where ultrasound images have already been captured, and comprehensively capturing images of the area to be examined.

[0085] Fifth embodiment If for some reason the position estimation unit 27 is unable to estimate the position of the occupied area R1 of the three-dimensional ultrasound image data, the ultrasound diagnostic device can also notify the user to return the ultrasound probe 1 to the posture and position it had when the three-dimensional ultrasound image data for which the position of the occupied area R1 had already been estimated was acquired.

[0086] 9 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to the fifth embodiment. The ultrasonic diagnostic apparatus according to the fifth embodiment includes a device body 2D instead of the device body 2 in the ultrasonic diagnostic apparatus according to the first embodiment shown in FIG. The device body 2D further includes a schema memory 52, an emphasis display unit 53, and a notification unit 55 in the device body 2 in the first embodiment, and includes a body control unit 28D instead of the body control unit 28. Here, the schema memory 52 and the emphasis display unit 53 in the fifth embodiment are the same as the schema memory 52 and the emphasis display unit 53 in the third and fourth embodiments.

[0087] In the device body 2D, a notification unit 55 is connected to the position and orientation sensor 13 and the position estimation unit 27. The notification unit 55 is connected to the display control unit 22 and the body control unit 28D. The image generation unit 21, the display control unit 22, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, the position estimation unit 27, the body control unit 28D, the highlighting unit 53, and the notification unit 55 configure a processor 31D for the device body 2D.

[0088] When the position estimation unit 27 is unable to estimate the position of the occupied area R1 of the three-dimensional ultrasound image data, the notification unit 55 prompts the user to move the ultrasound probe 1 based on the position and orientation information of the ultrasound probe 1 acquired by the position and orientation sensor 13 so that the position and orientation of the ultrasound probe 1 are the same as when the three-dimensional ultrasound image data whose position of the occupied area R1 has already been estimated by the position estimation unit 27 was acquired.

[0089] The notification unit 55 can, for example, acquire the final position and orientation of the ultrasound probe 1 when the three-dimensional ultrasound image data was acquired by referring to the position and orientation information when the three-dimensional ultrasound image data in which the position of the occupied region R1 has already been estimated by the position estimation unit 27 was acquired. The notification unit 55 can further, for example, superimpose the position and orientation of the ultrasound probe 1 acquired in this way on a schematic diagram (not shown) of the subject viewed from the outside and display it on the monitor 23, thereby urging the user to move the ultrasound probe 1. At this time, the notification unit 55 can display a message on the monitor 23 such as "Please move the ultrasound probe so that the position and orientation of the ultrasound probe are as shown on the schematic diagram."

[0090] In addition, the notification unit 55 can calculate the difference between the last position and posture of the ultrasound probe 1 when the three-dimensional ultrasound image data was acquired in which the position of the occupied area R1 has already been estimated by the position estimation unit 27, and the current position and posture of the ultrasound probe 1, and based on the calculated difference, display a message on the monitor 23 such as "Move the ultrasound probe xx cm in the xx direction and tilt it xx degrees toward the body surface," thereby notifying the user of the specific movement direction and posture of the ultrasound probe 1.

[0091] Regardless of the user's level of proficiency, the user can return the ultrasonic probe 1 to the position and posture when the 3D ultrasound image data in which the position of the occupied region R1 has already been estimated by the position estimation unit 27 was acquired by checking the contents of the notification by the notification unit 55, and reacquire the 3D ultrasound image data in the position of the occupied region R1 that the position estimation unit 27 was unable to estimate. This ensures that the position estimation unit 27 can successfully estimate the position of the occupied region R1, and enables the highlighting unit 53 to display the occupied region R1 accurately. While checking the occupied region R1 highlighted on the monitor 23 in this way, the user can more reliably and comprehensively image the required region in the subject.

[0092] The notification unit 55 can also specify an area not yet highlighted by the highlighting unit 53 in the three-dimensional schematic image A displayed on the monitor 23 and notify the user of the specified area. At this time, the notification unit 55 can notify the user of the area not yet highlighted by the highlighting unit 53 by, for example, displaying on the monitor 23 the position on the subject's body surface corresponding to the area not yet highlighted in the three-dimensional schematic image A and the attitude of the ultrasound probe 1 at that position superimposed on a schematic diagram (not shown) of the subject viewed from the outside.

[0093] In addition, the notification unit 55 estimates the position and orientation of the ultrasound probe 1 for photographing areas that are not highlighted by the highlighting unit 53 based on multiple pieces of position and orientation information already acquired by the position and orientation sensor 13, and displays the direction and distance in which the current ultrasound probe 1 should be moved and the angle at which the current ultrasound probe 1 should be tilted on the monitor 23 using messages or the like, thereby notifying the user of areas that are not highlighted by the highlighting unit 53.

[0094] In this way, the notification unit 55 notifies the user of areas that are not highlighted by the highlighting unit 53, so that the user can easily and reliably capture comprehensive images of the areas within the subject to be examined.

[0095] The ultrasound diagnostic apparatus of the fifth embodiment may also include a data synthesis unit 54, for example, as in the ultrasound diagnostic apparatus of the fourth embodiment. If the position estimation unit 27 is unable to estimate the position of the occupied region R1 and the data synthesis unit 54 sequentially synthesizes the three-dimensional ultrasound image data, slight positional deviations that occur when synthesizing the three-dimensional ultrasound image data may accumulate, making it impossible to synthesize the three-dimensional ultrasound image data at the correct position.

[0096] Therefore, when the three-dimensional ultrasound image data cannot be synthesized at the correct position, the notification unit 55 can prompt the user to move the ultrasound probe 1 so that it is at the last position and posture of the ultrasound probe 1 when the three-dimensional ultrasound image data in which the position estimation unit 27 last estimated the position of the occupied area R1 was acquired, or at the position and posture of the ultrasound probe 1 that is closest to the current position of the ultrasound probe 1 and when the three-dimensional ultrasound image data in which the position estimation unit 27 estimated the position of the occupied area R1 was acquired.

[0097] This allows the user to redo the acquisition of an ultrasound image from the position and orientation of the ultrasound probe 1 when the three-dimensional ultrasound image data in which the position of the occupied region R1 was estimated by the position estimation unit 27 was acquired. This allows the position estimation unit 27 to reliably and successfully estimate the position of the occupied region R1, and enables the highlighting unit 53 to display the occupied region R1 accurately. While checking the occupied region R1 highlighted on the monitor 23 in this way, the user can more reliably and comprehensively capture the required region within the subject.

[0098] The notification unit 55 has an interval threshold value for the interval between adjacent combined 3D ultrasound image data, and can determine that the 3D ultrasound image data has been combined at a correct position when the interval between the adjacent combined 3D ultrasound image data is equal to or less than the interval threshold value. Also, the notification unit 55 can determine that the 3D ultrasound image data has not been combined at a correct position when the interval between the adjacent combined 3D ultrasound image data is wider than the interval threshold value.

[0099] The notification unit 55 can also determine whether the three-dimensional ultrasound image data has been combined at a correct position based on the similarity of the boundary portion in two pieces of three-dimensional ultrasound image data that are adjacent to each other and are to be combined with each other. The notification unit 55 has a similarity threshold value for the similarity of the boundary portion in the two pieces of three-dimensional ultrasound image data, calculates the similarity of the boundary portion in the two pieces of three-dimensional ultrasound image data, and determines that the three-dimensional ultrasound image data has been combined at a correct position when the calculated similarity is equal to or greater than the similarity threshold, and determines that the three-dimensional ultrasound image data has not been combined at a correct position when the calculated similarity is less than the similarity threshold.

[0100] The notification unit 55 has a smoothness threshold value for the smoothness of the three-dimensional structure in the three-dimensional ultrasound data after synthesis, calculates the smoothness of the three-dimensional structure in the three-dimensional ultrasound data after synthesis, and determines that the three-dimensional ultrasound image data has been synthesized at the correct position when the calculated smoothness is equal to or greater than the smoothness threshold value, and determines that the three-dimensional ultrasound image data has not been synthesized at the correct position when the calculated smoothness is less than the smoothness threshold value. Here, the smoothness of the three-dimensional structure is an index value having a higher value as the surface of the three-dimensional structure has a smoother shape, and a lower value as the surface of the three-dimensional structure has an uneven shape.

[0101] In addition, the notification unit 55 pre-stores the ideal three-dimensional structure within the subject, and calculates the similarity between the three-dimensional structure in the synthesized three-dimensional ultrasound image data and the ideal three-dimensional structure, and can determine whether the three-dimensional ultrasound image data has been synthesized in the correct position based on whether the similarity is equal to or greater than a specified threshold value.

[0102] Also, although it has been described that the notification unit 55 notifies the user by displaying a message or the like on the monitor 23, the notification method by the notification unit 55 is not particularly limited thereto. For example, when the ultrasonic diagnostic device has a speaker (not shown), the notification unit 55 can notify the user by sound via the speaker. Also, for example, when the ultrasonic probe 1 has a vibration device (not shown), the notification unit 55 can notify the user by vibrating the ultrasonic probe 1 with the vibration device. Here, the vibration device includes a so-called vibration motor or the like, and is a device that vibrates the ultrasonic probe 1. Also, for example, when the ultrasonic diagnostic device has a lamp (not shown), the notification unit 55 can notify the user by turning on the lamp. In this way, the notification unit 55 can notify the user by appealing to the five senses of the user by various methods.

[0103] Sixth embodiment A user who is not very skilled in an examination using an ultrasound diagnostic device may have difficulty in grasping what kind of ultrasound image will be acquired at what part of the subject, and may not be able to grasp whether the intended part has been imaged even if the acquired ultrasound image is checked. Therefore, the ultrasound diagnostic device can present the ultrasound image to be observed in advance.

[0104] FIG. 10 shows a block diagram illustrating the configuration of an ultrasonic diagnostic apparatus according to the sixth embodiment. The ultrasonic diagnostic apparatus according to the sixth embodiment includes a device body 2E instead of the device body 2 in the ultrasonic diagnostic apparatus according to the first embodiment shown in FIG. The device body 2E further includes a schema memory 52, an emphasis display unit 53, a notification unit 55, a reference image memory 56, and a reference image presenting unit 57 in the device body 2 according to the first embodiment, and includes a body control unit 28E instead of the body control unit 28. Here, the schema memory 52 and the emphasis display unit 53 are the same as the schema memory 52 and the emphasis display unit 53 in the third to fifth embodiments, and the notification unit 55 is the same as the notification unit 55 in the fifth embodiment.

[0105] In the device main body 2E, a reference image memory 56 is connected to the main body control unit 28E. Furthermore, a reference image presenting unit 57 is connected to the notification unit 55 and the reference image memory 56. The reference image presenting unit 57 is connected to the display control unit 22 and the main body control unit 28E. Furthermore, the image generation unit 21, the display control unit 22, the three-dimensional image data generation unit 25, the three-dimensional structural information extraction unit 26, the position estimation unit 27, the main body control unit 28E, the highlighting unit 53, the notification unit 55 and the reference image presenting unit 57 constitute a processor 31E for the device main body 2E.

[0106] The reference image memory 56 is a memory that stores in advance a plurality of reference ultrasound images to be observed at each part inside the subject. As the plurality of reference ultrasound images, for example, a plurality of ultrasound images acquired in a past examination and stored in association with the position and orientation information of the ultrasound probe 1, a plurality of ultrasound images acquired by another ultrasound diagnostic device (not shown) and stored in association with the position and orientation information of the ultrasound probe used in the ultrasound diagnostic device, etc. can be used. In addition, as the reference image memory 56, for example, a recording medium such as a flash memory, HDD, SSD, FD, MO disk, MT, RAM, CD, DVD, SD card, or USB memory can be used.

[0107] The notification unit 55 identifies an area that is not highlighted by the highlighting unit 53 in the three-dimensional schematic image A displayed on the monitor 23, and notifies the user of the identified area.

[0108] The reference image presenting unit 57 reads out a reference ultrasound image to be observed in the area specified by the notification unit 55 from the reference image memory 56 and presents it on the monitor 23. The reference image presenting unit 57 can, for example, refer to the position and orientation information of the ultrasound probe 1 corresponding to the multiple reference ultrasound images stored in the reference image memory 56, extract at least one reference ultrasound image corresponding to the position of the area specified by the notification unit 55, and present the at least one reference ultrasound image on the monitor 23.

[0109] Since the user captures ultrasound images while checking the reference ultrasound image presented on the monitor 23, even if the user has a low level of proficiency, the user can sequentially capture ultrasound images while easily understanding whether the intended area has been captured or not. This allows the user to comprehensively capture the required areas in the subject more reliably. [Explanation of symbols]

[0110] 1 Ultrasonic probe, 2, 2A, 2B, 2C, 2D, 2E Device body, 11 Transducer array, 12 Transmitting and receiving circuit, 13 Position and orientation sensor, 21 Image generation unit, 22 Display control unit, 23 Monitor, 24 Image memory, 25 Three-dimensional image data generation unit, 26 Three-dimensional structure information extraction unit, 27 Position estimation unit, 28, 28A, 28B, 28C, 28D, 28E Main body control unit, 29 Input device, 31, 31A, 31B, 31C, 31D, 31E Processor, 32 Image acquisition unit, 41 Pulser, 42 Amplification unit, 43 AD conversion unit, 44 Beamformer, 45 Signal processing unit, 46 DSC, 47 Image processing unit, 51 Reconstruction unit, 52 Schematic memory, 53 Highlighting unit, 54 Data synthesis unit, 55 Notification unit, 56 Reference image memory, 57 Reference image presentation area, A three-dimensional schematic image, B1 liver, B2 gallbladder, B3 stomach, B4 spleen, B5 pancreas, B6 duodenum, R1 occupied area.

Claims

1. An ultrasonic probe; a position and orientation sensor for acquiring position and orientation information of the ultrasonic probe; an image acquisition unit that acquires an ultrasonic image representing a cross section of a subject by transmitting and receiving an ultrasonic beam using the ultrasonic probe; a three-dimensional image data generating unit that generates three-dimensional ultrasound image data of the subject based on the position and orientation information of the ultrasound probe acquired by the position and orientation sensor and the ultrasound image acquired by the image acquiring unit; a three-dimensional structure information extraction unit that extracts three-dimensional structure information related to a three-dimensional structure included in the three-dimensional ultrasound image data from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit; a position estimation unit having a position estimation model that has learned a position of the three-dimensional structure in the three-dimensional image data obtained by capturing the three-dimensional structure, and estimating a position of an area occupied by the three-dimensional ultrasound image data of the subject using the position estimation model based on the three-dimensional structure information extracted by the three-dimensional structure information extraction unit; An ultrasound diagnostic device comprising:

2. a reconstruction unit that extracts a two-dimensional tomographic image from the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit, and reconstructs the three-dimensional ultrasound image data of the subject based on the extracted tomographic image and position and orientation information of the ultrasound probe acquired by the position and orientation sensor, The ultrasound diagnostic apparatus according to claim 1 , wherein the three-dimensional structural information extraction unit extracts the three-dimensional structural information from the three-dimensional ultrasound image data reconstructed by the reconstruction unit.

3. A monitor for displaying a three-dimensional schematic image; 2. The ultrasound diagnostic apparatus according to claim 1, further comprising an emphasis display unit that highlights, in the 3D schematic image, an area occupied by the 3D ultrasound image data, the position of which has been estimated by the position estimation unit.

4. 4. The ultrasound diagnostic apparatus according to claim 3, further comprising a data synthesis unit that sequentially synthesizes the three-dimensional ultrasound image data generated by the three-dimensional image data generation unit based on the three-dimensional structure included in the three-dimensional ultrasound image data whose position of the occupied area has been estimated by the position estimation unit.

5. 4. The ultrasound diagnostic apparatus according to claim 3, further comprising: a notification unit that, when the position estimation unit is unable to estimate the position of the occupied area of ​​the three-dimensional ultrasound image data, prompts a user to move the ultrasound probe so that the position and orientation of the ultrasound probe are the same as those when the three-dimensional ultrasound image data, whose position of the occupied area has already been estimated by the position estimation unit, was acquired, based on position and orientation information of the ultrasound probe acquired by the position and orientation sensor.

6. The ultrasonic diagnostic apparatus according to claim 3 , further comprising a notification unit that identifies an area not highlighted by the highlighting unit and notifies a user of the identified area.

7. The ultrasonic diagnostic apparatus according to claim 6 , further comprising a reference image presenting unit presenting a reference ultrasonic image to be observed in the region specified by the notification unit.

8. The ultrasonic diagnostic apparatus according to claim 1 , wherein the three-dimensional structure information includes a shape pattern of the three-dimensional structure.

9. The ultrasonic diagnostic apparatus according to claim 8 , wherein the three-dimensional structure information includes a brightness pattern of the three-dimensional structure.

10. The ultrasonic diagnostic apparatus according to claim 1 , wherein the position and orientation sensor includes an inertial sensor, a magnetic sensor, or an optical sensor.

11. Acquire position and orientation information of the ultrasound probe; obtaining an ultrasonic image representing a cross section of a subject by transmitting and receiving an ultrasonic beam using the ultrasonic probe; generating three-dimensional ultrasound image data of the subject based on the acquired position and orientation information of the ultrasound probe and the acquired ultrasound image; extracting three-dimensional structure information relating to a three-dimensional structure contained in the three-dimensional ultrasound image data from the generated three-dimensional ultrasound image data; The extracted three-dimensional structure information is input to a position estimation model that has learned the position of the three-dimensional structure in the three-dimensional image data in which the three-dimensional structure is captured, thereby estimating the position of the area occupied by the three-dimensional ultrasound image data of the subject. A method for controlling an ultrasonic diagnostic device.

Images