Ultrasonic diagnostic device and program

Abstract

To assist an operator in performing operations when looking for a medical instrument on an ultrasonic image.SOLUTION: An ultrasonic diagnostic device according to an embodiment comprises a first generation unit, a second generation unit and an estimation unit. The first generation unit executes a first scanning with an ultrasonic probe on an analyte and thereby generates a first ultrasonic image of the analyte. The second generation unit executes a second scanning encompassing the first scanning and thereby generates a second ultrasonic image containing a medical instrument. The estimation unit estimates first positional information for the ultrasonic probe capable of generating the first ultrasonic image containing a medical instrument, based on the second ultrasonic image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in the present specification and drawings relate to an ultrasound diagnostic apparatus and a program. [Background technology]

[0002] Conventionally, ultrasonic diagnostic devices for imaging medical instruments used in catheter interventions for structural heart disease (SHD) have been known. This type of ultrasonic diagnostic device generates an ultrasonic image including the medical instrument based on the output of an ultrasonic probe and displays the ultrasonic image on a display. For example, during catheter intervention, the ultrasonic diagnostic device generates and displays an ultrasonic image including the tip of the medical instrument based on the output of a transesophageal probe, regarding the heart of a subject into which a medical instrument such as a guide wire or a device (clip or artificial valve) is inserted. In this case, from the standpoint of frame rate and resolution, for example, the ultrasonic diagnostic device mainly displays a two-dimensional ultrasonic image including the tip of the medical instrument.

[0003] However, for example, if the operator loses sight of the tip of the medical instrument or is observing a heart valve or other part other than the tip, the ultrasound diagnostic device displays a 2D ultrasound image that does not include the tip of the medical instrument. After this display, the operator searches for the tip on the 2D ultrasound image in order to observe it again. For example, the operator adjusts the angle of the scan plane of the transesophageal probe while observing the 2D ultrasound image. Alternatively, the operator switches from 2D mode, which generates 2D ultrasound images, to 3D mode, which performs a 3D scan, and observes the 3D ultrasound image generated by the 3D scan. The operator confirms the overall image of the area including the tip of the medical instrument by observing the 3D ultrasound image, then switches back to 2D mode and adjusts the angle of the scan plane.

[0004] Although the above-described ultrasound diagnostic device presents no particular problems, the inventors' investigations have revealed that there is room for improvement in that the operation of searching for the tip of a medical instrument can be burdensome for some operators. Therefore, a technology to assist the operator in searching for a medical instrument on an ultrasound image is desired. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-153953 Summary of the Invention [Problem to be solved by the invention]

[0006] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to assist an operator in searching for a medical instrument on an ultrasound image. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0007] An ultrasound diagnostic apparatus according to an embodiment includes a first generator, a second generator, and an estimation unit. The first generator generates a first ultrasound image of a subject by performing a first scan on the subject using an ultrasound probe. The second generator generates a second ultrasound image including a medical instrument by performing a second scan that includes the first scan. The estimation unit estimates first position information of the ultrasound probe that can generate the first ultrasound image including the medical instrument based on the second ultrasound image. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an ultrasound diagnostic apparatus according to the first embodiment. [Figure 2]FIG. 2 is a diagram for explaining the position information of the ultrasound probe according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating a trained model according to the first embodiment. [Figure 4] FIG. 4 is a time chart for explaining an example of the operation of the ultrasonic diagnostic apparatus of the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of a two-dimensional ultrasound image generated by the ultrasound diagnostic apparatus according to the first embodiment. [Figure 6] FIG. 6 is a flowchart for explaining an example of the operation of the ultrasound diagnostic apparatus of the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a three-dimensional ultrasound image generated by the ultrasound diagnostic apparatus according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a two-dimensional ultrasound image generated by the ultrasound diagnostic apparatus according to the first embodiment. [Figure 9] FIG. 9 is a flowchart for explaining an example of the operation of the ultrasonic diagnostic apparatus according to the modified example of the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of the configuration of an ultrasonic diagnostic apparatus according to the second embodiment. [Figure 11] FIG. 11 is a flowchart for explaining an example of the operation of the ultrasound diagnostic apparatus of the second embodiment. [Figure 12] FIG. 12 is a diagram showing an example of the configuration of an ultrasonic diagnostic apparatus according to the third embodiment. [Figure 13] FIG. 13 is a flowchart for explaining an example of the operation of the ultrasound diagnostic apparatus of the third embodiment. [Figure 14] FIG. 14 is a flowchart for explaining an example of the operation of the ultrasonic diagnostic apparatus according to the modified example of the third embodiment. [Figure 15] FIG. 15 is a diagram showing an example of the configuration of an ultrasonic diagnostic apparatus according to the fourth embodiment. [Figure 16]FIG. 16 is a flowchart for explaining an example of the operation of the ultrasound diagnostic apparatus of the fourth embodiment. [Figure 17] FIG. 17 is a flowchart for explaining an example of the operation of the ultrasonic diagnostic apparatus according to the modified example of the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of an ultrasound diagnostic apparatus will be described with reference to the drawings. In the following description, the same reference numerals are used to designate substantially the same parts in different drawings, and redundant description will be omitted.

[0010] (First embodiment) Fig. 1 is a diagram showing an example of the configuration of an ultrasonic diagnostic apparatus 100 according to the first embodiment. As shown in Fig. 1, the ultrasonic diagnostic apparatus 100 has an ultrasonic probe 1 and an apparatus main body 2. The ultrasonic probe 1 is detachably connected to the apparatus main body 2.

[0011] The ultrasound diagnostic device 100 images, for example, the tip of a medical device D used in catheter intervention for structural heart disease, as well as a heart valve other than the tip. An example of the medical device D is a clip (e.g., MitraClip (registered trademark)) placed on the mitral valve in percutaneous mitral valve clipping, a type of catheter intervention. Another example of the medical device D is an artificial valve that replaces the aortic valve in transcatheter aortic valve implantation (TAVI), another type of catheter intervention. Another example of the medical device D is a catheter or guidewire used in catheter intervention. During catheter intervention, the ultrasound diagnostic device 100 generates and displays an ultrasound image of the heart of a subject into which the medical device D has been inserted, including the tip of the medical device D, based on the output of the ultrasound probe 1. The clip or artificial valve is an example of a first medical device D or a second medical device D. The clip or artificial valve may also be referred to as a device. Furthermore, a catheter or a guidewire may also be an example of the first or second medical device D.

[0012] The ultrasonic probe 1 is, for example, a transesophageal probe that is inserted into the esophagus of a subject and receives reflected wave signals related to the medical instrument D from the esophagus. The ultrasonic probe 1 includes a tip portion, a bending portion, a guide tube portion, a handle portion, a cable portion, and a connector portion, all of which are not shown. The tip portion includes, for example, a plurality of piezoelectric vibrators arranged one-dimensionally, a matching layer provided on the piezoelectric vibrators, and a backing material that prevents ultrasonic waves from propagating backward from the piezoelectric vibrators. The plurality of piezoelectric vibrators generate ultrasonic waves based on a drive signal supplied via the cable portion from a transmission circuit 21 included in the device main body 2. The plurality of piezoelectric vibrators are configured to be rotatable, for example, by receiving power from a motor (not shown) housed in the ultrasonic probe 1. The bending portion is connected to the tip portion and is flexible. The guide tube portion is connected to the bending portion and is inserted into the body cavity of the subject when capturing an ultrasound image of the subject. The handle portion is connected to the guide tube portion and is held by the operator. The handle portion includes an input interface 11. The cable section transmits signals exchanged between the device main body 2 and the tip and handle sections. The connector section electrically connects the ultrasonic probe 1 to the device main body 2. Note that the ultrasonic probe 1 is not limited to a probe in which a plurality of piezoelectric vibrators are arranged one-dimensionally, and the ultrasonic probe 1 may be, for example, a 2D array probe in which a plurality of piezoelectric vibrators are arranged in a two-dimensional matrix. Furthermore, the ultrasonic probe 1 may be, for example, a mechanical 4D probe capable of performing ultrasonic scanning by mechanically moving the piezoelectric vibrator array in a direction perpendicular to the arrangement direction.

[0013] When ultrasonic waves are transmitted from the ultrasonic probe 1 to the subject, they are reflected successively by discontinuous surfaces of acoustic impedance in the subject's internal tissues and received as reflected wave signals (echo signals) by multiple piezoelectric transducers. The amplitude of the received reflected wave signals depends on the difference in acoustic impedance at the discontinuous surfaces where the ultrasonic waves are reflected. When the transmitted ultrasonic pulses are reflected by the surface of a moving blood flow or heart wall, the reflected wave signals undergo a frequency shift due to the Doppler effect, depending on the velocity component of the moving object in the direction of ultrasonic transmission. The ultrasonic probe 1 receives the reflected wave signals from the subject.

[0014] The ultrasonic probe 1 receives two-dimensional reflected wave signals by executing a 2D mode in which a non-3D scan is performed on the medical instrument D, forming at least one scan plane. Specifically, for example, during 2D mode, the ultrasonic probe 1 performs a scan in which two mutually orthogonal scan planes (bi-plane, x-plane) are simultaneously or repeatedly formed, in terms of frame rate and resolution. Among 2D modes, a mode in which the device main body 2 generates two orthogonal cross-sectional images (two orthogonal cross-sectional images) by performing the scan may be referred to as a bi-plane mode. However, this is not limited thereto, and the ultrasonic probe 1 may also perform a scan in which three or more scan planes are simultaneously or repeatedly formed. Furthermore, for example, during 2D mode, the ultrasonic probe 1 may perform a 2D scan in which multiple scan planes are repeatedly formed. In this case, the device main body 2 generates one 2D cross-sectional image. During 2D scanning, the ultrasonic probe 1 mechanically rotates multiple piezoelectric transducers to change the angle of the scan plane of the ultrasonic probe 1. The ultrasonic probe 1 receives two-dimensional reflected wave signals according to the angle of the scan plane. Furthermore, the ultrasonic probe 1 receives 3D reflected wave signals by executing a 3D mode that performs a 3D scan on the medical instrument D. Specifically, the ultrasonic probe 1 receives 3D reflected wave signals by mechanically rotating multiple piezoelectric vibrators during 3D scanning. Note that 4D scanning using a mechanical 4D probe or the like is not included in non-3D scanning. The ultrasonic probe 1 is configured to be able to switch between 2D mode and 3D mode as appropriate. Note that 3D scanning is an example of a first scan. Non-3D scanning is an example of a second scan.

[0015] The input interface 11 is a variety of user interfaces arranged on the handle portion. An example of the input interface 11 is a button that is pressed to perform offset processing, freeze an ultrasound image, etc. Another example of the input interface 11 is a mode switching button that inputs an instruction to switch from one of 2D mode and 3D mode to the other mode.

[0016] An example of the input interface 11 is an angle adjustment button that is pressed when adjusting the angle of the scan plane. For example, while the 2D mode is being executed, the operator presses the angle adjustment button and adjusts the angle of the scan plane in order to observe the tip of the medical instrument D or an imaging target other than the tip, such as a heart valve, from various angles.

[0017] An example of the input interface 11 is an execution instruction input button for inputting an execution instruction to execute a 3D scan in the background while a 2D ultrasound image is being displayed. The input interface 11 transmits the input execution instruction to the processing circuitry 25. The execution instruction input button is an optional additional item and may be omitted.

[0018] Here, with reference to FIG. 2, the position of the ultrasonic probe 1 in three-dimensional space, the orientation of the ultrasonic probe 1 at that position, and the angle of the scan plane of the ultrasonic probe 1 will be described. As shown in FIG. 2, the ultrasonic probe 1 generates ultrasonic waves from a transducer array surface P11 on which a plurality of piezoelectric transducers are one-dimensionally arranged. The position of the ultrasonic probe 1 in a three-dimensional Cartesian coordinate system (x, y, z) with a predetermined position as the origin O is assumed to be the center C (cx, cy, cz) of the transducer array surface P11. The orientation of the ultrasonic probe 1 is defined here by a normal vector r, which is a unit vector that is perpendicular to the transducer array surface P11 and indicates the direction of the ultrasonic wave transmission side. Specifically, the orientation of the ultrasonic waves is expressed by the components (rx, ry, rz) of the normal vector r. Note that the three-dimensional Cartesian coordinate system is assumed to be, for example, a right-handed coordinate system in which the positive direction of the Z axis is opposite to the direction of gravity. The X axis is assumed to be parallel to the longitudinal direction of a bed (not shown) on which the subject is placed. The Y axis is assumed to be parallel to the shorter direction of the bed. Furthermore, as shown in FIG. 2, the transducer array plane Pl1 rotates around a rotation axis Pz, which is a line that passes through the center C and is perpendicular to the transducer array plane Pl1. In FIG. 2, plane Pl2 represents a plane corresponding to the scan plane in 2D mode when the angle of the scan plane is an initial angle that has not yet been changed. In FIG. 2, plane Pl3 represents a scan plane in 2D mode when the angle of the scan plane relative to the initial angle is θ. As shown in FIG. 2, this scan plane includes the medical device D. Hereinafter, the scan plane including the medical device D will be referred to as the reference cross section. Furthermore, a set of the position of the ultrasound probe 1, the attitude angle, and the angle of the scan plane (cx, cy, cz, rx, ry, rz, θ) will be referred to as position information. The term "position information" may be appropriately rephrased as "position parameters."

[0019] The device body 2 generates an ultrasound image based on the reflected wave signals received by the ultrasound probe 1. The device body 2 generates a two-dimensional ultrasound image based mainly on the two-dimensional reflected wave signals received while executing the 2D mode. The device body 2 generates two orthogonal cross-sectional images, for example, by executing the biplane mode. The device body 2 may also generate one 2D cross-sectional image by executing a 2D scan. The device body 2 also generates a three-dimensional ultrasound image based on the three-dimensional reflected wave signals received while executing a 3D scan. The device body 2 is a computer having a transmission circuitry 21, a reception circuitry 22, a transmission / reception control circuitry 23, a generation circuitry 24, a processing circuitry 25, a memory 26, an input interface 27, a communication interface 28, and a display 29.

[0020] The transmission circuit 21 transmits ultrasonic beams via the ultrasonic probe 1 under the control of the transmission / reception control circuit 23. Specifically, the transmission circuit 21 imparts a delay time to each drive signal to impart transmission directivity and supplies the signal to each transducer in order to transmit ultrasonic beams deflected at a given transmission beam angle. The transmission circuit 21 repeatedly transmits ultrasonic waves while changing the transmission beam angle.

[0021] The receiving circuit 22 performs various processes on the reflected wave signals received by the ultrasound probe 1 under the control of the transmission / reception control circuit 23 to generate received signals. Specifically, the receiving circuit 22 is realized by, for example, a preamplifier (preamplifier group), an A / D converter, a demodulator, and a beamformer (reception delay and addition circuit). The preamplifier amplifies the reflected wave signals for each channel and performs gain correction processing. The A / D converter converts the gain-corrected reflected wave signals into digital signals. The demodulator demodulates the digital signals. The beamformer, for example, applies a delay time required to determine the reception directivity to the demodulated digital signals and adds together the multiple digital signals that have been applied with the delay time. The addition processing of the beamformer generates a received signal in which the reflection components from the direction corresponding to the reception directivity are emphasized.

[0022] The transmission and reception control circuit 23, under the control of the processing circuit 25, synchronously controls the transmission circuit 21 and the reception circuit 22 so as to perform a non-3D scan or a 3D scan on the imaging target via the ultrasound probe 1.

[0023] The generation circuit 24 performs B-mode processing, color Doppler processing, and the like on the received signal transmitted from the receiving circuit 22. In B-mode processing, the generation circuit 24 performs logarithmic amplification, envelope detection, logarithmic compression, and the like on the received signal to generate B-mode information in which the signal strength is expressed as brightness for each of a plurality of sample points. Based on the B-mode information, the generation circuit 24 then generates a two-dimensional or three-dimensional B-mode image in which the signal strength is expressed as a brightness value. In Doppler processing, the generation circuit 24 performs frequency analysis on the received signal to estimate Doppler information, such as the velocity, dispersion, and power of moving objects such as blood and tissue, for each of a plurality of sample points. The generation circuit 24 then generates a two-dimensional or three-dimensional Doppler image in which the velocity, dispersion, and power of moving objects such as blood and tissue are expressed as color values. When there is no need to distinguish between B-mode images and Doppler images, they are referred to as ultrasound images. The ultrasound image is stored in the memory 26. The generation circuit 24 can be implemented using any processor.

[0024] The generation circuit 24 generates a 2D ultrasound image of a subject into which a medical instrument D is inserted by performing a non-3D scan using the ultrasound probe 1. The generation circuit generates, for example, a 2D ultrasound image such as a mid-esophageal five-chamber image or a mid-esophageal two-chamber image by performing the non-3D scan. When the position information of the ultrasound probe 1 is position information that allows a 2D ultrasound image including the medical instrument D to be generated, the generation circuit 24 generates a 2D ultrasound image including the medical instrument D. Hereinafter, this position information will be referred to as first position information. Note that the generation circuit 24 generates a 2D ultrasound image including the medical instrument D by performing a non-3D scan using the ultrasound probe 1 whose position information has been updated to the first position information. Specifically, the generation circuit 24 generates a 2D ultrasound image including the medical instrument D when the scan plane of the non-3D scan corresponds to the reference cross section. Note that the generation circuit 24 generates a 2D ultrasound image including the medical instrument D when the position information of the ultrasound probe 1 is position information different from the first position information. Hereinafter, this other position information will be referred to as second position information. Specifically, the generation circuitry 24 generates a two-dimensional ultrasound image that does not include the medical device D when the scan plane of the non-3D scan does not correspond to the reference cross section.

[0025] Furthermore, the generation circuitry 24 generates a 3D ultrasound image including the medical instrument D by performing a 3D scan including a non-3D scan in the background while the 2D ultrasound image is displayed. The generation circuitry 24 generates ultrasound images frame by frame or volume by volume, for example, according to a preset frame rate of the non-3D scan or a volume rate of the 3D scan. The generation circuitry 24 mainly generates a 2D ultrasound image by performing a non-3D scan. The generation circuitry 24 also generates a 3D ultrasound image by performing a 3D scan based on an execution instruction. The 3D scan is performed for, for example, only one volume. Therefore, the generation circuitry 24 generates one 3D ultrasound image including the medical instrument D. Note that the generation circuitry 24 may generate two or more 3D ultrasound images by performing a 3D scan for two or more volumes. After generating a 3D ultrasound image, the generation circuitry 24 again generates a 2D ultrasound image by performing a non-3D scan. Note that "performing a 3D scan in the background" means that, while a non-3D scan is being performed, a 3D scan is performed for a short period of time (for one volume) by switching from the non-3D scan, and then a non-3D scan is performed again. Furthermore, the generation circuit 24 is an example of a first generation unit or a second generation unit. Furthermore, a two-dimensional ultrasound image is an example of a first ultrasound image. Furthermore, a three-dimensional ultrasound image is an example of a second ultrasound image.

[0026] The processing circuitry 25 includes a processor such as a CPU (Central Processing Unit) that controls the ultrasound diagnostic apparatus 100. The processing circuitry 25 executes an ultrasound diagnostic program stored in the memory 26 to realize functions corresponding to the program. The processing circuitry 25 realizes, for example, a scan control function 251, an estimation function 252, an update function 253, and a display control function 254. The functions 251 to 254 do not necessarily have to be realized by a single processing circuit 25. The processing circuitry 25 may be configured by combining multiple independent processors, and each processor may execute a control program that is a division of the ultrasound diagnostic program to realize the functions 251 to 254. Furthermore, the functions 251 to 254 may be implemented as modules that constitute the control program, or may be implemented as individual hardware. The same applies to the functions of the processing circuitry 25 described in the second embodiment and subsequent embodiments.

[0027] By implementing the scan control function 251, the processing circuitry 25 performs a non-3D scan using the ultrasound probe 1 on an imaging target within the subject. Furthermore, the processing circuitry 25 performs a 3D scan, including a non-3D scan, in the background while a 2D ultrasound image is being displayed. Specifically, the processing circuitry 25 controls the transmission / reception control circuitry 23 to perform a non-3D scan or a 3D scan. The processing circuitry 25 performs a 3D scan for one volume based on an execution instruction received from the input interface 11. After performing the 3D scan, the processing circuitry 25 performs a non-3D scan using the ultrasound probe 1. The scan control function 251 is an example of a first or second generation unit. Furthermore, in the description of this embodiment, the second generation unit is configured to perform a 3D scan in the background while a 2D ultrasound image is being displayed, thereby generating a 3D ultrasound image including a medical instrument D. However, this is not limiting. The second generation unit may perform a 3D scan to generate the 3D ultrasound image even when a non-3D scan is not being performed and a 2D ultrasound image is not being displayed. In other words, the second generator does not necessarily have to perform the 3D scan in the background.

[0028] By implementing the estimation function 252, the processing circuitry 25 estimates the first position information based on the three-dimensional ultrasound image. For example, the processing circuitry 25 estimates the first position information based on the three-dimensional ultrasound image generated when the position information of the ultrasound probe 1 is the second position information. Specifically, the processing circuitry 25 estimates, as the first position information, the position, posture, and angle of the scan plane of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the medical instrument D. For example, the processing circuitry 25 estimates the position and posture included in the second position information as the position and posture included in the first position information. On the other hand, for example, the processing circuitry 25 estimates, as the angle of the scan plane included in the first position information, an angle different from the angle of the scan plane included in the second position information. Specifically, for example, the second position information is assumed to be (cx, cy, cz, rx, ry, rz, 160°). At this time, the processing circuitry 25 estimates, for example, (cx, cy, cz, rx, ry, rz, 90°) as the first position information. In the above example, the position and orientation (cx~rz) included in the second position information are the same as the position and orientation (cx~rz) included in the first position information, and the angle of the scan plane (160°) included in the second position information is different from the angle of the scan plane (90°) included in the first position information. In addition, the processing circuitry 25 may estimate the first position information by inputting the generated three-dimensional ultrasound image into a trained model Md10, which will be described later. Note that the estimation function 252 is an example of an estimation unit.

[0029] By implementing the update function 253, the processing circuit 25 updates the position information of the ultrasound probe 1 from the second position information to the first position information. Specifically, the processing circuit 25 updates the position information of the ultrasound probe 1 by changing the angle of the scan plane from the angle of the scan plane included in the second position information to the angle of the scan plane included in the first position information. The processing circuit 25 controls, for example, a motor. The motor generates power for rotating multiple piezoelectric vibrators under control of the processing circuit 25. The multiple piezoelectric vibrators receive power from the motor and rotate around a rotation axis. The rotation of the multiple piezoelectric vibrators changes the angle of the scan plane. In the above example, the processing circuit 25 rotates the multiple piezoelectric vibrators so that the angle of the scan plane included in the second position information (160°) becomes the angle of the scan plane included in the first position information (90°). Note that the update function 253 is an optional additional feature and may be omitted. In other words, the processing circuit 25 may or may not automatically update the position information after executing the process of estimating the first position information. The update function 253 is an example of an update unit.

[0030] By implementing the display control function 254, the processing circuitry 25 causes the display 29 to display a 2D or 3D ultrasound image. The processing circuitry 25 causes the display 29 to display a 2D ultrasound image including the medical instrument D, which was generated by performing a non-3D scan using the ultrasound probe 1 whose position information has been updated to the first position information. Specifically, the processing circuitry 25 causes the display 29 to display a 2D ultrasound image including the medical instrument D that was automatically generated after the position information update process. The processing circuitry 25 causes the display 29 to display a 3D ultrasound image only when, for example, a display instruction is received from the operator. Note that the processing circuitry 25 may be configured to display a 3D ultrasound image on the display 29 even if a display instruction is not received from the operator. Furthermore, displaying a 2D ultrasound image that is automatically generated after the position information update is an optional additional feature and may be omitted. In other words, after the position information is updated, the processing circuitry 25 may or may not automatically display the 2D ultrasound image. The display control function 254 is an example of a display control unit.

[0031] The memory 26 is a memory 26 such as a ROM, RAM, HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit memory that stores various types of information. The memory 26 may also be a drive or the like that reads and writes various types of information from and to a portable storage medium such as a CD-ROM drive, DVD drive, or flash memory. For example, the memory 26 stores an ultrasound diagnostic program for the ultrasound diagnostic device 100. This program may be pre-stored in the memory 26. For example, the program may be stored in a non-transitory computer-readable storage medium and distributed, and then read from the non-transitory computer-readable storage medium and installed in the memory 26. The memory 26 may also store, for example, one trained model Md10. The trained model Md10 may be pre-stored in the memory 26 when the ultrasound diagnostic device 100 is delivered. Alternatively, the memory 26 may store the trained model Md10 acquired from a server device (not shown) after the ultrasound diagnostic device 100 is delivered.

[0032] The trained model Md10 is a trained machine learning model Md1 obtained by having the machine learning model Md1 perform machine learning based on training data in accordance with a model training program. The machine learning model Md1 is, for example, a DNN (Deep Neural Network) using a CNN (Convolutional Neural Network). Here, an example of training of the machine learning model Md1 of this embodiment is shown in FIG. 3(a). Of the training dataset of the machine learning model Md1, input data is a previously acquired 3D ultrasound image during training that includes a medical instrument D. Specifically, the input data is a previously acquired 3D ultrasound image during training that includes a clip. Furthermore, of the training dataset of the machine learning model Md1, output data is first position information of the ultrasound probe 1 that can generate a training 2D ultrasound image that includes the medical instrument D. Specifically, the output data is first position information of the ultrasound probe 1 that can generate a training 2D ultrasound image that includes the clip. Note that, without being limited to this, the input data may also be a previously acquired 3D ultrasound image during training that includes another medical instrument D, such as an artificial valve, a catheter, or a guidewire. Furthermore, the output data may be first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image during learning that includes the other medical instrument D. Furthermore, during learning, the machine learning model Md1 is caused to perform machine learning on one type of medical instrument. However, this is not limiting, and during learning, the machine learning model Md1 may be caused to perform machine learning on multiple types of medical instruments.

[0033] The trained model Md10 is the trained machine learning model Md1 implemented as shown in FIG. 3(b). When a 3D ultrasound image including a medical instrument D generated by a 3D scan performed during an examination is input, the trained model Md10 outputs first position information of the ultrasound probe 1 capable of generating a 2D ultrasound image including the medical instrument D. Specifically, for example, when a 3D ultrasound image including a clip generated by a 3D scan performed during an examination is input, the trained model Md10 outputs first position information of the ultrasound probe 1 capable of generating a 2D ultrasound image including the clip. When a 3D medical image including another medical instrument D such as an artificial valve, a catheter, or a guidewire is input, the trained model Md10 outputs first position information of the ultrasound probe 1 capable of generating a 2D ultrasound image including the medical instrument D. The trained model Md10 may output only the angle of the scan plane of the ultrasound probe 1 capable of generating a 2D ultrasound image including the medical instrument D.

[0034] The input interface 27 of the device body 2 is, for example, various user interfaces on a touch panel or an operation panel. The operator can input various operations and instructions to the ultrasound diagnostic device 100 through the input interface 27. The various buttons described above may be provided on the input interface 27 of the device body 2.

[0035] The communication interface 28 communicates data between a PACS (Picture Archiving and Communication System) server, an HIS (Hospital Information System) server, an MWM (Modality Worklist Management) server, and the like via a LAN (Local Area Network) or the like.

[0036] The display 29 displays various information in accordance with instructions from the processing circuit 25. The display 29 may be, for example, a CRT display, a liquid crystal display (LCD), a cathode ray tube (CRT) display, an organic electroluminescence display (OLED), a plasma display, or any other display. A projector may also be provided as the display 29. The display 29 displays, for example, a two-dimensional ultrasound image generated by performing a non-3D scan. The display 29 also displays a two-dimensional ultrasound image including the medical instrument D, generated by performing a non-3D scan with the ultrasound probe 1 whose position information has been updated to the first position information.

[0037] Next, an example of the operation of the ultrasound diagnostic apparatus 100 configured as described above will be described with reference to Figures 4 to 8. Below, the operation of the ultrasound diagnostic apparatus 100 during the procedure of percutaneous mitral valve clipping, which is one of catheter interventions, will be described.

[0038] First, during the procedure, the ultrasound diagnostic device 100 executes a 2D mode scan of the subject's heart, into which a clip has been inserted, to generate and display a 2D ultrasound image including the tip of the clip. At this time, as shown on the left side of FIG. 4, the frame rate fr of the non-3D scan is set to, for example, 50 Hz. In this case, the scan time per frame is 20 milliseconds. For example, if the operator loses sight of the tip of the clip or wishes to observe a heart valve other than the tip, the ultrasound diagnostic device 100 displays a 2D ultrasound image U1 (a midesophageal five-chamber view) that does not include the tip on the display 29, as shown in FIG. 5. In FIG. 5, the clip is not included in the 2D ultrasound image U1. At this time, the position information of the ultrasound probe 1 is assumed to be the second position information (cx, cy, cz, rx, ry, rz, 160°). After the operator confirms that the medical instrument D is not included in the 2D ultrasound image U1, step ST10 is initiated as shown in FIG. 6.

[0039] In step ST10, an instruction to perform a 3D scan is input to the input interface 11 provided on the ultrasound probe 1 by an input operation by the operator who has confirmed that the 2D ultrasound image U1 does not include the medical instrument D while the 2D ultrasound image U1 is being displayed. The input interface 11 transmits the input instruction to the processing circuitry 25.

[0040] After step ST10, in step ST20, the processing circuitry 25 performs a 3D scan for one volume based on the execution instruction received from the input interface 11. At this time, the volume rate Vr is set to 3 Hz, for example, as shown in the center of FIG. 4. In this case, the scan time per volume data is approximately 333 milliseconds. By performing a 3D scan for one volume, the generation circuitry 24 generates one 3D ultrasound image U2 including a clip, as shown in FIG. 7. After performing the 3D scan, the processing circuitry 25 performs a non-3D scan again. Note that the 3D ultrasound image U2 includes a clip, as shown in region R1 in FIG. 7.

[0041] After step ST20, in step ST30, the processing circuitry 25 inputs one three-dimensional ultrasound image U2 including the clip to the trained model Md10, thereby estimating first position information (90°) that is position information of the ultrasound probe 1 that can generate a two-dimensional ultrasound image including the clip.

[0042] After step ST30, in step ST40, the processing circuit 25 changes the angle of the scan plane by rotating the multiple piezoelectric vibrators so that the angle of the scan plane included in the second position information (160°) becomes the angle of the scan plane included in the first position information (90°).

[0043] After step ST40, in step ST50, the generation circuitry 24 performs a non-3D scan to generate a two-dimensional ultrasound image U3 (mid-esophageal two-chamber image) when the scan plane angle is 90°, as shown in FIG. 8. The processing circuitry 25 displays the generated two-dimensional ultrasound image U3 on the display 29. Note that the two-dimensional ultrasound image U3 includes a clip, as shown in region R2 in FIG. 8. After step ST50, the processing ends.

[0044] Although the above description has been given using an example of the operation during a percutaneous mitral valve clip procedure, the present invention is not limited to this. For example, the ultrasound diagnostic device 100 may be operated during a transcatheter aortic valve implantation procedure. In describing this operation, the term "clip" will be read as "artificial valve" in the following description. Furthermore, during catheter intervention, for example, an operator may lose sight of the tip of a catheter due to the catheter tip being moved while observing the tip. In describing the operation of the ultrasound diagnostic device 100 in this case, the term "clip" will be read as "catheter" in the following description.

[0045] As described above, according to the first embodiment, the ultrasound diagnostic apparatus 100 includes a generation circuit 24 (first generation unit or second generation unit) and a processing circuit 25 that realizes a scan control function 251 (first generation unit or second generation unit) and an estimation function 252 (estimation unit). The processing circuit 25 performs a non-3D scan (first scan) using the ultrasound probe 1 on a subject into which a medical instrument D is inserted. The generation circuit 24 generates a two-dimensional ultrasound image (first ultrasound image) of the subject by performing the non-3D scan. The processing circuit 25 performs a 3D scan (second scan) that includes the non-3D scan. As a result, the generation circuit 24 generates a three-dimensional ultrasound image (second ultrasound image) including the medical instrument D. The processing circuit 25 estimates first position information of the ultrasound probe 1 that can generate a two-dimensional ultrasound image including the medical instrument D, based on the three-dimensional ultrasound image. In this way, the ultrasound diagnostic device 100 is configured to generate a three-dimensional ultrasound image by performing a 3D scan and estimate the first position information based on the three-dimensional ultrasound image. Therefore, the ultrasound diagnostic device 100 performs a series of processes from performing the second scan to estimating the first position information, thereby assisting the operator in searching for the medical instrument D on the ultrasound image.

[0046] Additionally, the ultrasound diagnostic device 100 can assist the operator in searching for a medical device D, such as a clip placed on the mitral valve or an artificial valve that has replaced the aortic valve, on an ultrasound image. This can assist catheter interventions, such as percutaneous mitral valve clipping and transcatheter aortic valve implantation, for structural heart disease.

[0047] An ultrasound diagnostic apparatus according to a comparative example will now be described. The ultrasound diagnostic apparatus according to the comparative example does not have a configuration for executing the above series of processes. Therefore, when the operator re-observes the distal end, the ultrasound diagnostic apparatus according to the comparative example receives an instruction to switch from 2D mode to 3D mode so as to grasp the overall image of the area including the medical instrument D. When the instruction to switch is received, the ultrasound diagnostic apparatus according to the comparative example switches from 2D mode to 3D mode and displays a 3D ultrasound image on the display. Normally, this process does not pose any particular problems, but when a 3D ultrasound image is displayed on the display, the amount of processing by the ultrasound diagnostic apparatus increases compared to when the 2D mode is executed, which takes time and places a load on the processor.

[0048] On the other hand, the ultrasound diagnostic device 100 according to the first embodiment performs a 3D scan for only one volume during the execution of the above series of processes, and omits the process of displaying a 3D ultrasound image on the display 29. This reduces the amount of processing by the ultrasound diagnostic device 100, thereby shortening the processing time and reducing the load on the processor.

[0049] Furthermore, according to the first embodiment, the ultrasound diagnostic apparatus 100 further includes an execution instruction input button (first input unit) into which an instruction to execute a 3D scan is input. The processing circuitry 25 executes a 3D scan based on the execution instruction. Therefore, in addition to the effects described above, the operator can execute the above series of processes by simply pressing the button, and grasp the position information of the medical instrument D on the ultrasound image, thereby further assisting the operator in searching for the medical instrument D.

[0050] Furthermore, according to the first embodiment, the first scan is a non-3D scan that forms at least one scan plane. Therefore, in addition to the effects described above, the ultrasound diagnostic device 100 can perform a scan that forms two scan planes that are orthogonal to each other, a 2D scan that forms one scan plane, etc., to generate two-dimensional ultrasound images, such as orthogonal two-sectional images or 2D sectional images.

[0051] Furthermore, according to the first embodiment, the second scan is a 3D scan. Therefore, in addition to the above-mentioned effects, the ultrasound diagnostic apparatus 100 can perform a 3D scan to generate a three-dimensional ultrasound image.

[0052] Furthermore, according to the first embodiment, the processing circuitry 25 further includes an update function 253 that updates the position information of the ultrasonic probe 1 from the second position information to the first position information. Therefore, in addition to the above-described effects, the position information of the ultrasonic probe 1 can be automatically updated to position information of the ultrasonic probe 1 that can generate a two-dimensional ultrasonic image including the medical instrument D.

[0053] Furthermore, according to the first embodiment, the first position information includes the angle of the scan plane of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image. The processing circuitry 25 updates the position information of the ultrasound probe 1 by changing the angle of the scan plane from the angle of the scan plane included in the second position information to the angle of the scan plane included in the first position information. Therefore, in addition to the effects described above, by changing the angle of the scan plane, the position information of the ultrasound probe 1 can be updated to position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the medical instrument D.

[0054] Furthermore, according to the first embodiment, the generation circuitry 24 generates a two-dimensional ultrasound image including the medical instrument D by performing a non-3D scan using the ultrasound probe 1 whose position information has been updated to the first position information. Therefore, in addition to the above-mentioned effects, a two-dimensional ultrasound image including the medical instrument D can be automatically generated.

[0055] Furthermore, according to the first embodiment, the processing circuitry 25 further has a display control function 254 (display control unit) that causes the display 29 to display a two-dimensional ultrasound image including the medical instrument D, which is generated after the position information is updated. Therefore, in addition to the above-described effects, the two-dimensional ultrasound image including the medical instrument D can be automatically displayed on the display 29.

[0056] Furthermore, according to the first embodiment, the processing circuitry 25 inputs a three-dimensional ultrasound image including the medical instrument D, generated by performing a 3D scan during an examination, to a trained model Md10 that outputs first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the medical instrument D during training based on a three-dimensional ultrasound image including the medical instrument D. The processing circuitry 25 estimates first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the medical instrument D based on the output of the trained model Md10. Therefore, in addition to the above-mentioned effects, the first position information can be estimated using the trained model Md10.

[0057] Furthermore, according to the first embodiment, the medical instrument D includes a first medical instrument or a second medical instrument. When a second ultrasound image including the first medical instrument is input to the trained model Md10, the processing circuitry 25 estimates first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the first medical instrument. When a second ultrasound image including the second medical instrument is input to the trained model Md10, the processing circuitry 25 estimates first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the second medical instrument. In this way, regardless of whether the medical instrument individually included in the three-dimensional ultrasound image input to one trained model Md10 is the first medical instrument or the second medical instrument, the configuration is such that first position information corresponding to the medical instrument can be estimated. Therefore, with this configuration, in addition to the above-mentioned effects, there is no need to store multiple trained models, thereby reducing memory usage.

[0058] (Modification of the first embodiment) The first embodiment may be modified as follows: The modifications may be combined with each other, or may be combined with the following embodiments.

[0059] According to the first embodiment, the processing circuitry 25 is configured to display on the display 29 a two-dimensional ultrasound image including the medical instrument D, which is generated by performing a non-3D scan with the ultrasound probe 1 whose position information has been updated to the first position information. However, this is not limiting. For example, the processing circuitry 25 may be configured to display on the display 29 display screen data in which the two-dimensional ultrasound image and a three-dimensional ultrasound image generated by performing a 3D scan are arranged side by side.

[0060] According to this modification, the ultrasound diagnostic apparatus 100 executes the processes of steps ST10 to ST40 in the same manner as described above, as shown in Fig. 9. After step ST40, the process proceeds to step ST51.

[0061] In step ST51, the ultrasound diagnostic apparatus 100 displays on the display 29 display screen data in which a two-dimensional ultrasound image including the medical instrument D and a three-dimensional ultrasound image are arranged side by side, the two-dimensional ultrasound image being generated by the ultrasound probe 1 whose position information has been updated to the first position information. After step ST51, the processing ends.

[0062] According to the configuration of this modification, the operator can view these two ultrasound images on one display screen. Therefore, in addition to the effects of the first embodiment, even if the medical instrument D is not included in the two-dimensional ultrasound image displayed after updating the position information, the operator can adjust the angle of the scan plane while viewing the three-dimensional ultrasound image.

[0063] (Second embodiment) The second embodiment is a modification of the first embodiment, and is configured to notify the operator of first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the medical instrument D.

[0064] 10 is a diagram showing an example of the configuration of an ultrasound diagnostic apparatus 100 according to the second embodiment. As shown in FIG.

[0065] By implementing the notification function 255, the processing circuitry 25 notifies the operator of the first position information. The ultrasound diagnostic apparatus 100 further includes, for example, a speaker (not shown). The processing circuitry 25 notifies the operator of the first position information by causing the speaker to output the first position information. Alternatively, the processing circuitry 25 may notify the operator of the first position information by, for example, displaying the first position information on the display 29. At this time, the operator adjusts the angle of the scan plane based on the notified first position information. The ultrasound diagnostic apparatus 100 may have any configuration as long as it is capable of notifying the operator of the first position information. The notification function 255 is an example of a first notification unit.

[0066] The other configurations are the same as those in the first embodiment.

[0067] According to the second embodiment, the ultrasound diagnostic apparatus 100 executes the processes of steps ST10 to ST30 in the same manner as described above, as shown in Fig. 11. After step ST30, the process proceeds to step ST31.

[0068] In step ST31, the processing circuit 25 notifies the operator of the first position information by causing the speaker to output the first position information. After step ST31, the processing ends.

[0069] As described above, according to the second embodiment, the processing circuitry 25 further includes a notification function 255 (first notification unit) that notifies the operator of the first position information. Therefore, in addition to the effects of the first embodiment, the first position information of the ultrasound probe 1 that can generate a two-dimensional ultrasound image including the medical instrument D can be made known to the operator. To add to that, the operator who has grasped the position information can adjust the angle of the scan plane by his / her own operation, without relying on the ultrasound diagnostic device 100 automatically updating the position information.

[0070] (Third embodiment) The third embodiment is a modified example of the first or second embodiment, and is configured to store multiple trained models Md10, determine the type of medical instrument D included in a three-dimensional ultrasound image, and select a trained model Md10 corresponding to the type of medical instrument D.

[0071] 12 is a diagram showing an example of the configuration of an ultrasound diagnostic apparatus 100 according to the third embodiment. As shown in FIG.

[0072] Unlike the configuration described above, the memory 26 may store multiple trained models Md10. The multiple trained models Md10 include, for example, a first trained model that outputs first position information of an ultrasound probe 1 capable of generating a 2D ultrasound image including a clip based on a 3D ultrasound image including the clip. The multiple trained models Md10 also include a second trained model that outputs first position information of an ultrasound probe 1 capable of generating a 2D ultrasound image including an artificial valve based on a 3D ultrasound image including the artificial valve. However, this is not limited to this. The memory 26 may also store a trained model that outputs first position information of an ultrasound probe 1 capable of generating a 2D ultrasound image including another medical device D based on a 3D ultrasound image including the other medical device D, such as a catheter or a guidewire. The memory 26 may store three or more trained models.

[0073] By implementing the determination function 256, the processing circuitry 25 determines, for example, based on a three-dimensional ultrasound image, whether the medical instrument D included in the three-dimensional ultrasound image is a clip or an artificial valve. The determination function 256 is an example of a determination unit.

[0074] By realizing the estimation function 252, in addition to the above-described configuration, if the result of the determination is that the medical instrument D included in the 3D ultrasound image is a paperclip, the processing circuitry 25 inputs the 3D ultrasound image into a first trained model. If the result of the determination is that the medical instrument D included in the 3D ultrasound image is an artificial valve, the processing circuitry 25 inputs the 3D ultrasound image into a second trained model.

[0075] The other configurations are the same as those in the first embodiment.

[0076] Next, an example of the operation of the ultrasound diagnostic apparatus 100 configured as described above will be described with reference to the flowchart in Figure 13. As described above, step ST10 starts when the operator confirms that the medical instrument D is not included in the two-dimensional ultrasound image during the percutaneous mitral valve clipping procedure. However, the memory 26 stores the first trained model and the second trained model.

[0077] The ultrasound diagnostic apparatus 100 executes the processes of steps ST10 and ST20 in the same manner as described above. After step ST20, the process proceeds to step ST25.

[0078] After step ST20, in step ST25, the processing circuit 25 determines, based on the 3D ultrasound image, whether the medical instrument D included in the 3D ultrasound image is a clip (first medical instrument) or an artificial valve (second medical instrument). In this example, since a percutaneous mitral valve clipping procedure is in progress, the medical instrument D included in the 3D ultrasound image is a clip. In this case, since the medical instrument D included in the 3D ultrasound image is the first medical instrument, the process proceeds to step ST30A. Note that, if a transcatheter aortic valve implantation procedure is in progress, the medical instrument D included in the 3D ultrasound image is an artificial valve. In this case, since the medical instrument D included in the 3D ultrasound image is not the first medical instrument, the process proceeds to step ST30B.

[0079] After step ST25, in step ST30A, the processing circuitry 25 inputs the three-dimensional ultrasound image into the first trained model, thereby estimating first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including a clip.

[0080] After step ST25, in step ST30B, ​​the processing circuitry 25 inputs the 3D ultrasound image into the second trained model, thereby estimating first position information of the ultrasound probe 1 capable of generating a 2D ultrasound image including the artificial valve.

[0081] After step ST30A or step ST30B, ​​the ultrasound diagnostic apparatus 100 executes the processes of steps ST40 and ST50 in the same manner as described above. After step ST50, the process ends.

[0082] As described above, according to the third embodiment, the medical instrument D includes a first medical instrument or a second medical instrument. The trained model Md10 includes a first trained model that outputs first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image (first ultrasound image) including the first medical instrument based on a three-dimensional ultrasound image (second ultrasound image) including the first medical instrument. The trained model Md10 includes a second trained model that outputs first position information of the ultrasound probe 1 capable of generating a two-dimensional ultrasound image including the second medical instrument based on a three-dimensional ultrasound image including the second medical instrument. In this manner, the medical instrument D included in the three-dimensional ultrasound image is identified, and the medical instrument D is input to the trained model Md10 corresponding to the identified medical instrument D to estimate the first position information. Therefore, with this configuration, the first position information can be estimated using the trained model Md10 specialized for one medical instrument D, thereby achieving the effect of the first embodiment as well as improved estimation accuracy of the first position information. Furthermore, since the multiple machine learning models Md1 can be trained in parallel, the time required for training can be reduced.

[0083] (Modification of the third embodiment) The third embodiment may be modified as follows: The modifications may be combined with each other, or may be combined with the following embodiments.

[0084] According to the third embodiment, the processing circuitry 25 determines the medical instrument D included in the 3D ultrasound image and inputs the medical instrument D into the trained model Md10 corresponding to the determined medical instrument D. However, this is not limiting. For example, in addition to the above-described configuration, the input interface 11 of the ultrasound probe 1 may input a first input instruction for inputting the 3D ultrasound image into a first trained model or a second input instruction for inputting the 3D ultrasound image into a second trained model. Furthermore, in addition to the above-described configuration, the processing circuitry 25 realizing the estimation function 252 may input the 3D ultrasound image into the first trained model based on the first input instruction and input the 3D ultrasound image into the second trained model based on the second input instruction. Specifically, for example, if the medical instrument D is a clip, the operator inputs a first input instruction into the input interface 11. On the other hand, if the medical instrument D is an artificial valve, the operator inputs a second input instruction into the input interface 11. However, the input interface 11 may be configured to receive an input instruction to input a 3D ultrasound image to a trained model other than the first and second trained models. In this case, the processing circuitry 25 inputs the 3D ultrasound image to the trained model based on the input instruction. The input interface 27 of the device main body 2 may also be configured to receive a first input instruction or a second input instruction. The input interface 11 is an example of a second input unit.

[0085] Next, an example of the operation of the ultrasound diagnostic device 100 configured as described above will be described with reference to the flowchart in Figure 14. During the percutaneous mitral valve clipping procedure, step ST1 is started in a state where the ultrasound diagnostic device 100 is displaying a two-dimensional ultrasound image that does not include the distal end portion on the display 29.

[0086] In step ST1, in this example, since the procedure is a percutaneous mitral valve clipping procedure, the operator inputs a first input instruction to the input interface 11. The input interface 11 transmits the input first input instruction to the processing circuitry 25. Note that, if the procedure is a transcatheter aortic valve implantation procedure, the operator inputs a second input instruction to the input interface 11. The operator may input the input instruction before or during the procedure.

[0087] After step ST1, the ultrasound diagnostic apparatus 100 executes the processes of steps ST10 and ST20 in the same manner as described above.

[0088] After step ST20, in this example, in step ST30C, the processing circuitry 25 inputs the 3D ultrasound image to the first trained model based on the first input instruction. As a result, the processing circuitry 25 estimates first position information of the ultrasound probe 1 capable of generating a 2D ultrasound image including the clip. Note that, when a transcatheter aortic valve implantation procedure is in progress, the processing circuitry 25 inputs the 3D ultrasound image to the second trained model based on the second input instruction. As a result, the processing circuitry 25 estimates first position information of the ultrasound probe 1 capable of generating a 2D ultrasound image including the artificial valve.

[0089] After step ST30C, the ultrasound diagnostic apparatus 100 executes the processes of steps ST40 and ST50 in the same manner as described above. After step ST50, the process ends.

[0090] According to this modification, it is possible to omit the process of determining the medical instrument D included in the three-dimensional ultrasound image. Therefore, in addition to the effect of the third embodiment, it is possible to reduce the amount of processing by the ultrasound diagnostic apparatus 100 from when an execution instruction is input until the first position information is estimated, thereby shortening the processing time and reducing the load on the processor.

[0091] (Fourth embodiment) The fourth embodiment is a specific example of the second or third embodiment, and further includes a configuration for detecting whether or not a medical instrument D is included in a two-dimensional ultrasound image, and notifying the user if the medical instrument D is not included in the two-dimensional ultrasound image.

[0092] 15 is a diagram showing an example of the configuration of an ultrasound diagnostic apparatus 100 according to the fourth embodiment. As shown in FIG.

[0093] The processing circuit 25 that realizes the detection function 257 detects whether or not a medical instrument D is included in the two-dimensional ultrasound image. As the detection result, for example, a status such as "medical instrument D is included" or "medical instrument D is not included" can be used as appropriate. The detection function 257 is an example of a detection unit.

[0094] By realizing the notification function 255, in addition to the above-described configuration, if the detection result shows that the medical instrument D is not included in the two-dimensional ultrasound image, the processing circuitry 25 notifies the operator that the medical instrument D is not included. The processing circuitry 25 may, for example, output the notification content as audio from a speaker or display it on the display 29. Note that the ultrasound diagnostic apparatus 100 may have any configuration as long as it is capable of notifying the operator of the notification content. The notification function 255 is an example of a second notification unit.

[0095] The other configurations are the same as those of the second or third embodiment.

[0096] Next, an example of the operation of the ultrasound diagnostic apparatus 100 configured as above will be described with reference to the flowchart of FIG.

[0097] First, during the percutaneous mitral valve clipping procedure, the ultrasound diagnostic apparatus 100 executes the 2D mode to generate and display a two-dimensional ultrasound image of the subject's heart into which a clip has been inserted. In this state, step ST5 is started.

[0098] In step ST5, the processing circuitry 25 detects whether the tip of the clip is included in the two-dimensional ultrasound image. If the processing circuitry 25 detects that the tip of the clip is included in the two-dimensional ultrasound image, the processing proceeds to step ST60. On the other hand, if the processing circuitry 25 detects that the tip of the clip is not included in the two-dimensional ultrasound image, the processing proceeds to step ST6.

[0099] After step ST5, in step ST6, the processing circuitry 25 notifies the operator by audio output from the speaker that the medical instrument D is not included in the two-dimensional ultrasound image.

[0100] After step ST6, the ultrasound diagnostic apparatus 100 executes the processes of steps ST10 to ST50 in the same manner as described above.

[0101] After step ST5 or step ST50, if ultrasonic imaging is to be continued rather than terminated in step ST60, the process proceeds to step ST5, and the processes of steps ST5 to ST50 are repeatedly executed. On the other hand, if ultrasonic imaging is to be terminated in step ST60, the ultrasonic diagnostic apparatus 100 terminates the process.

[0102] As described above, according to the fourth embodiment, the processing circuit 25 further has a detection function 257 (detection unit) that detects whether or not the two-dimensional ultrasound image (first ultrasound image) includes a medical instrument D. Therefore, in addition to the effects of the second or third embodiment, it is possible for the operator to determine whether or not the medical instrument D is included in the two-dimensional ultrasound image without visually recognizing the two-dimensional ultrasound image.

[0103] Furthermore, according to the fourth embodiment, the processing circuitry 25 further has a notification function 255 (second notification unit) that notifies the operator that the medical instrument D is not included in the two-dimensional ultrasound image when the detection result shows that the medical instrument D is not included in the two-dimensional ultrasound image. Therefore, in addition to the effects described above, the operator can be made to understand that the medical instrument D is not included in the two-dimensional ultrasound image even if the operator does not visually recognize the two-dimensional ultrasound image, and therefore the operator can be prompted to input an execution instruction to execute a 3D scan.

[0104] (Modification of the fourth embodiment) The first embodiment may be modified as follows: The modifications may be combined with each other, or may be combined with the following embodiments.

[0105] According to the fourth embodiment, the configuration detects whether or not the medical instrument D is included in the two-dimensional ultrasound image, and if the medical instrument D is not included in the two-dimensional ultrasound image, notifies the operator of the absence of the medical instrument D. However, this is not limiting. In addition to the configuration described above, the generation circuitry 24 (second generation unit) may be configured to perform a 3D scan (second scan) if the detection result shows that the medical instrument D is not included in the two-dimensional ultrasound image (first ultrasound image). Specifically, when the processing circuitry 25 detects that the medical instrument D is not included in the two-dimensional ultrasound image, the ultrasound diagnostic device 100 may automatically perform a series of processes from performing a 3D scan to estimating the first position information. Furthermore, after performing the series of processes, the ultrasound diagnostic device 100 may automatically update the position information of the ultrasound probe 1 and perform processes to generate and display a two-dimensional ultrasound image.

[0106] According to this modification, as shown in Fig. 17, the ultrasound diagnostic apparatus 100 executes the process of step ST5 in the same manner as described above. However, if the processing circuitry 25 detects that the tip of the clip is not included in the two-dimensional ultrasound image, the process proceeds to step ST20.

[0107] After step ST5, the ultrasound diagnostic apparatus 100 executes the processes of steps ST20 to ST60 in the same manner as described above. However, in step ST20, after detecting that the tip of the clip is not included in the two-dimensional ultrasound image, the processing circuitry 25 executes a 3D scan for only one volume even if it has not received an execution instruction from the input interface 11.

[0108] According to the configuration of this modification, after the processing circuitry 25 detects that the medical instrument D is not included in the two-dimensional ultrasound image, it automatically executes a series of processes from executing a 3D scan to estimating the first position information without any operation by the operator. Furthermore, after the series of processes are executed, it updates the position information of the ultrasound probe 1 and executes a process of generating and displaying a two-dimensional ultrasound image. Therefore, in addition to the effects described above, it is possible to further support the operator's operations when searching for the medical instrument D.

[0109] According to at least one of the embodiments described above, it is possible to assist the operator in searching for a medical instrument on an ultrasound image.

[0110] The term "processor" used in the above description refers to a circuit such as a CPU, a GPU, an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). A processor realizes its function by reading and executing a program stored in a memory circuit. Note that instead of storing a program in a memory circuit, the program may be directly embedded in the processor circuit. In this case, the processor realizes its function by reading and executing the program embedded in the circuit. On the other hand, if the processor is, for example, an ASIC, instead of storing a program in a memory circuit, the function is directly embedded in the processor circuit as a logic circuit. Note that each processor in this embodiment is not limited to being configured as a single circuit for each processor, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, multiple components in FIGS. 1, 10, 12 and 15 may be integrated into one processor to realize the functions thereof.

[0111] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0112] 1 ultrasound probe 2. Device body 11 Input Interface 21 Transmitting circuit 22 Receiving circuit 23 Transmit / receive control circuit 24 Generation circuit 25 Processing circuit 26 memory 27 Input Interface 28 Communication Interface 29 Display 100 Ultrasound diagnostic equipment 251 Scanning Control Function 252 Estimation Function 253 Update function 254 Display Control Function 255 Notification function 256 Judgment Function 257 detection function C center D Medical equipment fr frame rate Md1 machine learning model Md10 pre-trained model O Origin P Subject Pl1 transducer array surface Pl2, Pl3 plane Pz rotation axis R1, R2 area r normal vector U1, U3 2D ultrasound images U2 3D ultrasound image Vr Volume Rate θ Scan plane angle

Claims

1. a first generation unit that generates a first ultrasound image of a subject by performing a first scan with an ultrasound probe on the subject into which a medical instrument is inserted; a second generation unit that generates a second ultrasound image including the medical instrument by performing a second scan that includes the first scan; an estimation unit that estimates first position information of an ultrasound probe that can generate a first ultrasound image including the medical instrument based on the second ultrasound image; An ultrasound diagnostic device comprising:

2. a first input unit to which an instruction to execute the second scan is input, the second generation unit executes the second scan based on the execution instruction. The ultrasonic diagnostic apparatus according to claim 1 .

3. The ultrasound diagnostic apparatus of claim 1 , wherein the first scan is a non-3D scan that forms at least one scan plane.

4. The ultrasound diagnostic apparatus according to claim 1 , wherein the second scan is a 3D scan.

5. The ultrasound diagnostic apparatus according to claim 1 , further comprising an update unit that updates the position information of the ultrasound probe from second position information different from the first position information to the first position information.

6. the first position information includes an angle of a scan plane of an ultrasound probe capable of generating the first ultrasound image; the updating unit updates the scan plane by changing the angle of the scan plane from the angle of the scan plane included in the second position information to the angle of the scan plane included in the first position information. The ultrasonic diagnostic apparatus according to claim 5 .

7. 6. The ultrasound diagnostic device of claim 5, wherein the first generation unit generates a first ultrasound image including the medical instrument by performing the first scan using an ultrasound probe whose position information has been updated to the first position information.

8. The ultrasound diagnostic apparatus according to claim 7 , further comprising a display control unit that causes the first ultrasound image including the medical instrument to be displayed on a display.

9. The ultrasonic diagnostic apparatus according to claim 8 , wherein the display control unit causes a display to display display screen data in which the first ultrasonic image and the second ultrasonic image are arranged side by side.

10. The ultrasound diagnostic apparatus according to claim 1 , further comprising a first notification unit that notifies an operator of the first position information.

11. 2. The ultrasound diagnostic device of claim 1, wherein the estimation unit estimates the first position information of an ultrasound probe capable of generating a first ultrasound image including the medical instrument by inputting the second ultrasound image including the medical instrument, generated by executing the second scan, to a trained model that outputs first position information of an ultrasound probe capable of generating a first ultrasound image including the medical instrument based on the second ultrasound image including the medical instrument.

12. the medical device includes a first medical device or a second medical device; When a second ultrasound image including the first medical instrument is input to the trained model, the estimation unit estimates first position information of an ultrasound probe capable of generating a first ultrasound image including the first medical instrument, and when a second ultrasound image including the second medical instrument is input, the estimation unit estimates first position information of an ultrasound probe capable of generating a first ultrasound image including the second medical instrument. The ultrasonic diagnostic apparatus according to claim 11.

13. the medical device includes a first medical device or a second medical device; the trained model includes a first trained model that outputs first position information of an ultrasound probe that can generate a first ultrasound image including a first medical instrument based on a second ultrasound image including the first medical instrument, or a second trained model that outputs first position information of an ultrasound probe that can generate a first ultrasound image including a second medical instrument based on a second ultrasound image including the second medical instrument; a determination unit that determines, based on the second ultrasound image, whether the medical instrument included in the second ultrasound image is the first medical instrument or the second medical instrument; If the determination result indicates that the medical instrument included in the second ultrasound image is the first medical instrument, the estimation unit inputs the second ultrasound image into the first trained model, and if the medical instrument included in the second ultrasound image is the second medical instrument, the estimation unit inputs the second ultrasound image into the second trained model. The ultrasonic diagnostic apparatus according to claim 11.

14. the medical device includes a first medical device or a second medical device; the trained model includes a first trained model that outputs first position information of an ultrasound probe that can generate a first ultrasound image including a first medical instrument based on a second ultrasound image including the first medical instrument, or a second trained model that outputs first position information of an ultrasound probe that can generate a first ultrasound image including a second medical instrument based on a second ultrasound image including the second medical instrument; a second input unit to which a first input instruction to input the second ultrasound image into the first trained model or a second input instruction to input the second ultrasound image into the second trained model is input; The estimation unit inputs the second ultrasound image to the first trained model based on the first input instruction, and inputs the second ultrasound image to the second trained model based on the second input instruction. The ultrasonic diagnostic apparatus according to claim 11.

15. The ultrasound diagnostic apparatus according to claim 1 , further comprising a detector configured to detect whether the medical instrument is included in the first ultrasound image.

16. 16. The ultrasound diagnostic apparatus of claim 15, further comprising: a second notification unit that, when the detection result shows that the medical instrument is not included in the first ultrasound image, notifies an operator that the medical instrument is not included.

17. The ultrasound diagnostic apparatus according to claim 15 , wherein the second generation unit performs the second scan when the first ultrasound image does not include the medical instrument as a result of the detection.

18. a function of generating a first ultrasound image of a subject by performing a first scan with an ultrasound probe on the subject into which a medical instrument is inserted; performing a second scan that encompasses the first scan to generate a second ultrasound image that includes the medical device; a function of estimating first position information of an ultrasound probe capable of generating a first ultrasound image including the medical instrument based on the second ultrasound image; A program that makes the computer realize the above.

Images