Ultrasound diagnostic equipment

Abstract

To provide an ultrasonic diagnostic device that makes a puncture to a subject by a user easier.SOLUTION: An ultrasonic diagnostic device 1 for supporting a puncture of a puncture needle QT to a subject HT includes: an ultrasonic probe 20; an optical camera 30 attached to a base end side of the ultrasonic probe 20 for imaging a state of the puncture needle QT being punctured to the subject HT; an optical image analysis part 15 for detecting the posture of the puncture needle QT by executing image analysis of an optical image acquired by the optical camera 30; and an erroneous puncture determination part 17 for determining an occurrence of an erroneous puncture of the puncture needle QT on the basis of the detected posture of the puncture needle QT and a target posture of the puncture needle QT stored beforehand.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to an ultrasonic diagnostic apparatus.

Background Art

[0002] An ultrasonic diagnostic apparatus that transmits and receives ultrasonic waves to capture an anatomical structure of a subject as an ultrasonic image is known. The ultrasonic diagnostic apparatus can obtain an ultrasonic image of the shape and movement of tissues in the subject by an operation of applying an ultrasonic probe to the body surface of the subject.

[0003] The ultrasonic diagnostic apparatus is used in various departments and clinics such as the abdomen, cardiovascular system, superficial, blood vessels, obstetrics and gynecology, urology, orthopedics, intraoperative, and screening. Also, in orthopedics, anesthesiology, pain clinic, dialysis, etc., doctors, clinical engineers, etc. (hereinafter referred to as "users") use the ultrasonic diagnostic apparatus to perform ultrasound-guided puncture while confirming the anatomical structure of the subject and the position of the puncture needle.

[0004] At that time, in order to perform the puncture accurately, it is necessary to accurately grasp the positional relationship between the puncture position and puncture angle on the body surface of the patient and the affected part (i.e., the target) in the ultrasonic image displayed on the monitor.

[0005] From such a background, various technologies and functions for assisting puncture of a subject have been developed in ultrasonic diagnostic apparatuses.

[0006] For example, Patent Document 1 discloses a needle guide attached to an ultrasonic probe. Patent Document 1 describes configuring a needle guide so that a puncture needle can be punctured into a subject at a predetermined puncture angle, thereby assisting puncture of the subject. Also, Patent Document 1 describes displaying a guide line of the puncture needle on a monitor so that a user can easily recognize the puncture direction of the needle.

Prior Art Documents

Patent Documents

[0007] [Patent Document 1] Special Publication No. 2011-505227 [Overview of the project] [Problems that the invention aims to solve]

[0008] Incidentally, while needle guides like the one described in the prior art document 1 are useful in facilitating puncture of a subject, such needle guides are actually less convenient because the puncture angle is fixed.

[0009] For example, when puncturing a central vein such as the external jugular vein in humans, the course of the blood vessel is generally confirmed (see the left diagram in Figure 10), and the puncture is performed using the cross-legged method (see the right diagram in Figure 10). In this case, the user performs the puncture freehand, while confirming the puncture position in the blood vessel by looking at the ultrasound image and correcting the trajectory of the puncture needle. Figure 10 shows how the ultrasound probe is pressed against the body surface of the subject in the parallel method (left diagram in Figure 10) and the cross-legged method (right diagram in Figure 10), and the ultrasound images acquired at that time. In Figure 10, 20 represents the ultrasound probe, HT represents the subject, and QT represents the puncture needle.

[0010] However, when performing a freehand puncture, the points to keep in mind are the puncture angle (representing the elevation angle of the puncture needle relative to the subject's body surface; the same applies hereinafter) and the orientation of the needle's cutting edge at the time of puncture.

[0011] Figure 11 shows an example of a proper puncture method for a QT puncture needle (in this case, a blood collection needle).

[0012] During puncture, it is generally recommended to insert the puncture needle QT from the surface of the subject HT towards the target site in the subject's tissue (e.g., the central vein), with a puncture angle of 10° to 30°, as shown in Figure 11. Once it is confirmed that the puncture needle QT has entered the blood vessel, it is recommended to advance the puncture needle QT horizontally along the course of the blood vessel. Furthermore, it is recommended that the puncture needle QT be inserted with the blade facing upwards.

[0013] If the QT puncture needle is initially inserted into the subject's HT at an incorrect angle (e.g., 10° to 30°), there is a risk that the QT puncture needle may damage tissues other than the target site within the subject's HT (e.g., arteries or nerves). Furthermore, if the QT puncture needle is inserted with its blade facing downwards, there is a risk that the tip of the QT puncture needle may damage the wall of the central vein as it is advanced within the blood vessel.

[0014] However, such a puncture is a delicate procedure, and if not performed by a skilled examiner, there is a risk of mispuncture.

[0015] This disclosure has been made in view of the above-mentioned problems and aims to provide an ultrasound diagnostic device that makes it easier for users to perform punctures on subjects. [Means for solving the problem]

[0016] The main disclosure that addresses the aforementioned issues is: An ultrasound diagnostic device that assists in puncturing a subject, Ultrasound probe and An optical camera attached to the proximal end of the ultrasound probe is used to photograph the state in which the puncture needle is inserted into the subject. An optical image analysis unit detects the orientation of the puncture needle by performing image analysis on the optical image acquired by the optical camera, An erroneous puncture determination unit that determines the occurrence of an erroneous puncture of the puncture needle based on the detected posture of the puncture needle and the target posture of the puncture needle stored in advance. It is an ultrasonic diagnostic apparatus comprising the above.

Effect of the Invention

[0017] According to the ultrasonic diagnostic apparatus according to the present disclosure, it is possible to make it easier for a user to puncture a subject.

Brief Description of the Drawings

[0018] [Figure 1] A diagram showing the overall configuration of an ultrasonic diagnostic apparatus according to an embodiment of the present invention [Figure 2] A diagram showing the configuration of an ultrasonic probe according to an embodiment of the present invention [Figure 3] A diagram showing an example of the attachment state of an optical camera and a laser pointer to an ultrasonic probe in an ultrasonic diagnostic apparatus according to an embodiment of the present invention [Figure 4] A diagram showing an example of a monitor screen displayed on a monitor of an ultrasonic diagnostic apparatus in a puncture support mode activated during puncture according to an embodiment of the present invention [Figure 5] A diagram showing functional blocks of a control device according to an embodiment of the present invention [Figure 6] A diagram showing an example of a puncture needle reflected in an optical image during puncture in an ultrasonic diagnostic apparatus according to an embodiment of the present invention [Figure 7] A diagram for explaining a puncture by a user when puncturing a puncture needle on the body surface of a subject using an ultrasonic diagnostic apparatus according to an embodiment of the present invention [Figure 8] A flowchart showing an example of the operation of an ultrasonic diagnostic apparatus according to an embodiment of the present invention during puncture [Figure 9] A diagram showing an example of a notification mode of an ultrasonic diagnostic apparatus according to an embodiment of the present invention when an erroneous puncture occurs during puncture [Figure 10]Figure 10 shows how the ultrasound probe is pressed against the subject's body surface using the parallel method (left diagram) and the crossed-leg method (right diagram), and the ultrasound images acquired in each case. [Figure 11] This diagram shows an example of the proper method of inserting a puncture needle (in this case, a blood collection needle) during puncture. [Modes for carrying out the invention]

[0019] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same function are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0020] [Overall configuration of an ultrasound diagnostic system] First, with reference to Figures 1 and 2, the overall configuration of an ultrasound diagnostic device according to one embodiment of the present invention (hereinafter referred to as "ultrasound diagnostic device 1") will be described.

[0021] Figure 1 shows the overall configuration of the ultrasound diagnostic device 1 according to this embodiment.

[0022] Figure 2 shows the configuration of the ultrasonic probe 20 according to this embodiment. Figure 2A shows the configuration of the attachment 20T to be attached to the ultrasonic probe 20, and Figure 2B shows an example of how to attach the attachment 20T to the ultrasonic probe 20.

[0023] Figure 3 shows an example of the mounting state of the optical camera 30 and laser pointer 40 to the ultrasonic probe 20 according to this embodiment. Figure 3A is a side view of the ultrasonic probe 20 (meaning a view of the short axis side of the ultrasonic probe 20; the same applies hereinafter), and Figure 3B is a front view of the ultrasonic probe 20 (meaning a view of the long axis side of the ultrasonic probe 20; the same applies hereinafter).

[0024] Figure 4 shows an example of the monitor screen displayed on the monitor 50 of the ultrasound diagnostic device 1 in the puncture support mode activated during puncture. In Figure 4, the left region R1 of the monitor screen displays the ultrasound image acquired by the ultrasound probe 20, and the right region R2 of the monitor screen displays the optical image acquired by the optical camera 30.

[0025] The ultrasound diagnostic device 1 comprises a control unit 10, an ultrasound probe 20, an optical camera 30, a laser pointer 40, a monitor 50, a speaker 60, and an operation input unit 70. The ultrasound probe 20, optical camera 30, and laser pointer 40 are each connected to the control unit 10 via cables.

[0026] In the puncture using the ultrasound diagnostic device 1 according to this embodiment, the puncture needle QT is assumed to be inserted into the subject's HT freehand by the user, for example. The user operates the ultrasound diagnostic device 1 by bringing the ultrasonic beam transmitting and receiving surface of the ultrasound probe 20 into contact with the body surface of the subject's HT and obtains an ultrasound image of the subject's HT. The user then looks at the monitor 50 to confirm the target position within the subject's HT shown in the ultrasound image R1, and uses the optical image R2 acquired by the optical camera 30 to determine the target puncture position of the puncture needle QT when inserting it into the subject's HT, and then performs the puncture. At this time, the target puncture position of the puncture needle QT is displayed in the optical image R2 acquired by the optical camera 30 as a projection image 40L formed on the body surface of the subject's HT by the laser light emitted from the laser pointer 40, thereby enabling the user to perform an accurate puncture (see Figure 4).

[0027] The ultrasonic probe 20 transmits an ultrasonic beam (for example, about 1 to 30 MHz) into the subject HT (for example, the human body) and functions as an acoustic sensor that receives ultrasonic echoes reflected from the transmitted ultrasonic beam within the subject HT and converts them into electrical signals. In this embodiment, a linear probe is shown as an example of the ultrasonic probe 20, but any other probe such as a convex probe, sector probe, or three-dimensional probe can be applied to the ultrasonic probe 20.

[0028] The ultrasonic probe 20 is composed of, for example, a housing 21 and a transducer array 22 disposed at the probe tip 20a of the housing 21 (see Figure 2B).

[0029] The housing 21, for example, has an elongated shape and also serves as a gripping part that can be held by the user. An attachment 20T is attached to the base end of the housing 21, and the optical camera 30 and laser pointer 40 are fixed to the housing 21 via the attachment 20T.

[0030] The transducer array 22 is arranged on the probe tip 20a of the housing 21 so as to form an ultrasonic transmitting and receiving surface. The transducer array 22 is composed of a plurality of transducers (e.g., piezoelectric elements) arranged along the long axis direction (direction 20LL in Figure 2B) of the probe tip 20a of the housing 21. When acquiring an ultrasonic image, the on / off state of each transducer of the transducer array 22 is switched sequentially, thereby performing an ultrasonic scan within the subject HT along the long axis direction of the probe tip 20a, and generating an ultrasonic image representing the two-dimensional structure in the cross-section, including the ultrasonic transmission direction (i.e., the depth direction of the subject HT) and the ultrasonic scanning direction (i.e., the long axis direction of the probe tip 20a).

[0031] The control device 10 is the main unit of the ultrasound diagnostic device 1 and generates an ultrasound image based on the received signal acquired from the ultrasound probe 20. The control device 10 then adds an optical image acquired by the optical camera 30 to the ultrasound image to generate a display image to be shown on the monitor 50.

[0032] The control device 10 is configured around a well-known computer consisting of, for example, a CPU, ROM, and RAM, and in addition, it includes a detection circuit for receiving and processing the received signal acquired from the ultrasonic probe 20, and a cine memory for storing time-series ultrasonic images and time-series optical images. However, it goes without saying that part or all of the control device 10 can also be realized by dedicated hardware circuits (for example, ASIC or FPGA) that do not have a CPU, etc.

[0033] The optical camera 30 is, for example, a general-purpose visible light camera, and it performs AD conversion on the image signal generated by its own image sensor to generate image data related to the optical image (hereinafter abbreviated as "optical image"). The optical camera 30 then continuously generates optical images in frame units and outputs the optical images arranged in time series (i.e., moving images of the optical images) to the control device 10. The optical camera 30 may also have a zoom lens and be capable of magnifying the target to be photographed (in this case, the body surface area of ​​the subject HT) and capturing images.

[0034] The optical camera 30 is attached to the proximal end of the ultrasound probe 20 and captures the state in which the puncture needle QT is inserted into the subject HT from above the body surface of the subject HT. It captures the position of the probe tip 20a of the ultrasound probe 20 on the body surface of the subject HT. The optical camera 30 is attached to the ultrasound probe 20 so that the probe tip 20a of the ultrasound probe 20, the projected image 40L formed by the laser light from the laser pointer 40 projected onto the body surface of the subject HT, and the observation target area of ​​the ultrasound image R1 on the body surface of the subject HT are all captured in the optical image R2.

[0035] This allows the user to recognize from the optical image R2 the relative positional relationship between the probe tip 20a of the ultrasound probe 20 and the ultrasound scan cross-section (i.e., the tomographic plane of the ultrasound image) on the body surface of the subject HT, as well as the target puncture position of the puncture needle QT on the body surface of the subject HT during puncture (see Figure 4).

[0036] The laser pointer 40 is a general semiconductor laser that emits visible colored laser light (for example, red laser light with a wavelength of 635 nm to 690 nm). The laser pointer 40 is attached to the proximal end of the ultrasound probe 20 and emits laser light onto the body surface of the subject HT to form a predetermined projection image 40L, thereby guiding the target puncture position of the puncture needle QT when puncturing the subject HT with the optical camera 30 in the optical image R2.

[0037] The laser pointer 40 outputs laser light, for example, using its built-in diffraction grating or slit, so that the shape of the projected image 40L of the laser light onto the body surface of the subject HT (i.e., the irradiation shape) is linear.

[0038] The projection image 40L formed by the laser beam of the laser pointer 40 guides the target puncture position of the puncture needle QT within the optical image R2 acquired by the optical camera 30, using the probe tip 20a of the ultrasound probe 20 as the reference position. For example, the projection image 40L formed by the laser beam of the laser pointer 40 exhibits a line shape on the body surface of the subject HT, starting from the central position 20aa (hereinafter also referred to as the "sound axis center") in the long axis direction (direction 20LL shown in Figure 2B) of the probe tip 20a of the ultrasound probe 20, and extending in a direction perpendicular to the long axis direction (i.e., away from the probe tip 20a).

[0039] This makes it possible to allow the user to recognize the central position 20aa of the probe tip 20a of the ultrasound probe 20 in the longitudinal direction, and the direction perpendicular to the longitudinal direction, on the body surface of the subject HT. In other words, this makes it possible to guide the user to the position of the target site (for example, the blood vessel to be punctured) shown in the ultrasound image R1, and furthermore, the target puncture position of the puncture needle QT when puncturing the subject HT, using the position of the probe tip 20a of the ultrasound probe 20 placed on the body surface of the subject HT as a reference (see Figure 4).

[0040] Furthermore, the target puncture position HTb of the puncture needle QT (see Figure 4) can be recognized, for example, when the orientation of the puncture needle QT is adjusted to align with the projected image 40L of the laser beam in a plan view, and the elevation angle of the puncture needle QT relative to the body surface of the subject HT is adjusted to a predetermined angle (e.g., 30 degrees). For example, if the target site HTA is located 2 cm from the body surface of the subject HT, the target puncture position HTb of the puncture needle QT will be located approximately 3.5 cm (= 2 cm / tan 30°) away from the central position 20aa in the long axis direction of the probe tip 20a, along the projected image 40L of the laser beam. The target puncture position HTb of the puncture needle QT can be recognized, for example, by visually observing the position of the target site as seen in the ultrasound image R1, and the projected image 40L of the laser beam and the probe tip 20a as seen in the optical image R2. In Figure 4, horizontal and vertical dash lines R2b and R2a are superimposed on the optical image R2 to further facilitate the recognition of the target puncture position HTb of the puncture needle QT.

[0041] The camera 30 and laser pointer 40 are set up so that, in a front view, the sound axis center of the ultrasound probe 20, the optical axis of the camera 30, and the optical axis of the laser pointer 40 coincide (see Figure 3). In other words, the optical axis of the optical camera 30 and the sound axis center of the ultrasound probe 20 are set up so that, when projected onto the body surface of the subject HT, they coincide with the central axis of the projected image 40L of the laser beam output by the laser pointer 40. As a result, the projected image 40L of the laser beam projected onto the body surface of the subject HT points in a direction perpendicular to the long axis, starting from the central position 20aa in the long axis direction of the probe tip 20a of the ultrasound probe 20. That is, the user can recognize the target puncture position of the puncture needle QT by looking at the projected image 40L of the laser beam projected onto the optical image R2.

[0042] In the ultrasound diagnostic apparatus 1 according to this embodiment, the camera 30 and laser pointer 40 are attached to the housing 21 of the ultrasound probe 20 via a detachable attachment 20T so as to be in a predetermined positional relationship with the ultrasound probe 20 (see Figure 2). The positional relationship of the camera 30 and laser pointer 40 with respect to the ultrasound probe 20 is determined by the attachment 20T.

[0043] The monitor 50 is a display that shows the display image generated by the control device 10, and is configured as, for example, a liquid crystal display. The monitor 50 acquires the display image data from the display image generation unit 14 and displays the display image. The speaker 60 outputs sound commands from the control device 10 (mis-puncture detection unit 17).

[0044] The operation input unit 70 receives input such as commands to instruct the start of diagnosis or information about the subject's HT. The operation input unit 70 includes, for example, an operation panel with multiple input switches, a keyboard, and a mouse. The operation input unit 70 may also be configured as a touch panel integrated with the monitor 50.

[0045] Figure 5 shows the functional blocks of the control device 10 according to this embodiment.

[0046] The control device 10 includes a transmitting / receiving unit 11, an ultrasound image generation unit 12, an optical image acquisition unit 13, a display image generation unit 14, an optical image analysis unit 15, a target posture data selection unit 16, and a mispuncture determination unit 17.

[0047] The transmitting / receiving unit 11 sends a voltage pulse, which is a drive signal, to the ultrasonic probe 20 in accordance with the instructions of the control unit 19, and also receives and processes the received signal related to the ultrasonic echo generated by the ultrasonic probe 20.

[0048] When the transmitting / receiving unit 11 functions as a transmitter, it adjusts, for example, a voltage pulse to a predetermined voltage amplitude, pulse width, and transmission timing, and transmits it to each channel of the ultrasonic probe 20. The transmitting / receiving unit 11 then changes the target depth or generates different pulse waveforms by setting an appropriate delay time for each of the multiple channels.

[0049] When the transmitting / receiving unit 11 functions as a receiver, for example, it amplifies the received signal related to the weak ultrasonic echo for each channel and converts the received signal into a digital signal. Then, the transmitting / receiving unit 11 combines the received signals of multiple channels into one by phase-correcting summation using a receiving beamformer, and converts it into acoustic line data.

[0050] The ultrasound image generation unit 12 acquires a received signal (acoustic line data) from the transmitting / receiving unit 11 and generates an ultrasound image (i.e., a tomographic image) of the inside of the subject's HT.

[0051] The ultrasound image generation unit 12, for example, continuously stores the signal intensity of the ultrasound echo detected after the ultrasound probe 20 transmits a pulsed ultrasound beam in the depth direction in line memory over time. Then, as the ultrasound beam from the ultrasound probe 20 scans the inside of the subject HT, the ultrasound image generation unit 12 sequentially stores the signal intensity of the ultrasound echo at each scanning position in line memory, generating two-dimensional data in frame units. The ultrasound image generation unit 12 then converts the signal intensity of the two-dimensional data into brightness values ​​to generate an ultrasound image representing the two-dimensional structure in the cross-section, including the ultrasound transmission direction and the ultrasound scanning direction.

[0052] The optical image acquisition unit 13 sequentially acquires optical images that are continuously generated over time from the optical camera 30. For example, the optical image acquisition unit 13 sequentially acquires optical images at intervals corresponding to the frame rate at which optical images are generated by the optical camera 30.

[0053] The display image generation unit 14 generates a display image by arranging the ultrasound image R1 acquired by the ultrasound probe 20 and the optical image R2 acquired by the optical camera 30 side by side, as shown in Figure 4, for example. The display image generation unit 14 then sends the generated display image data to the monitor 50, causing the monitor 50 to display the display image. The display image generation unit 14 then sequentially updates the display image, displaying a video-format display image on the monitor 50 (see Figure 4).

[0054] Furthermore, the display image generation unit 14 identifies the temporal correspondence between the timing at which the ultrasound image was generated and the timing at which the optical image was generated, based, for example, on the timestamp attached to the ultrasound image and the timestamp attached to the optical image, and generates a display image that includes the ultrasound image and optical image generated at the same or close timing. The display image generation unit 14 then updates the display image in real time each time it acquires new ultrasound image data from the ultrasound image generation unit 12 and / or new optical image data from the optical image acquisition unit 13, and displays the display image on the monitor 50 in video format.

[0055] The optical image analysis unit 15 detects the orientation of the puncture needle QT by analyzing the optical image R2 acquired by the optical camera 30 using a known pattern recognition process. Here, the orientation of the puncture needle QT that the optical image analysis unit 15 detects is the orientation of the cutting edge of the puncture needle QT and the puncture angle of the puncture needle QT.

[0056] The optical image analysis unit 15 detects the orientation of the puncture needle QT, for example, using a pre-trained classifier model that has undergone machine learning. This type of classifier, through training, extracts features of the pattern to be identified and is autonomously optimized to accurately identify the pattern to be identified even from data with superimposed noise. In other words, this type of classifier model is trained using the orientation (in this case, the direction of the blade surface and the puncture angle) as labels and the image features of the puncture needle QT as training data, so that it can identify the orientation of the puncture needle QT in an optical image even when an unknown optical image is input.

[0057] The classifier model according to this embodiment is trained using machine learning with training data consisting of, for example, information on various orientations of the puncture needle QT (e.g., puncture angle and blade orientation) and optical images of the puncture needle QT taken when it is in that orientation. This allows the classifier model to identify the orientation of the puncture needle QT from any optical image. Examples of such classifier models include SVM (Support Vector Machine), Random Forest, or Convolutional Neural Network.

[0058] Figure 6 shows an example of the image characteristics of the puncture needle QT as seen in the optical image R2 during puncture.

[0059] Figure 6A shows the appearance of the lancet QT as captured by the optical camera 30 when the puncture angle is the target angle (e.g., 30 degrees) and the blade surface is facing the correct direction (i.e., upward). Figure 6B shows the appearance of the lancet QT as captured by the optical camera 30 when the puncture angle becomes greater than the target angle (e.g., 30 degrees) from the state in Figure 6A. Figure 6C shows the appearance of the lancet QT as captured by the optical camera 30 when the blade surface is facing the incorrect direction (i.e., downward) from the state in Figure 6A. Note that QTa represents the tip QTa of the lancet QT.

[0060] In this embodiment, the optical camera 30 is attached to a predetermined position on the ultrasound probe 20 so as to capture the process of the puncture needle QT being inserted into the subject HT from above the surface of the subject HT. Therefore, the optical image R2 will show the external appearance of the puncture needle QT from the proximal end to the tip QTa in a plan view.

[0061] The orientation of the cutting edge of the puncture needle QT and the puncture angle of the puncture needle QT can generally be determined from the contour shape of the puncture needle QT as seen in the optical image R2. The puncture angle of the puncture needle QT can be determined, for example, as the ratio of the diameter of the proximal end to the diameter of the tip QTa of the puncture needle QT as seen in the optical image R2. This is because when the puncture angle of the puncture needle QT is greater than the target angle, the ratio of the diameter of the proximal end to the diameter of the tip QTa of the puncture needle QT as seen in the optical image R2 is greater than the ratio when the puncture angle of the puncture needle QT is at the target angle (see Figure 6B). Furthermore, the orientation of the cutting edge of the puncture needle QT can be determined, for example, by the orientation of the needle hole in the tip QTa of the puncture needle QT as seen in the optical image R2.

[0062] The target posture data selection unit 16 selects target posture data to be applied to the judgment process of the mispuncture determination unit 17 from the group of target posture data stored in the database D1, based on the type of puncture needle QT specified by the user. This target posture data indicates the target posture of the puncture needle QT, including the orientation of the cutting edge of the puncture needle QT and the puncture angle of the puncture needle QT.

[0063] Furthermore, the target posture data stored in database D1 is stored in association with the type of puncture needle QT (e.g., blood collection needle or indwelling needle) and the thickness of the puncture needle QT.

[0064] The mispuncture detection unit 17 determines whether or not a mispuncture occurred during puncture based on the orientation of the puncture needle QT detected by the optical image analysis unit 15 and the target orientation data related to the target orientation of the puncture needle QT selected by the target orientation data selection unit 16. If a mispuncture has occurred, the mispuncture detection unit 17 notifies the user of this fact by display output to the monitor 50 or by audio output to the speaker 60.

[0065] Furthermore, the erroneous puncture detection unit 17 performs the detection process by, for example, comparing the orientation of the blade surface and puncture angle of the current puncture needle QT detected by the optical image analysis unit 15 with the orientation of the blade surface and puncture angle of the puncture needle QT related to the target posture set in the target posture data.

[0066] [Operation Flow of Ultrasound Diagnostic Device 1] First, referring to Figure 7, we will explain the user's puncture technique when inserting the puncture needle QT into the body surface of the subject HT using the ultrasound diagnostic device 1.

[0067] First, the user activates the puncture support mode on the ultrasound diagnostic device 1, and then, while viewing the ultrasound image R1 displayed on the monitor 50, moves the ultrasound probe 20 so that the target site HTa within the subject's HT is at the center of the scanning direction of the ultrasound image R1 (i.e., the center position 20aa in the long axis direction of the probe tip 20a) (step S1).

[0068] Next, the user looks at the optical image R2 displayed on the monitor 50 and inserts the puncture needle QT into the body surface of the subject HT along the linear laser beam projection image 40L of the laser pointer 40 formed on the body surface of the subject HT (step S2). At this time, the user looks at the ultrasound image R1 and optical image R2 displayed on the monitor 50 to confirm the target puncture position when inserting the puncture needle QT into the body surface of the subject HT. The target puncture position of the puncture needle QT is, for example, a position on the linear laser beam projection image 40L, and a predetermined distance from the probe tip 20a on the body surface of the subject HT, from the central position 20aa in the long axis direction of the probe tip 20a of the ultrasound probe 20. If the depth of the target site HTa from the body surface of the subject HT is 2 cm, and assuming an elevation angle of 30°, the target puncture position of the puncture needle QT will be approximately 3.5 cm away from the probe tip 20a on the body surface of the subject HT, from the central position 20aa of the probe tip 20a of the ultrasound probe 20 along the long axis.

[0069] Next, the user looks at the optical image R2 displayed on the monitor 50 and advances the puncture needle QT into the subject HT so that the orientation (i.e., angle) of the puncture needle QT in the plan view does not deviate from the projected image 40L of the laser beam (or from the vertical imaginary line R2a) (step S3).

[0070] Through the above procedure and technique, the QT puncture needle reaches the target site HTa within the subject's HT without requiring any adjustments during puncture. The user then confirms in the ultrasound image R1 that the tip of the QT puncture needle appears at the target site HTa (in this case, 2 cm from the body surface of the subject's HT) and completes the puncture (step S4). The tip of the QT puncture needle will typically appear as a white bright spot in the ultrasound image R1.

[0071] Next, the operation flow of the ultrasound diagnostic device 1 according to this embodiment during puncture will be described.

[0072] Figure 8 is a flowchart illustrating an example of the operation of the ultrasound diagnostic device 1 during puncture. This flowchart shows, for example, the processes that the control device 10 executes according to the computer program when the puncture support mode is started.

[0073] Figure 9 shows an example of how the ultrasound diagnostic device 1 notifies when an accidental puncture occurs during puncture.

[0074] First, the control device 10 uses the optical camera 30 to acquire an optical image R2 of the state in which the puncture needle QT is inserted into the subject's HT (step S11).

[0075] Next, the control device 10 detects the puncture needle QT from the optical image R2 using, for example, a known background subtraction method (step S12).

[0076] Next, the control device 10 detects the orientation of the puncture needle QT as seen in the optical image R2, for example, by image analysis using a trained classifier model (step S13). In step S13, the control device 10 also detects, for example, the puncture angle of the puncture needle QT and the orientation of the cutting edge of the puncture needle QT.

[0077] Next, the control device 10 determines whether or not a false puncture has occurred based on the orientation of the puncture needle QT detected in step S13 and the pre-stored target orientation of the puncture needle QT (step S14). If a false puncture has occurred (S14: YES), the control device 10 proceeds to step S15. On the other hand, if a false puncture has not occurred (S14: NO), the process shown in the flowchart of Figure 8 is terminated without executing the process in step S15.

[0078] Next, the control device 10 embeds a notification image (for example, an image prompting correction of a puncture error) into the display image and displays the said display image on the monitor 50 (step S15). At the same time, the control device 10 outputs a notification sound from the speaker 60.

[0079] The control device 10 repeats the processes of steps S11 to S15 at predetermined intervals, monitors for the occurrence of erroneous punctures, and guides the user to ensure that the puncture needle QT is inserted into the subject HT in the target position.

[0080] [effect] As described above, the ultrasound diagnostic device 1 according to this embodiment is Ultrasound probe 20, An optical camera 30 is attached to the proximal end of the ultrasound probe 20 to capture images of the puncture needle QT being inserted into the subject HT, and an optical image analysis unit 15 detects the orientation of the puncture needle QT by analyzing the optical images acquired by the optical camera 30. The system includes a mispuncture detection unit 17 that determines the occurrence of a mispuncture of the puncture needle QT based on the detected orientation of the puncture needle QT and a pre-stored target orientation of the puncture needle QT.

[0081] Therefore, according to the ultrasound diagnostic device 1 of this embodiment, it is possible to suppress the situation in which the puncture needle QT damages other tissues (e.g., arteries or nerves) other than the target site due to the puncture needle QT being inserted into the subject HT at an inappropriate puncture angle or blade orientation.

[0082] [Example 1] It is more preferable to perform the puncture of the subject HT with a QT needle while a background cover sheet (not shown) is placed on the body surface of the subject HT.

[0083] This clarifies the boundary between the background of the puncture needle QT and the region where the puncture needle QT is located in the optical image, allowing the optical image analysis unit 15 to more accurately detect the orientation (puncture angle, etc.) of the puncture needle QT.

[0084] For example, a white sheet material having a flat surface can be used as the sheet material. By placing such a white sheet material having a flat surface on the body surface of the subject HT and then performing the puncture with the puncture needle QT, it is possible to suppress deterioration of the detection accuracy of the optical image analysis unit 15 due to individual differences depending on the skin color of the subject HT.

[0085] [Differentiation 2] A stereo camera is more preferable as the optical camera 30.

[0086] This makes it possible to accurately determine the position of the puncture needle QT in the depth direction, allowing the optical image analysis unit 15 to more accurately perform the process of detecting the orientation (puncture angle, etc.) of the puncture needle QT.

[0087] Although specific examples of the present invention have been described in detail above, these are merely illustrative and do not limit the scope of the claims. The technologies described in the claims include various modifications and changes to the specific examples illustrated above. [Industrial applicability]

[0088] The ultrasound diagnostic device described herein makes it possible to make it easier for the user to perform punctures on the subject. [Explanation of symbols]

[0089] 1. Ultrasound diagnostic equipment 10 Control device 11 Transmitter / Receiver 12 Ultrasound Image Generation Unit 13 Optical image acquisition unit 14 Display Image Generation Unit 15 Optical image analysis department 16. Target posture data selection unit 17 Erroneous puncture determination unit 20 Ultrasound probes 30 Optical Cameras 40 Laser Pointers 50 monitors 60 speakers 70 Operation Input Section HT subject QT puncture needle

Claims

1. An ultrasound diagnostic device that assists in puncturing a subject, Ultrasound probe and An optical camera attached to the proximal end of the ultrasound probe is used to photograph the state in which the puncture needle is inserted into the subject. An optical image analysis unit detects the orientation of the puncture needle by performing image analysis on the optical image acquired by the optical camera, A mispuncture determination unit determines the occurrence of a mispuncture by the puncture needle based on the detected orientation of the puncture needle and a pre-stored target orientation of the puncture needle, Equipped with, The target position of the puncture needle is set based on the type of puncture needle specified by the user. Ultrasound diagnostic equipment.

2. The orientation of the puncture needle to be detected by the optical image analysis unit is the orientation of the cutting edge of the puncture needle and the puncture angle of the puncture needle. The ultrasound diagnostic apparatus according to claim 1.

3. The optical image analysis unit detects the orientation of the puncture needle using a trained classifier model that has undergone machine learning. The ultrasound diagnostic apparatus according to claim 1.

4. The aforementioned mispuncture detection unit notifies the user of the occurrence of a mispuncture by displaying an output on the monitor or by outputting an audio message to the speaker. The ultrasound diagnostic apparatus according to claim 1.

5. The optical camera is a stereo camera. The ultrasound diagnostic apparatus according to claim 1.

6. The puncture of the subject with the puncture needle is performed with a background cover sheet placed on the subject's body surface. The ultrasound diagnostic apparatus according to claim 1.

7. The system further includes a display image generation unit that generates a display image so that the optical image is displayed on the same monitor screen as the ultrasound image. The ultrasound diagnostic apparatus according to claim 1.

Images