Ultrasound imaging device and medical support system

Abstract

To provide an ultrasonic imaging apparatus which allows a user to grasp a three-dimensional physical relation between a guide wire and an imaging object such as a blood vessel by using a linear array probe.SOLUTION: An ultrasonic imaging apparatus comprises: an ultrasonic probe 20 which irradiates a subject with an ultrasonic wave to receive a reflection wave of the ultrasonic wave and receives the ultrasonic wave from a beacon 13 inserted to the subject; a probe position acquisition unit 425 which acquires a three-dimensional position and a direction of the ultrasonic probe; a beacon position acquisition unit 424 which acquires a three-dimensional position of the beacon from a relative position or a relative speed with respect to the ultrasonic probe of the beacon obtained from an ultrasonic image based on the ultrasonic wave received by the ultrasonic probe and the three-dimensional position of the ultrasonic probe acquired by the probe position acquisition unit; and an image formation unit 43 which forms an image displayed on a display unit by using the ultrasonic image based on the ultrasonic wave received by the ultrasonic probe.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an ultrasound imaging apparatus that captures ultrasound images by inserting a guidewire equipped with an ultrasound generating source such as a photoacoustic ultrasound source into a body, a medical treatment support system, and an image display method. [Background technology]

[0002] Catheterization is a procedure that places less of a burden on the patient than surgery such as thoracotomy, and is therefore widely used primarily for the treatment of vascular stenosis. In catheterization, it is important to understand the relationship between the area to be treated and the catheter, and X-ray fluoroscopy is used as an imaging method to support this. There have also been attempts to use ultrasound images as support images instead of X-ray fluoroscopy images, as in Patent Document 1.

[0003] In detail, Patent Document 1 discloses a technology in which the position of the guidewire tip is estimated using an image of the ultrasound source, which is dependent on the arrival time difference of ultrasound (ultrasound from the ultrasound source) that reaches the element array constituting the ultrasound probe or the distance from the imaging area, regarding ultrasound generated by an ultrasound source mounted on a guidewire, and this estimation result is used to grasp the relative positional relationship between the imaging result and the guidewire tip. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-213680 A Summary of the Invention [Problem to be solved by the invention]

[0005] According to the above-mentioned prior art, it is possible to estimate the position of the tip of the insertion object (guide wire) within the imaging area. However, to grasp the three-dimensional positional relationship between the imaging target, such as a blood vessel, and the tip of the insertion object, it is necessary to use a two-dimensional array probe in which the element array constituting the ultrasound probe is arranged in a matrix. Furthermore, while it is possible to grasp the relative positional relationship of the catheter and the like with respect to the imaging region, it is not possible to detect the absolute position, and therefore it is not possible to align the position with information on the living body obtained by, for example, another imaging method.

[0006] The present invention aims to provide an ultrasound imaging device, a treatment support system, and an image display method that can grasp the three-dimensional positional relationship between a guidewire and an imaging target such as a blood vessel using a linear array probe. [Means for solving the problem]

[0007] In order to solve the above problems, the ultrasound imaging device of the present invention includes an ultrasound probe that irradiates ultrasound to a subject and receives reflected waves of the ultrasound, as well as receiving ultrasound from a beacon inserted inside the subject; a probe position acquisition unit that acquires a three-dimensional position and orientation of the ultrasound probe; a beacon position acquisition unit that acquires a three-dimensional position of the beacon from a relative position or relative velocity of the beacon with respect to the ultrasound probe obtained from an ultrasound image generated by ultrasound received by the ultrasound probe and the three-dimensional position and orientation of the ultrasound probe acquired by the probe position acquisition unit; and a display image forming unit that forms an image to be displayed on a display unit using the ultrasound image generated by ultrasound received by the ultrasound probe. Effect of the Invention

[0008] According to the present invention, the linear array probe can be used to show the surgeon the three-dimensional positional relationship between the guide wire and the imaging target such as a blood vessel. [Brief description of the drawings]

[0009] [Figure 1]1 is a diagram illustrating an overall configuration of a medical support system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating the state of the tip of a guide wire. [Diagram 3] FIG. 1 is a block diagram of an ultrasound imaging device. [Figure 4] FIG. 2 is a process flow diagram of the ultrasound imaging device. [Diagram 5] FIG. 2 is a diagram for explaining a coordinate system of an ultrasonic probe. [Figure 6] 1A to 1C are diagrams illustrating a method for detecting the absolute position of a PA signal generation source from changes in the positions of an ultrasonic probe and the PA signal generation source. [Figure 7] 1A to 1C are diagrams illustrating a method for detecting the absolute position of a PA signal generation source from a change in the rotational position of an imaging plane of an ultrasound probe. [Figure 8] FIG. 13 is a process flow diagram for detecting the absolute position of a PA signal generation source using the velocity of an ultrasonic probe. [Figure 9] FIG. 11 is a block diagram of an ultrasound imaging device having another configuration. [Figure 10] FIG. 11 is another process flow diagram of the ultrasonic imaging apparatus. [Figure 11] FIG. 4 is a diagram showing a display example of a display unit. [Figure 12] FIG. 11 is a diagram showing another display example of the display unit. [Figure 13A] FIG. 11 is a diagram showing an example of a movement plan of the robot arm by a motion planning unit. [Figure 13B] FIG. 2 is a diagram showing an example of an ultrasound image captured by an ultrasound probe. [Figure 14A] FIG. 11 is a diagram showing another example of a movement plan of the robot arm by the motion planning unit. [Figure 14B] FIG. 13 is a diagram showing another example of an ultrasound image captured by an ultrasound probe. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram for explaining the overall configuration of an ultrasonic imaging apparatus according to this embodiment and a catheter treatment support system (hereinafter, abbreviated as medical support system) using the ultrasonic imaging apparatus. FIG. 2 is a diagram for explaining the state of the tip of the guidewire.

[0011] As shown in FIG. 1, the medical support system 100 includes a bioinsertion tool 11 (guidewire) equipped with an ultrasound generating device 10 consisting of an ultrasound generating source 13 (beacon) and a light generating unit 15, an ultrasound probe 20 (probe), and a robot arm 90, as well as an ultrasound imaging unit 30 for acquiring ultrasound images of a subject 80 into which the bioinsertion tool 11 has been inserted and a display unit 34 for the same.

[0012] The living body insertion device 11 is, for example, a long and slender tubular medical device such as a medical device for treatment, such as a balloon catheter, a microcatheter, or a nutrition catheter, or a guidewire for carrying such a medical device to a target site. In the following, a case where the living body insertion device is a guidewire will be described.

[0013] Below, an ultrasonic generating device 10 including a photoacoustic (PA) signal source 13 that generates an ultrasonic signal by the PA effect will be described, but ultrasonic waves may also be generated by a piezoelectric element.

[0014] 1 and 2, the ultrasound generating device 10 includes an optical fiber 12 (not shown) disposed in the hollow portion of a flexible hollow guidewire 11, a photoacoustic ultrasound generating source 13 fixed to the insertion end face of the optical fiber 12, and a light generating unit 15 that generates laser light and is connected to the other end of the optical fiber 12 (the end opposite to the end to which the photoacoustic ultrasound generating source 13 is fixed). The optical fiber 12 functions as a light guiding member that guides the laser light generated by the light generating unit 15 to the photoacoustic ultrasound generating source 13 at the tip. The ultrasound generating device 10 and the hollow guidewire 11 are sometimes collectively referred to as a photoacoustic source-mounted wire.

[0015] 2, ultrasonic waves emitted by a photoacoustic ultrasonic source 13 (hereinafter, referred to as a PA signal source 13) at the tip of a guide wire 11 inserted into a blood vessel 82 are detected by an ultrasonic probe 20. Then, an ultrasonic imaging unit 30 superimposes an image of the detected PA signal source 13 on a cross-sectional image of the subject 80 and blood vessel 82 formed based on the ultrasonic waves emitted by the ultrasonic probe 20. This allows the position of the guidewire 11 in the subject 80 and blood vessel 82 to be ascertained.

[0016] The PA signal generating source 13 is made of a material that emits ultrasonic waves such as a PA signal by adiabatic expansion upon receiving laser light, such as a known dye (photosensitizer), metal nanoparticles, carbon-based compounds, etc. The tip of the optical fiber 12 including the PA signal generating source 13 is covered with a resin sealing member. In FIG. 2, the PA signal generating source 13 is positioned at the tip of the guidewire 11, but is not limited to the wire tip.

[0017] Next, details of the ultrasonic imaging section 30 constituting the ultrasonic imaging device of the embodiment will be described.

[0018] The ultrasonic imaging unit 30 includes a control unit 40, which will be described in detail later, a transmission unit 31 that transmits ultrasonic signals to the ultrasonic probe 20, a reception unit 32 that receives reflected waves (RF signals) detected by the ultrasonic probe 20 and performs processing such as phasing and addition, an input unit 33 through which the user inputs conditions and commands required for imaging, a display unit 34 that displays ultrasonic images acquired by the ultrasonic imaging unit 30, a GUI (Graphic User Interface), etc., and a memory 35 that stores processing results and display images formed by a display image forming unit 43 based on the processing results.

[0019] The control unit 40 includes a signal processing unit 41 that processes ultrasound images (including reflected ultrasound signals and PA signals) received by the ultrasound probe 20, a PA sound source position detection unit 42 that detects the sound source position of the PA signal from the signal processed by the signal processing unit 41, a display image forming unit 43 that forms an image to be displayed on the display unit 34 using the PA sound source position detected by the PA sound source position detection unit 42 and pre-acquired 3D anatomical information 44 of the subject that has been acquired in advance, and an action planning unit 45 that determines the operation position of the ultrasound probe 20 using the sound source position detected by the PA sound source position detection unit 42 and the pre-acquired 3D anatomical information 44 of the subject that has been acquired in advance.

[0020] The pre-acquired 3D anatomical information 44 may be a 3D volume image acquired by the subject using CT (Computer Tomography) or MRI (Magnetic Resonance Imaging), or a 3D volume image captured by sweeping with an ultrasound imaging device.

[0021] The signal processing unit 41 includes a reflected ultrasonic signal processing unit 411 that creates an ultrasonic image such as a B-mode image using an RF signal, which is a reflected wave received by the receiving unit 32, and an ultrasonic signal analysis unit (PA signal analysis unit) 412 that detects and processes the PA signal emitted from the PA signal generating source 13 and detected by each transducer element of the ultrasonic probe 20 based on the timing of emitting the laser light from the light generating unit 15.

[0022] The control unit 40 has the same configuration as a general ultrasonic imaging device, except that a PA signal analysis unit 412 that receives a PA signal and a PA sound source position detection unit 42 that detects the generation position of the PA signal are added.

[0023] The PA sound source position detection unit 42 includes a relative position detection unit 421 that estimates the position of the PA signal source 13 within the imaging area from the PA signal analyzed by the PA signal analysis unit 412, a relative speed measurement unit 422 that differentiates the position of the PA signal source 13 detected by the relative position detection unit 421 to derive the speed, a probe speed measurement unit 425 that measures the speed of the ultrasonic probe 20 based on the operation position information of the robot arm 90, an absolute position detection unit 424 that detects the absolute position of the PA signal source 13 based on the speed measured by the relative speed measurement unit 422 and the speed of the ultrasonic probe 20 measured by the probe speed measurement unit 425, and a position filter 423 that filters the absolute position detected by the absolute position detection unit 424 to reduce errors.

[0024] The position filter 423 uses any filtering process to reduce errors included in the detection result of the absolute position detection unit 424. For example, if the PA signal generation source 13 moves at a low speed, the position detection error can be reduced by smoothing using a filter such as a moving average filter or an LPF (Low Pass Filter).

[0025] Furthermore, the position filter 423 may be a filter that takes into consideration an error model that models a detection error that may occur in principle in the position detection method of the PA sound source position detection unit 42. Specifically, a Kalman filter or a particle filter that estimates a state with the highest statistical probability by comprehensively considering the position and speed of the PA signal generation source 13 detected by the PA sound source position detection unit 42, the position, speed, and attitude of the ultrasonic probe 20, and the error model of the PA sound source position detection unit 42, can be applied. Other components of the PA sound source position detector 42 will be described in detail later.

[0026] Some or all of the functions of the control unit 40 can be realized by executing software in which the functions are programmed in a computer equipped with a CPU or GPU and memory. It is also possible to realize some or all of the functions of each unit using hardware such as electronic circuits, ASICs, and FPGAs. The control unit 40 may be implemented in a single computer, or each function may be implemented in multiple computers.

[0027] The ultrasonic probe 20 is a 1D array probe (linear array probe) having one array arrangement in which many transducer elements are arranged in a one-dimensional direction. In addition, various ultrasonic probes 20 can be used, such as a 1D3 array probe having two or three array arrangements in a direction perpendicular to the array arrangement direction of the 1D array probe, and a 2D array probe having many array arrangements in a two-dimensional direction. The signal processing unit 41 adopts an analysis method according to the type of ultrasonic probe 20 used.

[0028] Next, the operation of the medical support system 100 according to the embodiment will be described. Here, the ultrasonic imaging device performs ultrasonic imaging of a subject 80 using an ultrasonic probe 20, which is a 1D array probe, while an operator inserts a guidewire 11 having a PA signal generator 13 at its tip for guiding a catheter or the like into the body of the subject, and the ultrasonic imaging device monitors the tip position of the guidewire 11 by the PA signal. A case will be described in which the ultrasonic imaging device operates a robot arm 90 so that the ultrasonic probe 20 tracks the tip position of the guidewire.

[0029] FIG. 4 is a process flow diagram of the ultrasonic imaging device. In step S401, the ultrasonic imaging apparatus determines whether or not the assist operation for tracking of the ultrasonic probe 20 is valid, and if it is invalid (No in S401), ends the process. If it is valid (Yes in S401), the process proceeds to step S402.

[0030] In step S402, the ultrasound imaging device performs reflected ultrasound imaging (hereinafter referred to as imaging mode) using the ultrasound probe 20. In this imaging mode, ultrasound imaging is performed in the same manner as in a conventional ultrasound imaging device.

[0031] In detail, ultrasonic waves are transmitted from the transmission unit 31 by the ultrasonic probe 20, and the reflected waves of the transmitted ultrasonic waves reflected from tissue inside the subject are received by the ultrasonic probe 20. The reception unit 32 performs processing such as phasing and addition on the reception signals received for each frame from the ultrasonic probe 20, and sends the received signals to the signal processing unit 41. The reflected ultrasonic signal processing unit 411 creates an ultrasonic image, for example, a B-mode image, using the frame signals from the reception unit 32, and passes it to the display image forming unit 43, which forms an image to be displayed on the display unit 43.

[0032] In this imaging mode, when the ultrasound probe 20 is a 1D array probe, information on the reflected wave intensity in the direction of the array contact and in the depth direction can be obtained, so that information on the reflected wave intensity can be used to obtain two-dimensional information on the reflected wave intensity. Furthermore, when a 2D array probe is used as the ultrasonic probe 20, information corresponding to the intensity of three-dimensional reflected waves combining the probe surface and the depth direction can be obtained in a single imaging mode.

[0033] In step S403, the ultrasound imaging device receives a PA signal from the ultrasound probe 20 (hereinafter referred to as a PA reception mode). In this PA reception mode, the PA signal from the PA signal generation source 13 is monitored while the catheter is inserted inside the body of the subject, for example, into a blood vessel.

[0034] Specifically, in the PA reception mode, the operation of the transmission unit 31 is temporarily stopped, and the light generating unit 15 is activated to irradiate pulsed laser light from the light generating unit 15. The light emitted by the light generating unit 15 is irradiated to the PA signal generating source 13 via the optical fiber 12 of the guide wire 11 inserted inside the body. This irradiated light causes a PA signal (ultrasound wave) to be generated from the photoacoustic material constituting the PA signal generating source 13, and the PA signal (ultrasound wave) is detected by each element of the ultrasound probe 20.

[0035] The PA signal analysis unit 412 creates signal data synchronized with the light irradiation from the light generation unit 15 from the PA signal received by the ultrasonic probe 20, and passes the signal data to the relative position detection unit 421 of the PA sound source position detection unit 42. For synchronization of the received PA signal, the timing of light irradiation may be obtained from a trigger signal output to the PA signal analysis unit 412 when the light generation unit 15 irradiates light, or the timing of light irradiation may be estimated from the PA signal received by each element of the ultrasonic probe 20. Furthermore, when using an ultrasonic generating device 10 that generates ultrasonic waves using a piezoelectric element, the timing at which a signal was generated can be estimated by transmitting ultrasonic waves using the transmitting unit 31, and synchronization with an external signal source can be achieved using a trigger signal, as in the case of using a PA signal generating source.

[0036] In step S404, the PA sound source position detection unit 42 detects the position of the PA signal source 13 based on the PA signal passed from the PA signal analysis unit 412 and the absolute position (three-dimensional) and posture (direction) information of the ultrasound probe 20 passed from the robot arm 90.

[0037] More specifically, in the PA sound source position detector 42 , first, the relative position detector 421 detects the relative position of the PA signal generator 13 with respect to the ultrasound probe 20 based on the PA signal passed from the PA signal analyzer 412 . Next, the relative velocity is detected from the time change of the detected relative position by the relative velocity measuring unit 422, and the probe velocity measuring unit 425 detects the velocity and angular velocity of the ultrasonic probe 20 from the time change of the information on the absolute position and attitude (direction) of the ultrasonic probe 20 passed from the robot arm 90. Furthermore, the relative speed measurement unit 422 may measure the relative speed using the Doppler effect that occurs in the received PA signal.

[0038] Then, absolute position detection section 424 detects the absolute position of PA signal generation source 13 based on these values. The detected absolute position of PA signal source 13 is filtered by position filter 423 to reduce detection errors.

[0039] In step S405, the display image forming unit 43 forms display content and displays it on the display unit .

[0040] In detail, the display image forming unit 43 forms a display image that allows the surgeon to grasp the three-dimensional positional relationship of the PA signal source 13 using the reflected ultrasound image formed by the reflected ultrasound signal processing unit 411, the position of the PA signal source 13 detected by the PA sound source position detection unit 42, and a 3D volume image of the anatomical structure of the subject 80 acquired in advance and recorded in the pre-acquired 3D anatomical information 44, and displays the display image on the display unit 34. Specific examples of the display on the display unit 34 will be described later with reference to FIGS.

[0041] In step S406, the motion planning unit 45 plans the motion of the robot arm 90 using the pre-acquired 3D anatomical information 44 of the subject 80, the position and coordinates of the probe obtained from the robot arm 90, and the reflected ultrasound image and PA sound source position obtained by the ultrasound imaging device, and instructs the robot arm 90 on the operation position.

[0042] For example, the motion planning unit 45 moves the robot arm 90 so that the PA signal generation source 13 is positioned at a predetermined position on the reflected ultrasound image in step S402 and tracks the PA signal generation source 13. This allows the ultrasound probe 20 to continue receiving the PA signal (ultrasound) generated from the PA signal generation source 13. A more specific tracking example of the ultrasonic probe 20 will be described later with reference to Figs. 13A and 14A.

[0043] Then, the process returns to step S401, and steps S402 to S406 are repeated. Note that the imaging mode (step S402) and the PA reception mode (step S403) are not limited to those shown in the flow diagram of Fig. 4. For example, the imaging mode may be executed four times and the PA reception mode may be executed once, or the imaging mode and the PA reception mode may be executed once each.

[0044] Hereinafter, the process of the absolute position detection unit 424 in step S404 in FIG. 4 will be described in detail.

[0045] 5 is a diagram for explaining the coordinate system of the ultrasonic probe 20 for the following explanation. In the coordinate system, the array arrangement direction of the acoustic element array 21 of the ultrasonic probe 20, which is a 1D array probe, is the long axis 22, an axis parallel to the array receiving surface and perpendicular to the long axis 22 is the short axis 23, an axis perpendicular to the array receiving surface is the depth axis 24, and the center of the long axis and short axis directions on the surface of the acoustic element array 21 is the origin of the coordinate system. When the ultrasonic probe 20 is a 2D array probe having multiple arrays arranged in two dimensions, the orientations of the major axis 22 and the minor axis 23 may be determined arbitrarily.

[0046] First, referring to Figure 6, we will explain a method for detecting the absolute position of the PA signal source 13 from the relationship between the position and posture when the ultrasonic probe 20 moves in a translational manner in the direction of the short axis 23, that is, the position change (translation) of the ultrasonic probe 20 relative to the PA signal source 13.

[0047] 6, when the ultrasonic probe 20, which is a 1D array probe, translates 711 from 712B to 712A in the direction of the short axis 23, the absolute positions of the ultrasonic probe 20 at 712A and 712B can be acquired by the robot arm 90. In addition, since the PA signal analysis unit 412 can measure the arrival time of the PA signal of the ultrasonic probe 20 from the light emission of the light generation unit 15, the distances 713A and 713B from 712A and 712B to the PA signal generation source 13 can be obtained.

[0048] Therefore, since position 715 of PA signal source 13 is the intersection of arc 714A of distance 713A and arc 714B of distance 713B, the absolute position can be calculated from the absolute positions of ultrasound probe 20 at 712A and 712B, respectively, and distances 713A and 713B to PA signal source 13.

[0049] For details, the distance to the PA signal source 13 is 713A (l a ) and 713B(l b ) is the arrival time of the RA signal before and after the movement. a, t b Then, it can be obtained by equation (1).

number

[0050] If the ultrasonic probe 20 is translated from 712B to 712A at a speed v and a short time Δt, the position 715 (y PA , z PA ) is calculated using equation (2).

number

[0051] Although the PA signal source 13 is assumed to be stationary in the above description, it may be moving. Therefore, an error occurs in the absolute position of the PA signal source 13 determined due to the movement of the PA signal source 13, but this error is reduced by the position filter 423.

[0052] Next, referring to Figure 7, we will explain a method for detecting the absolute position of the PA signal source 13 from the relationship between the position and posture when the ultrasound probe 20 rotates around the depth axis 24, i.e., the change in the rotational position of the imaging plane of the ultrasound probe 20.

[0053] As shown in Figure 7, when the ultrasonic probe 20, which is a 1D array probe, rotates from 722B to 722A around the depth axis 24 in the rotation direction 721, the PA signal analysis unit 412 can acquire the longitudinal positions 723B and 723A of each PA signal source 13, and the robot arm 90 can acquire the absolute positions of the longitudinal positions 723B and 723A of each PA signal source 13.

[0054] Therefore, since position 725 of PA signal source 13 is the intersection of lines 724A and 724B, the absolute position of position 725 of PA signal source 13 can be calculated from the absolute positions of 723A and 723B.

[0055] In detail, the position change x in the direction of the long axis 22 of the PA signal source 13 before and after the movement PA , the angular velocity of the rotation 721 around the depth axis 24 is ω, the arrival time of the PA signal is t, and the sound speed is c, then the position 725 (y PA , z PA ) is calculated using equation (3).

number

[0056] In the above-mentioned method of detecting the absolute position of the PA signal source 13 from the positional changes of the ultrasonic probe 20 and the PA signal source 13, or the rotational position change of the imaging surface of the ultrasonic probe 20, the absolute position of the PA signal source 13 is detected using the positions before and after the movement, but if the time difference before and after the movement is short, the absolute position of the PA signal source 13 is detected using the speed of the ultrasonic probe 20.

[0057] A process flow for detecting the absolute position of the PA signal generating source 13 using the velocity of the ultrasonic probe 20 in the PA sound source position detector 42 will be described with reference to FIG. In this flow, the translation and rotation are extracted from the movement of the ultrasound probe 20 to detect the absolute position of the PA signal generation source 13 .

[0058] In step S801, the probe speed measurement unit 425 measures the moving speed and angular velocity of the ultrasonic probe 20. Here, the moving speed and angular velocity of the ultrasonic probe 20 may be acquired from the robot arm 90.

[0059] In step S802, the absolute position detection unit 424 extracts the translational velocity in the direction of the short axis 23 of the ultrasonic probe 20 and the rotational velocity component around the depth axis 24 of the ultrasonic probe 20 from the measured movement velocity and angular velocity of the ultrasonic probe 20.

[0060] In step S803, the absolute position detection unit 424 determines whether the extracted translational speed in the direction of the minor axis 23 is equal to 0, and if it is equal to 0 ("=0" in S803), proceeds to step S805. If it is not equal to 0 ("≠0" in S803), proceeds to step S804.

[0061] In step S804, the absolute position detection unit 424 detects the absolute position of the PA signal source 13 from the translational speed of the PA signal source 13 in the direction of the minor axis 23 of the ultrasound probe 20, similar to the method described with reference to Fig. 6. In detail, the absolute position of the PA signal source 13 is detected assuming that the PA signal source 13 is located at a point equal to the distance measured by the PA signal analysis unit 412 from two points separated by the speed measurement time at that translational speed. Then, the process proceeds to step S805.

[0062] In step S805, the absolute position detection unit 424 determines whether the extracted rotation speed of the ultrasound probe 20 around the depth axis 24 is equal to 0, and if it is equal to 0 ("=0" in S805), proceeds to step S807. If it is not equal to 0 ("≠0" in S805), proceeds to step S806.

[0063] In step S806, the absolute position detection unit 424 detects the absolute position of the PA signal source 13 using the rotation speed of the ultrasound probe 20 about the depth axis 24, similar to the method described with reference to Fig. 7. In detail, the absolute position of the PA signal source 13 is detected assuming that the PA signal source 13 is located at the intersection of normal lines at two points separated by the measurement time of the rotation speed. Then, the process proceeds to step S807.

[0064] In step S807, the position filter 423 performs filtering processing on at least one of the absolute positions of the PA signal generating source 13 obtained in steps S804 and S806, thereby improving the accuracy of position detection.

[0065] In the above, a case where a 1D array probe is used as the ultrasonic probe 20 has been described, but a case where a 2D array probe is used as the ultrasonic probe 20 will be described. In this case, the three-dimensional position of the PA signal source 13 relative to the ultrasonic probe 20 can be obtained without obtaining the three-dimensional position of the PA signal source 13 by translating or rotating the ultrasonic probe 20, so the three-dimensional absolute position of the PA signal source 13 can be detected by adding the obtained three-dimensional relative position to the three-dimensional absolute position and attitude of the ultrasonic probe 20 notified by the robot arm 90.

[0066] In the above-described ultrasonic imaging device and treatment support system, the robot arm 90 operates the ultrasonic probe 20, and the three-dimensional absolute position of the PA signal generator 13 is detected based on the absolute position of the ultrasonic probe 20 detected by the robot arm 90. However, a configuration may be provided in which a probe position / orientation sensor 91 is provided to detect the absolute position of the ultrasonic probe 20. Also, the ultrasonic probe 20 may be operated by the robot arm 90, and the absolute position of the ultrasonic probe 20 may be detected by the probe position / orientation sensor 91.

[0067] The probe position / orientation sensor 91 may be built into the ultrasonic probe 20 or may be configured as a separate body. The probe position / orientation sensor 91 may be, for example, a combination of a positioning sensor such as a geomagnetic sensor, an acceleration sensor, and a gyro sensor, or may use an external camera, and is not limited to any sensor that can measure the position and orientation of the probe.

[0068] FIG. 9 is a block diagram showing the configuration of an ultrasonic imaging apparatus and a treatment support system that detects the absolute position of an ultrasonic probe 20 by a probe position / orientation sensor 91. As shown in FIG. The ultrasonic imaging device in FIG. 9 differs from the ultrasonic imaging device described in FIG. 3 in that it is provided with a probe position / orientation sensor 91, except for the motion planning unit 45 of the robot arm 90, and otherwise has the same configuration.

[0069] The probe position / orientation sensor 91 periodically detects the absolute position and orientation of the ultrasonic probe 20 and notifies the control unit 40 of the same. The probe velocity measurement unit 425 , in place of the robot arm 90 , detects the velocity and angular velocity of the ultrasonic probe 20 from the time change in information on the absolute position and attitude (direction) of the ultrasonic probe 20 passed from the probe position / attitude sensor 91 . The display image forming unit 43 performs the same process as in the case of detecting the three-dimensional absolute position of the PA signal generation source 13 based on the absolute position of the ultrasound probe 20 detected by the robot arm 90 .

[0070] FIG. 10 is a process flow diagram of the ultrasonic imaging device. The processing flow of the ultrasonic imaging apparatus described in FIG. 4 differs in that the processing of step S406 in which the motion planning unit instructs the robot arm to perform a motion is omitted, and step S404 is replaced by step S407.

[0071] In step S407, the PA sound source position detection unit 42 detects the position of the PA signal source 13 based on the PA signal passed from the PA signal analysis unit 412 and the absolute position (three-dimensional) and posture (direction) information of the ultrasonic probe 20 passed from the probe position / posture sensor 91. This makes it possible to detect the three-dimensional absolute position of the PA signal generation source 13 even when the ultrasound probe 20 is manually operated.

[0072] Although the embodiments of the position detection of the PA signal generating source 13 have been described above, the embodiments can be appropriately combined as long as there is no technical contradiction, and such combinations are also included in the present invention.

[0073] Here, the display image displayed on the display unit 34 in step S405 in FIG. 4 and FIG. 10 will be described. FIG. 11 is a diagram showing an example of a display image 51 displayed on the display unit 34. As shown in FIG.

[0074] 11, a display image 51 displayed on the display unit 34 displays the position of the PA signal generation source 13 in a blood vessel 82 of a subject 80, and has an image 511 displaying a longitudinal cross section of the blood vessel 82, and an image 512 displaying a transverse cross section of the blood vessel 82. Images 511 and 512 display the body and blood vessel 82 of the subject 80, and a position display 519 and a trajectory display 518 of the PA signal generation source 13. All or some of these display elements may be displayed.

[0075] In addition, images 511 and 512 may be displayed using a display image captured by the ultrasound imaging unit 30 and formed by the display image forming unit 43, or may be formed using CG (Computer Graphics) using the PA signal source 13 detected by the PA sound source position detection unit 42 and pre-acquired 3D anatomical information 44 of the subject 80 acquired in advance, so that the display image forming unit 43 displays information about the anatomical structure outside the imaging area of ​​the ultrasound imaging unit 30.

[0076] The contents of the image displayed on the display unit 34 can include any image processing to assist in surgery, and can include displays that highlight points of interest such as blood vessels and lesions present in the ultrasound image or CG, and displays that indicate the location of lesions outside the screen.

[0077] In the image formation by CG, in order to detect and align the position of the pre-acquired 3D anatomical information 44 of the subject 80 on the absolute coordinates detected by the robot arm 90, feature points in the ultrasound image formed by the display image forming unit 43 are compared with feature points in the pre-acquired 3D anatomical information 44 to detect matching points. For example, blood vessel branches and bones can be used as feature points, and other feature points may also be used.

[0078] In addition, in forming an image by the CG, a display image may be formed directly using the pre-acquired 3D anatomical information 44, or a reflected ultrasound image formed during surgery may be used to detect deformations of the pre-acquired anatomical structure and the anatomical structure during surgery, and a display image may be formed using information obtained by deforming the pre-acquired 3D anatomical information 44 so that the pre-acquired 3D anatomical information 44 more closely matches the structure during surgery. For example, a process may be considered in which a vascular wall in a reflected ultrasound image formed during surgery is recognized, and the vascular wall included in the pre-acquired 3D anatomical information 44 is deformed so that the vascular wall has the same shape as the vascular wall in the reflected ultrasound image.

[0079] The content of the image displayed on the display unit 34 is not limited to the screen displaying images from two directions as exemplified in Fig. 11, and an image from one direction may be displayed, or an image may be displayed by third angle projection, as long as it can present the three-dimensional positional relationship between the lesion and the PA signal generating source 13. In addition, images from any viewpoint may be displayed on four or more screens, and the display method and the viewpoint of the displayed image are not limited.

[0080] FIG. 12 is a diagram showing an example of a display image 52 according to the third angle projection method. Display image 52 is made up of images from three directions: a side view 521 of a blood vessel, a cross-section view 522 of the blood vessel, and a top view 523 of the blood vessel. Reference numeral 528 in FIG. 12 indicates the trajectory of the PA signal generating source 13 , and reference numeral 529 indicates the position of the PA signal generating source 13 .

[0081] Next, the motion instructions for the robot arm planned by the motion planning unit 45 in step S406 in FIG. 4 will be described in detail. An example of a movement plan for the robot arm 90 by the motion planning unit 45 will be described with reference to Figs. 13A and 14A.

[0082] In FIG. 13A, when the PA signal source 13 moves from a pre-movement position 613 to a post-movement position 614, the robot arm 90 moves the ultrasound probe 20 so that the imaging area of ​​the ultrasound probe 20 slices the blood vessel 82 in a cross section, and tracks the PA signal source 13, moving from a pre-movement position 611 to a post-movement position 612.

[0083] FIG. 13B shows an ultrasound image when the robot arm 90 moves the ultrasound probe 20 as shown in FIG. 13A. In the ultrasound image, blood vessel 82 of subject 80 and PA signal generation source 13 (position 615) are depicted, making it easy to grasp position 615 of PA signal generation source 13 within blood vessel 82 of subject 80.

[0084] In FIG. 14A , the robot arm 90 tracks positions 623 and 624 before and after the movement of the PA signal source 13, and moves the ultrasound probe 20 from position 621 before the movement to position 622 after the movement so that the imaging region of the ultrasound probe 20 includes the long axis cross-section of the blood vessel 82.

[0085] FIG. 14B shows an ultrasound image when the robot arm 90 moves the ultrasound probe 20 as shown in FIG. 14A. The ultrasound image depicts blood vessel 82 of subject 80 and PA signal source 13 (position 625), and when an obstruction 83 is present in blood vessel 82 of subject 80, it makes it easier to grasp the positional relationship between the obstruction 83 and PA signal source 13 (position 625).

[0086] However, the manner of moving the ultrasonic probe 20 is not limited to the manner of movement shown in FIGS. 13A and 14A as long as it is possible to acquire the reflected ultrasonic signals and PA signals necessary to form the display image in the display image formation in step S405 of FIGS. 4 and 10.

[0087] For example, as described in FIG. 8, position detection is possible when the speed of translation 711 described in FIG. 6 and the speed of rotation 721 described in FIG. 7 are both non-zero. Therefore, the velocity v of the translation 711 of the ultrasonic probe 20 eand the angular velocity of the rotation ω d By the robot arm 90 swinging the ultrasonic probe 20 so that equation (4) holds, the translation 711 and the rotation 721 do not become 0 at the same time. This allows the position detection of the PA signal generation source 13 to be constantly continued. Here, v0 and ω0 are constants for adjusting the speed, and T is the swing period.

number

[0088] The above describes embodiments of the ultrasound imaging apparatus and catheter treatment support system of the present invention. However, the embodiments can be combined as appropriate as long as there is no technical contradiction, and such combinations are also encompassed by the present invention. [Explanation of symbols]

[0089] 10 Ultrasonic generating device 11 Guide wire (biological insertion device) 12 Optical Fiber 13 PA signal generator (photoacoustic ultrasound generator) (beacon) 20 Ultrasonic probe 30 Ultrasound imaging unit 31 Transmitter 32 Receiving section 33 Input section 34 Display section 35 Memory 40 Control section 41 Signal Processing Section 411 Reflected Ultrasound Signal Processing Unit 412 PA signal analysis section (ultrasonic signal analysis section) 42 PA sound source position detection section 421 Relative position detection unit 422 Relative Velocity Measurement Unit 423 Positional Filter 424 Absolute position detection unit (beacon position acquisition unit) 425 Probe velocity measurement unit (probe position acquisition unit) 43 Display image forming section 44 3D anatomical information 45 Motion Planning Division 51 Display images 52 Display images 80 Subjects 90 Robot Arm 91 Probe position / orientation sensor (probe position acquisition unit) 100 Medical Support System

Claims

1. an ultrasonic probe that irradiates an ultrasonic wave onto a subject and receives a reflected wave of the ultrasonic wave, and also receives ultrasonic waves from a beacon inserted inside the subject; a probe position acquisition unit that acquires a three-dimensional absolute position and orientation of the ultrasonic probe; a beacon position acquisition unit that obtains a distance between the beacon and the ultrasonic probe from an output signal of the ultrasonic probe that has received ultrasonic waves from the beacon, before and after at least one of a three-dimensional absolute position and an orientation of the ultrasonic probe changes, obtains a position that satisfies both of the obtained distances as a relative position of the beacon with respect to the ultrasonic probe, and obtains a three-dimensional absolute position of the beacon from the obtained relative position and the three-dimensional absolute position of the ultrasonic probe; A display image forming unit; The display image forming unit generates a display image that displays the three-dimensional absolute position of the beacon obtained by the beacon position acquisition unit on an image that displays the structure of the subject that has been previously imaged or formed, and displays the display image on a display unit.

2. 2. The ultrasonic imaging apparatus according to claim 1, further comprising a 3D anatomical information recording unit and a reflected ultrasonic signal processing unit, a 3D volume image of an anatomical structure of the subject is recorded in the anatomical information recording unit; the reflected ultrasonic signal processing unit generates an ultrasonic image of the subject based on the reflected waves of the ultrasonic waves received by the ultrasonic probe; The display image forming unit compares feature points of the 3D volume image with feature points of the ultrasound image, aligns the 3D volume image to a three-dimensional absolute position of the ultrasound probe, and generates an image displaying the structure of the subject from the 3D volume image after alignment.

3. 3. The ultrasonic imaging apparatus according to claim 2, The display image forming unit detects deformation of the anatomical structure of the subject during surgery relative to the anatomical structure of the 3D volume image using the ultrasound image as a reference, deforms the 3D volume image so that the anatomical structure of the 3D volume image conforms to the anatomical structure of the subject during surgery, and generates an image displaying the structure of the subject using the deformed 3D volume image.

4. 2. The ultrasonic imaging apparatus according to claim 1, The ultrasonic imaging device according to claim 1, wherein the beacon position acquisition unit performs a filter process to reduce an error when determining the relative position of the beacon with respect to the ultrasonic probe.

5. 5. The ultrasonic imaging apparatus according to claim 4, The ultrasonic imaging apparatus according to claim 1, wherein the beacon position acquisition unit models and filters an error when determining the relative position of the beacon with respect to the ultrasonic probe.

6. 6. The ultrasonic imaging apparatus according to claim 5, The beacon position acquisition unit estimates a statistically most probable state from an error model, the position and position change of the beacon relative to the ultrasound probe, and the position and position change of the ultrasound probe, and performs filtering processing.

7. 2. The ultrasonic imaging apparatus according to claim 1, A robot arm for manipulating the ultrasonic probe is provided. The ultrasonic imaging device according to claim 1, wherein the probe position acquisition unit sets operation position information of the robot arm as a three-dimensional absolute position of the ultrasonic probe.

8. 8. The ultrasonic imaging apparatus according to claim 7, The ultrasonic imaging device according to claim 1, wherein the robot arm tracks the beacon so that the beacon is recognized in an image of a cross-section of a blood vessel captured by the ultrasonic probe.

9. 8. The ultrasonic imaging apparatus according to claim 7, The ultrasonic imaging device according to claim 1, wherein the robot arm tracks the beacon so that the beacon is recognized in an image of a longitudinal cross section of a blood vessel captured by the ultrasonic probe.

10. 8. The ultrasonic imaging apparatus according to claim 7, The robot arm tracks the beacon so that the beacon position acquisition unit determines the relative position of the beacon with respect to the ultrasonic probe based on at least one of a positional change of the beacon and the ultrasonic probe on the imaging plane of the ultrasonic probe and a rotational position change of the imaging plane of the ultrasonic probe relative to the beacon.

11. An ultrasonic imaging device according to any one of claims 1 to 10, A medical support system comprising: a guide wire having the beacon at a tip thereof.

Images