Information processing apparatus, ultrasonic diagnostic apparatus, and program

Abstract

To achieve accurate puncture easily in puncture support using a robot arm.SOLUTION: An information processing device includes a setting unit and a puncture control unit. The setting unit sets control information on a puncture needle held by a second robot arm on the basis of puncture information input to an ultrasonic image collected by an ultrasonic probe held by a first robot arm. The puncture control unit controls the movement of the second robot arm that holds the puncture needle on the basis of the control information.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The embodiments disclosed in this specification and the like relate to an information processing apparatus, an ultrasonic diagnostic apparatus, and a program.

Background Art

[0002] Conventionally, in order to easily observe the state inside a subject, an ultrasonic diagnostic apparatus that transmits ultrasonic waves from the body surface into the body and displays an ultrasonic image based on the reflected waves is widely used. For example, since an ultrasonic diagnostic apparatus can display an ultrasonic image on a monitor in substantially real time, it is used when puncture is performed, such as in a biopsy, radiofrequency ablation (RFA), or treatment using irreversible electroporation (IRE).

[0003] In such a puncture using a puncture needle, a puncture plan is made based on images such as CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) taken in advance, and the puncture needle is inserted while guiding with an ultrasonic image during treatment. Here, when performing puncture while guiding with an ultrasonic image, a needle adapter that fixes the angle of the puncture needle may be used so that the puncture needle is depicted in the ultrasonic image. By using a needle adapter, the puncture needle will be on the same plane as the ultrasonic image, which is useful when it is difficult to visually confirm the needle during puncture.

[0004] Also, in guiding puncture with an ultrasonic image, since it may be difficult to depict the affected area with ultrasonic waves, a Fusion function may be used to align the images such as CT and MRI used to create the puncture plan with the real-time ultrasonic image and make the angles and positions of both planes coincide for display.

[0005] In recent years, a method of simultaneously inserting multiple needles has been spreading in punctures for therapeutic purposes. For example, when the affected area is large and cannot be covered by a single treatment, or when multiple needles are required in principle depending on the treatment method, multiple puncture needles are used. As an example of puncture using multiple puncture needles, for example, irreversible electroporation, also called Nanoknife (registered trademark), can be mentioned. In Nanoknife, since the needles are different for each electrode, at least two simultaneous punctures are required, and in some cases, six simultaneous punctures are performed. In Nanoknife, after puncture, treatment is performed on the target between the electrodes by irradiating a pulse wave for about 30 minutes to 1 hour. Such multiple punctures are difficult to perform without prior planning due to the complexity of the procedure, and a puncture plan is made in advance using CT, MRI, and ultrasonic images. The puncture is performed while following this puncture plan, guided by real-time ultrasonic images.

[0006] In addition, research on robotic ultrasonic diagnosis using a robotic arm to perform ultrasonic scanning has been progressing, and research on using a robotic arm for puncture treatment under ultrasonic guidance has also been progressing. When such a robotic arm is used for puncture treatment, it is assumed that a treatment plan is made in advance on a three-dimensional image such as CT or MRI, and the robotic arm guides the puncture at the planned puncture position and puncture path.

Prior Art Documents

Patent Documents

[0007]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0008] One of the problems that the embodiments disclosed in this specification etc. seek to solve is to easily achieve accurate puncture in puncture support using a robotic arm. However, the problems solved by the embodiments disclosed in this specification etc. are not limited to the above problem. The problems corresponding to the respective effects by each configuration shown in the embodiments described later can also be positioned as other problems solved by the embodiments disclosed in this specification etc.

Means for Solving the Problems

[0009] The information processing apparatus of the embodiment includes a setting unit and a puncture control unit. The setting unit sets control information for a puncture needle held by a second robotic arm based on puncture information input with respect to an ultrasonic image collected by an ultrasonic probe held by a first robotic arm. The puncture control unit controls the movement of the second robotic arm holding the puncture needle based on the control information.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Best Mode for Carrying Out the Invention

[0011] Hereinafter, with reference to the accompanying drawings, embodiments of an information processing apparatus, an ultrasonic diagnostic apparatus, and a program according to the present application will be described in detail. Note that the information processing apparatus, the ultrasonic diagnostic apparatus, and the program according to the present application are not limited to the embodiments shown below. In the following description, the same components are denoted by common reference numerals, and redundant descriptions are omitted.

[0012] (First Embodiment) First, a puncture support system including an information processing apparatus according to the present application will be described. The puncture support system includes an ultrasonic diagnostic apparatus and a robot arm apparatus, and adjusts a puncture needle held by the robot arm apparatus under the control of the information processing apparatus. FIG. 1 is a diagram schematically showing an example of a puncture support system 1000 according to the first embodiment. For example, as shown in FIG. 1, the puncture support system 1000 includes an ultrasonic diagnostic apparatus (US apparatus) 100 and a robot arm apparatus 200, and supports puncture of a subject.

[0013] The ultrasonic diagnostic apparatus 100 has an ultrasonic probe 12, and generates an ultrasonic image based on ultrasonic waves transmitted and received by the ultrasonic probe 12. For example, the ultrasonic diagnostic apparatus 100 generates an ultrasonic image of a puncture target site of a subject scanned by the ultrasonic probe 12.

[0014] The robotic arm device 200 has, for example, a robotic arm control device 21, an arm 22a, an arm 22b, and an arm 22c. The arm 22a holds the puncture needle 31 and changes its position or moves the puncture needle 31 in the puncturing direction (the longitudinal direction of the puncture needle 31) based on the control by the robotic arm control device 21. The arm 22b holds the puncture needle 32 and changes its position or moves the puncture needle 31 in the puncturing direction (the longitudinal direction of the puncture needle 31) based on the control by the robotic arm control device 21. The arm 22c holds the ultrasonic probe 12 and executes an ultrasonic scan on the subject by changing its position based on the control by the robotic arm control device 21.

[0015] The robotic arm control device 21 adjusts the positions of the puncture needle 31, the puncture needle 32, and the ultrasonic probe 12 with respect to the subject by moving the positions of the arm 22a, the arm 22b, and the arm 22c. Here, the robotic arm control device 21 can control the positions of the arm 22a, the arm 22b, and the arm 22c based on the control signal received from the information processing device. That is, the robotic arm control device 21 moves the arm 22a, the arm 22b, and the arm 22c based on the amount of movement of the arm included in the control signal received from the information processing device. Note that the positional relationship between the arms 22a, 22b, 22c and the subject can be grasped based on the ultrasonic image collected by the ultrasonic probe 12 held by the arm 22c.

[0016] In FIG. 1, a robotic arm device 200 having three arms, i.e., the arm 22a, the arm 22b, and the arm 22c, is shown. However, the embodiment is not limited to this, and the number of arms is arbitrary as long as it has an arm that holds a puncture needle and an arm that holds the ultrasonic probe 12. For example, it may have one or three or more arms that hold a puncture needle, or may have two or more arms that hold the ultrasonic probe 12.

[0017] The information processing device transmits a control signal to the robotic arm device 200 to adjust the positions of the arms 22a, 22b, and 22c of the robotic arm device 200, thereby assisting with the puncture of the subject. Here, the information processing device according to the present application can easily achieve accurate puncture in puncture assistance using a robotic arm.

[0018] As described above, when performing a puncture, a puncture plan is established based on three-dimensional images such as pre-taken CT (Computed Tomography) or MRI (Magnetic Resonance Imaging), and the puncture is executed along this puncture plan. However, since the body position and respiratory phase are different between CT or MRI imaging and puncture, at the actual time of puncture, the puncture position and puncture path are almost always determined by confirming the body position of the subject immediately before treatment with an image. Therefore, even in puncture assistance using a robotic arm, it is required to easily determine the puncture site in the body position of the subject immediately before treatment and accurately perform the puncture.

[0019] Furthermore, if accurate treatment with multiple puncture needles, which requires a very high level of skill in the currently performed manual puncture method, can be achieved, the application range of puncture treatment under ultrasonic guidance can be significantly expanded, and the treatment quality can also be improved. For example, in liver cancer, multiple simultaneous punctures are required when the tumor is large, in pancreatic cancer, there are nano knife treatment and cryotherapy, and in prostate cancer, there is systematic biopsy, and puncture with multiple puncture needles is required.

[0020] Therefore, the information processing device according to the present application enables easy achievement of accurate puncture by controlling the robotic arm using an ultrasonic image collected in the body position of the subject immediately before puncture in puncture assistance using a robotic arm.

[0021] Hereinafter, the details of the device configuration according to the first embodiment of the present application will be described. In the first embodiment, the case where the information processing device according to the present application is included in an ultrasonic diagnostic device will be described as an example, but the embodiment is not limited thereto. That is, various processes executed by the ultrasonic diagnostic device described below may be executed by any device in the above-described puncture support system 1000. Further, an information processing device may be further connected to the above-described puncture support system 1000, and the connected information processing device may execute various processes executed by the ultrasonic diagnostic device described below.

[0022] FIG. 2 is a block diagram showing an example of the configuration of an ultrasonic diagnostic device 100 according to the first embodiment. As shown in FIG. 2, the ultrasonic diagnostic device 100 according to the present embodiment includes an ultrasonic probe 12, a monitor 13, an input interface 14, and a device body 15, and the ultrasonic probe 12, the monitor 13, and the input interface 14 are communicably connected to the device body 15. And the ultrasonic diagnostic device 100 according to the present embodiment is further communicably connected to a robot arm control device 21. Note that the robot arm control device 21 may be built in the device body 15.

[0023] The robot arm control device 21 has a communication interface and a processing circuit.

[0024] The communication interface is connected to the processing circuit and transmits and receives various information to and from the ultrasonic diagnostic device 100. For example, the communication interface receives a control signal from the ultrasonic diagnostic device 100 and outputs the received control signal to the processing circuit. Further, the communication interface transmits information such as pressure detected by the arms 22a to 22c to the ultrasonic diagnostic device 100.

[0025] The processing circuit controls the driving of the arms 22a, 22b, and 22c by sending control signals to the driving mechanisms of the arms 22a, 22b, and 22c. Specifically, the processing circuit moves each arm by sending a control signal based on the movement information (movement direction and movement amount) of the arm received from the ultrasonic diagnostic apparatus 100 to the driving mechanism. Note that the processing circuit is a processor that reads and executes a program corresponding to each process from a memory (not shown) to realize the functions corresponding to the respective programs.

[0026] The arm 22a holds the puncture needle 31 and moves the puncture needle 31 under the control of the robot arm control device 21. Specifically, the arm 22a has a holding part that holds the puncture needle 31 and a driving mechanism for driving the arm. The arm 22a moves the puncture needle 31 held at the tip side to a desired position by the movement of a plurality of movable parts provided with a driving mechanism such as a motor or an actuator in response to the control received from the robot arm control device 21. Here, the arm 22a is configured to be able to dispose the puncture needle 31 at an arbitrary three-dimensional position with respect to the subject.

[0027] The arm 22b holds the puncture needle 32 and moves the puncture needle 32 under the control of the robot arm control device 21. Specifically, the arm 22b has a holding part that holds the puncture needle 32 and a driving mechanism for driving the arm. The arm 22b moves the puncture needle 32 held at the tip side to a desired position by the movement of a plurality of movable parts provided with a driving mechanism such as a motor or an actuator in response to the control received from the robot arm control device 21. Here, the arm 22b is configured to be able to dispose the puncture needle 32 at an arbitrary three-dimensional position with respect to the subject.

[0028] Here, the arm 22a and the arm 22b can also move the puncture needles 31 and 32 in the longitudinal direction of the needles. That is, the arm 22a and the arm 22b are configured to automatically insert and remove the puncture needles under the control of the robot arm control device 21. For example, the arm 22a has a rubber roller or the like on the tip side of the arm, is connected to a drive mechanism such as a motor or an actuator, holds the puncture needle 31 so as to sandwich it between a plurality of rollers, and moves the puncture needle 31 in the longitudinal direction of the needle by the rotation of the plurality of rollers.

[0029] Further, the arm 22a further has a pressure sensor to detect the pressure applied to the puncture needle 31. Then, the arm 22a transmits the detected pressure information to the robot arm control device 21. Note that the arm 22a can also hold the puncture needle 31 so that it can be manually moved.

[0030] Note that also in the arm 22b, by having a rubber roller or the like on the tip side of the arm in the same manner as the example of the arm 22a described above, the puncture needle can be automatically inserted and removed. Further, the arm 22b has a pressure sensor in the same manner as the arm 22a, detects the pressure applied to the puncture needle 32, and transmits the detected pressure information to the robot arm control device 21. Also, the arm 22b can hold the puncture needle 32 so that it can be manually moved.

[0031] The arm 22c holds the ultrasonic probe 12 and moves the ultrasonic probe 12 under the control of the robot arm control device 21. Specifically, the arm 22c has a holding portion that holds the ultrasonic probe 12 and a drive mechanism for driving the arm. The arm 22c moves the ultrasonic probe 12 held on the tip side to a desired position by the movement of a plurality of movable parts provided with a drive mechanism such as a motor or an actuator in response to the control received from the robot arm control device 21. Here, the arm 22c is configured to be able to arrange the ultrasonic probe 12 at an arbitrary three-dimensional position with respect to the subject.

[0032] The puncture needles 31 and 32 are, for example, electromagnetic needles that generate radio waves, and are connected to a treatment device that controls the output of the radio waves generated by the puncture needles 31 and 32. This treatment device can monitor the temperatures of the puncture needles 31 and 32, the output of the radio waves, and the impedance of the ablation region. The operator operates the treatment device to proceed with RFA using the puncture needles 31 and 32.

[0033] Also, the puncture needles 31 and 32 are, for example, electrode needles that pass an electric current through the tissue to be treated, and are connected to a treatment device that controls the output of the electric current generated by the puncture needles 31 and 32. Here, this treatment device performs treatment by passing an electric current between the puncture needles 31 and 32 to pass an electric current through the tissue to be treated between the puncture needles 31 and 32.

[0034] The ultrasonic probe 12 is connected to a transmission / reception circuit 151 included in the device main body 15. The ultrasonic probe 12 has, for example, a plurality of piezoelectric vibrators in the probe main body, and these plurality of piezoelectric vibrators generate ultrasonic waves based on a drive signal supplied from the transmission / reception circuit 151. Also, the ultrasonic probe 12 receives a reflected wave from the subject and converts it into an electrical signal. Further, the ultrasonic probe 12 has, in the probe main body, a matching layer provided on the piezoelectric vibrator, a backing material that prevents the propagation of ultrasonic waves backward from the piezoelectric vibrator, and the like. Note that the ultrasonic probe 12 is detachably connected to the device main body 15. For example, the ultrasonic probe 12 is an ultrasonic probe such as a sector type, a linear type, or a convex type.

[0035] When ultrasonic waves are transmitted from the ultrasonic probe 12 to the subject, the transmitted ultrasonic waves are successively reflected at the discontinuous surfaces of the acoustic impedance in the body tissues of the subject, and are received by the plurality of piezoelectric vibrators included in the ultrasonic probe 12 as reflected wave signals. The amplitude of the received reflected wave signal depends on the difference in acoustic impedance at the discontinuous surface where the ultrasonic wave is reflected. Note that when the transmitted ultrasonic pulse is reflected at the surface of a moving blood flow, a heart wall, or the like, the reflected wave signal undergoes a frequency shift depending on the velocity component of the moving object with respect to the ultrasonic wave transmission direction due to the Doppler effect.

[0036] Note that this embodiment is applicable even when the subject is scanned two-dimensionally by the ultrasonic probe 12, which is a one-dimensional ultrasonic probe in which a plurality of piezoelectric vibrators are arranged in a line, or when the subject is scanned three-dimensionally by the ultrasonic probe 12, which is an ultrasonic probe that mechanically swings a plurality of piezoelectric vibrators of the one-dimensional ultrasonic probe or an ultrasonic probe 12 in which a plurality of piezoelectric vibrators are arranged two-dimensionally in a lattice pattern.

[0037] The monitor 13 displays a GUI (Graphical User Interface) for the operator of the ultrasonic diagnostic apparatus 100 to input various setting requests using the input interface 14, and ultrasonic images and the like generated in the apparatus main body 15. Further, the monitor 13 displays various messages and display information in order to notify the operator of the processing status and processing results of the apparatus main body 15. Further, the monitor 13 has a speaker and can also output sound.

[0038] The input interface 14 is operated to set a predetermined position (for example, a target for puncture, etc.), and is realized by, for example, a trackball, a switch button, a mouse, a keyboard, a touch pad that performs an input operation by touching the operation surface, a touch monitor in which the display screen and the touch pad are integrated, a non-contact input circuit using an optical sensor, and an audio input circuit. The input interface 14 is connected to a processing circuit 155 described later, and converts an input operation received from the operator into an electrical signal and outputs it to the processing circuit 155. Note that in this specification, the input interface 14 is not limited to only those provided with physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the apparatus and outputs this electrical signal to the processing circuit 155 is also included in the example of the input interface.

[0039] The apparatus main body 15 includes a transceiver circuit 151, a B-mode processing circuit 152, a Doppler processing circuit 153, a memory 154, a processing circuit 155, and a communication interface 156. In the ultrasonic diagnostic apparatus 1 shown in FIG. 2, each processing function is stored in the memory 154 in the form of a program executable by a computer. The transceiver circuit 151, the B-mode processing circuit 152, the Doppler processing circuit 153, and the processing circuit 155 are processors that realize the functions corresponding to the respective programs by reading and executing the programs from the memory 154. In other words, each circuit in the state of having read each program has the function corresponding to the read program.

[0040] The transceiver circuit 151 includes a pulse generator, a transmission delay circuit, a pulser, etc., and supplies a drive signal to the ultrasonic probe 12. The pulse generator repeatedly generates rate pulses for forming transmission ultrasonic waves at a predetermined rate frequency. Further, the transmission delay circuit focuses the ultrasonic waves generated from the ultrasonic probe 12 into a beam shape, and gives the delay time for each piezoelectric vibrator necessary for determining the transmission directivity to each rate pulse generated by the pulse generator. Further, the pulser applies a drive signal (drive pulse) to the ultrasonic probe 12 at a timing based on the rate pulse. That is, the transmission delay circuit arbitrarily adjusts the transmission direction of the ultrasonic waves transmitted from the piezoelectric vibrator surface by changing the delay time given to each rate pulse.

[0041] Note that the transceiver circuit 151 has a function of instantaneously changing the transmission frequency, the transmission drive voltage, etc. in order to execute a predetermined scan sequence based on an instruction from the processing circuit 155 described later. In particular, the change of the transmission drive voltage is realized by a linear amplifier type transmission circuit capable of instantaneously switching its value, or a mechanism for electrically switching a plurality of power supply units.

[0042] In addition, the transceiver circuit 151 includes a preamplifier, an A / D (Analog / Digital) converter, a reception delay circuit, an adder, etc., and performs various processes on the reflected wave signal received by the ultrasonic probe 12 to generate reflected wave data. The preamplifier amplifies the reflected wave signal for each channel. The A / D converter performs A / D conversion on the amplified reflected wave signal. The reception delay circuit provides the delay time necessary to determine the reception directivity. The adder performs an addition process on the reflected wave signal processed by the reception delay circuit to generate reflected wave data. By the addition process of the adder, the reflection component from the direction corresponding to the reception directivity of the reflected wave signal is emphasized, and a comprehensive beam of ultrasonic transmission and reception is formed by the reception directivity and the transmission directivity.

[0043] The B-mode processing circuit 152 receives the reflected wave data from the transceiver circuit 151, performs logarithmic amplification, envelope detection processing, etc., and generates data (B-mode data) in which the signal intensity is represented by the brightness of the luminance.

[0044] The Doppler processing circuit 153 performs frequency analysis on the velocity information from the reflected wave data received from the transceiver circuit 151, extracts blood flow, tissue, and contrast agent echo components due to the Doppler effect, and generates data (Doppler data) in which moving body information such as velocity, variance, and power is extracted for multiple points. For example, the moving body is a fluid such as blood flowing in a blood vessel or lymph fluid flowing in a lymphatic vessel.

[0045] Note that the B-mode processing circuit 152 and the Doppler processing circuit 153 can process both two-dimensional reflected wave data and three-dimensional reflected wave data. That is, the B-mode processing circuit 152 generates two-dimensional B-mode data from two-dimensional reflected wave data and generates three-dimensional B-mode data from three-dimensional reflected wave data. Also, the Doppler processing circuit 153 generates two-dimensional Doppler data from two-dimensional reflected wave data and generates three-dimensional Doppler data from three-dimensional reflected wave data. The three-dimensional B-mode data is data in which luminance values corresponding to the reflection intensities of reflection sources located at a plurality of points (sample points) set on each scanning line in the three-dimensional scanning range are assigned. Also, the three-dimensional Doppler data is data in which luminance values corresponding to the values of blood flow information (velocity, variance, power) are assigned to each of a plurality of points (sample points) set on each scanning line in the three-dimensional scanning range.

[0046] Also, the B-mode processing circuit 152 and the Doppler processing circuit 153 can synthesize a plurality of two-dimensional reflected wave data to generate three-dimensional reflected wave data and generate three-dimensional data from the generated reflected wave data. For example, the B-mode processing circuit 152 synthesizes a plurality of two-dimensional reflected wave data to generate three-dimensional reflected wave data and generates three-dimensional B-mode data from the generated three-dimensional reflected wave data. Also, the Doppler processing circuit 153 synthesizes a plurality of two-dimensional reflected wave data to generate three-dimensional reflected wave data and generates three-dimensional Doppler data from the generated three-dimensional reflected wave data.

[0047] The memory 154 stores the display image data generated by the processing circuit 155. Also, the memory 154 can store the data generated by the B-mode processing circuit 152 or the Doppler processing circuit 153. Also, the memory 154 stores control programs for performing ultrasonic transmission and reception, image processing, and display processing, diagnostic information (e.g., patient ID, doctor's findings, etc.), diagnostic protocols, and various data such as various body marks.

[0048] The communication interface 156 is connected to the processing circuit 155 and controls the communication conducted between the ultrasonic diagnostic apparatus 100 and each device. Specifically, the communication interface 156 receives various information from each device and outputs the received information to the processing circuit 155. For example, the communication interface 156 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like. For example, the communication interface 156 transmits the control signal received from the processing circuit 155 to the robot arm control device 21.

[0049] The processing circuit 155 controls the entire processing of the ultrasonic diagnostic apparatus 100. Specifically, the processing circuit 155 reads out and executes the programs corresponding to the control function 155a, the image generation function 155b, the scanning control function 155c, the setting function 155d, the puncture control function 155e, and the alignment function 156f shown in FIG. 2 from the memory 154, thereby performing various processes. For example, the processing circuit 155 is a processor that realizes the functions corresponding to the respective programs by reading out and executing each program from the memory 154. In other words, the processing circuit 155 in the state of having read out each program has each function shown in the processing circuit 155 of FIG. 2.

[0050] Here, the control function 155a is an example of a display control unit. The image generation function 155b is an example of an image generation unit. The scanning control function 155c is an example of a scanning control unit. Also, the setting function 155d is an example of a setting unit. Also, the puncture control function 155e is an example of a puncture control unit. Also, the alignment function 156f is an example of an alignment unit. In the present embodiment, it is described that each processing function described below is realized by a single processing circuit 155, but it is also possible to configure a processing circuit by combining a plurality of independent processors and realize the functions by each processor executing a program.

[0051] The control function 155a controls the processing of the transmission / reception circuit 151, the B-mode processing circuit 152, and the Doppler processing circuit 153 based on various setting requests input from the operator via the input interface 14, various control programs read from the memory 154, and various data. Further, the control function 155a controls to display an ultrasonic image and various information on the monitor 13.

[0052] Also, the control function 155a acquires information on a pre-planned puncture plan via the communication interface 156. For example, the control function 155a acquires a puncture plan established based on three-dimensional medical image data collected by a CT device, an MRI device, or an ultrasonic diagnostic device. That is, the control function 155a acquires information on the puncture position with respect to the subject set based on the puncture plan.

[0053] The image generation function 155b generates ultrasonic image data from the data generated by the B-mode processing circuit 152 and the Doppler processing circuit 153. That is, the image generation function 155b generates B-mode image data representing the intensity of the reflected wave as luminance from the two-dimensional B-mode data generated by the B-mode processing circuit 152. The B-mode image data is data in which the tissue shape within the region scanned by ultrasonic waves is depicted. Further, the image generation function 155b generates Doppler image data representing moving body information from the two-dimensional Doppler data generated by the Doppler processing circuit 153. The Doppler image data is velocity image data, dispersion image data, power image data, or image data combining these. The Doppler image data is data indicating fluid information regarding the fluid flowing within the region scanned by ultrasonic waves.

[0054] Here, the image generation function 155b generally converts the scanning line signal sequence of ultrasonic scanning into a scanning line signal sequence in a video format typified by a television or the like (scan conversion) to generate ultrasonic image data for display. Specifically, the image generation function 155b generates ultrasonic image data for display by performing coordinate conversion according to the scanning form of ultrasonic waves by the ultrasonic probe 12. Further, in addition to scan conversion, the image generation function 155b performs various image processes such as, for example, an image process (smoothing process) of regenerating an average value image of luminance using a plurality of image frames after scan conversion, and an image process (edge enhancement process) using a differential filter within the image. Also, the image generation function 155b synthesizes character information, scales, body marks, and the like of various parameters onto the ultrasonic image data.

[0055] Furthermore, the image generation function 155b generates three-dimensional B-mode image data by performing coordinate conversion on the three-dimensional B-mode data generated by the B-mode processing circuit 152. Also, the image generation function 155b generates three-dimensional Doppler image data by performing coordinate conversion on the three-dimensional Doppler data generated by the Doppler processing circuit 15,3. Further, the image generation function 155b can perform rendering processing on the volume data in order to generate various two-dimensional image data for displaying the volume data on the monitor 13.

[0056] The scanning control function 155c controls the movement of the arm 22c that holds the ultrasonic probe 12. Specifically, the scanning control function 155c controls the movement of the arm 22c by transmitting a control signal to the robot arm control device 21. Here, the scanning control function 155c can receive a manual operation by an operator on the ultrasonic probe 12 and assist the manual operation on the ultrasonic probe 12 by controlling the arm 22c. Specifically, the scanning control function 155c controls so as to assist the force applied from the operator to the ultrasonic probe 12. [[ID=!0]]

[0057] The setting function 155d sets control information for the puncture needle 31 (or / and the puncture needle 32) held by the arm 22a (or / and the arm 22b) based on the puncture information input for the ultrasonic image collected by the ultrasonic probe 12 held by the arm 22c. Specifically, the setting function 155d sets control information for operating the arm holding the puncture needle based on the information regarding puncture (such as information on the target of puncture) input for the ultrasonic image collected immediately before the insertion of the puncture needle.

[0058] The puncture control function 155e controls the movement of the arm 22a (or / and the arm 22b) holding the puncture needle 31 (or / and the puncture needle 32). Specifically, the scanning control function 155c controls the movement of the arm 22a (or / and the arm 22b) by transmitting a control signal to the robot arm control device 21. Specifically, the puncture control function 155e controls the movement of the arm holding the puncture needle based on the control information set by the setting function 155d. More specifically, the puncture control function 155e adjusts the position and angle of the puncture needle 31 (or / and the puncture needle 32) by controlling the arm 22a (or / and the arm 22b) based on the control information, moves the puncture needle 31 (or / and the puncture needle 32) to the insertion position on the body surface of the subject, and inserts the puncture needle 31 (or / and the puncture needle 32) into the target of puncture.

[0059] The alignment function 155f performs alignment between the ultrasonic image and the reference image. Specifically, the alignment function 155f aligns the ultrasonic image collected immediately before puncture with an image such as a CT or MRI used for creating the puncture plan to match the angles and positions of both planes. For example, the alignment function 155f performs alignment between the ultrasonic image and the reference image based on the puncture target specified in the ultrasonic image and the puncture target set on the reference image. Thereby, the coordinate space in the robot arm device 200 and the coordinate space in the reference image are associated with each other.

[0060] The apparatus configuration according to the first embodiment has been described above. Next, details of the information processing apparatus (ultrasonic diagnostic apparatus 100) according to the first embodiment will be described. Here, first, a procedure of processing using the information processing apparatus of the present application will be described with reference to FIG. 3. FIG. 3 is a flowchart for explaining the procedure of processing according to the first embodiment. Here, steps S101 to S102 in FIG. 3 are procedures performed when formulating a puncture plan, and steps S103 to S106 are procedures performed when actually performing treatment by puncture. Note that step S103 is realized by the processing circuit 155 reading and executing a program corresponding to the control function 155a, the image generation function 155b, and the scanning control function 155c from the memory 154. Also, steps S104 and S105 are realized by the processing circuit 155 reading and executing a program corresponding to the setting function 155d from the memory 154. Further, step S106 is realized by the processing circuit 155 reading and executing a program corresponding to the puncture control function 155e from the memory 154.

[0061] As shown in FIG. 3, in the procedure using the information processing apparatus of the present application, for example, the operator first collects three-dimensional image data (volume data) using a CT apparatus, an MRI apparatus, the ultrasonic diagnostic apparatus 100, or the like (step S101). Then, the operator plans the puncture position using the collected three-dimensional image data (step S102).

[0062] For example, the operator observes the three-dimensional image data collected using a CT apparatus, an MRI apparatus, the ultrasonic diagnostic apparatus 100, or the like to identify the position of the target to be treated. Then, the operator determines the position and number of puncture needles to be inserted in consideration of the shape and size of the target and the positions of surrounding organs and bones. For example, the operator determines the insertion position and angle of each puncture needle on the body surface so as to avoid blood vessels, various organs, bones, etc. and enable appropriate treatment of the target. That is, in the puncture plan, the position on the body surface where the puncture needle is inserted and the insertion angle of the puncture needle are determined in the coordinates of the volume data.

[0063] After the puncture plan is established as described above, the puncture is performed. Specifically, the processing circuit 155 first controls the arm 22c to collect an ultrasonic image immediately before the puncture (step S103). Then, the processing circuit 155 receives puncture information for the collected ultrasonic image (step S104), and sets control information for controlling the arm 22a (or / and arm 22b) that holds the puncture needle based on the puncture information for the ultrasonic image.

[0064] Thereafter, the processing circuit 155 controls the arm 22a (or / and arm 22b) that holds the puncture needle based on the control information (step S106) and performs the puncture. In this way, the information processing apparatus (ultrasonic diagnostic apparatus 100) of the present application easily realizes accurate puncture in puncture support using the arm by controlling the arm based on the ultrasonic image collected immediately before the puncture. Hereinafter, the details of the processing by the ultrasonic diagnostic apparatus 100 will be described in order.

[0065] (Collection of Ultrasonic Image) As described above, the scanning control function 155c controls the movement of the arm 22c that holds the ultrasonic probe 12 to collect an ultrasonic image. Specifically, the scanning control function 155c controls the movement of the arm 22c by transmitting control information including coordinate information in the operation space of the robot arm device 200 to the robot arm control device 21.

[0066] Here, the scanning control function 155c receives a manual operation by the operator on the ultrasonic probe 12 and supports the manual operation on the ultrasonic probe by controlling the arm 22c. Specifically, when the scanning control function 155c receives an operation in which the ultrasonic probe 12 held by the arm 22c is moved by the operator's hand, the scanning control function 155c drives the drive mechanism in the arm 22c to support the movement of the ultrasonic probe 12 by the operator.

[0067] That is, the scanning control function 155c controls to assist the force applied by the operator to the ultrasonic probe 12. For example, the scanning control function 155c automatically calculates in real time the load such as the weight of the ultrasonic probe and the cable, and reduces the load of the probe scanning by the operator by holding such a load. Further, the scanning control function 155c detects the direction of manual movement by a sensor provided on the arm 22c, and controls to move the arm 22c in the detected direction, thereby reducing the load of the probe scanning by the operator.

[0068] Thereby, the operator can move the ultrasonic probe 12 more simply than manually scanning the conventional ultrasonic probe, and can place the ultrasonic probe 12 at an appropriate position where an ultrasonic image including the puncture target can be collected. As a result, the scanning control function 155c can grasp the position (coordinates) of the arm 22c for collecting an appropriate ultrasonic image when performing puncture in the coordinate space in the robot arm device 200.

[0069] Here, the scanning control function 155c can further adjust the position of the ultrasonic probe moved by manual operation. As described above, the ultrasonic probe 12 held by the arm 22c is placed on the body surface of the subject by the manual scanning of the operator. However, due to the body movement of the subject or the like, the position of the ultrasonic probe 12 for collecting an appropriate ultrasonic image may deviate slightly.

[0070] Therefore, the scanning control function 155c finely adjusts the position and angle of the ultrasonic probe 12 by controlling the arm 22c. FIG. 4 is a diagram showing an example of the arm 22c that holds the ultrasonic probe 12 according to the first embodiment. For example, as shown in FIG. 4, the arm 22c holds the ultrasonic probe 12 on the tip side. Then, the scanning control function 155c moves the ultrasonic probe 12 in the X-axis direction, Y-axis direction, and Z-axis direction by driving the drive mechanism of the arm 22c. Further, the scanning control function 155c changes the orientation of the ultrasonic probe 12 by rotating the ultrasonic probe 12 in the directions of arrows D1 to D3 by driving the drive mechanism of the holding part of the arm 22c. The scanning control function 155c finely adjusts the position and angle of the ultrasonic probe 12 by executing these movements at a fine level.

[0071] Thereby, it is possible to easily move the ultrasonic probe 12 by a small amount and fix the position after the movement, and it is possible to easily collect a more appropriate ultrasonic image.

[0072] (Setting of Control Information) As described above, the setting function 155d sets control information for controlling the arm 22a (or / and the arm 22b) based on the puncture information input to the ultrasonic image collected in real time during puncture. That is, the setting function 155d specifies the position corresponding to the puncture information in the space coordinates where the arm 22c and the arm 22a (and the arm 22b) operate, and sets the control information of the puncture needle based on the specified position.

[0073] Specifically, the setting function 155d specifies the position (coordinates) of the puncture information input to the ultrasonic image by specifying the position (coordinates) of the ultrasonic image in the coordinate space of the robot arm device 200 based on the position (coordinates) of the arm 22c in the coordinate space when the ultrasonic image in which the puncture information is input is collected.

[0074] FIG. 5 is a diagram for explaining an example of processing by the setting function 155d according to the first embodiment. Here, FIG. 5 shows the processing in a state where an ultrasonic image in which a puncture target, a puncture path, etc. are appropriately shown is displayed. For example, when a doctor designates a target T1 on the ultrasonic image as shown in the upper part of FIG. 5 and further designates a puncture path, the setting function 155d sets control information for performing a puncture along the designated puncture path with respect to the designated target T1.

[0075] That is, based on the position (coordinates) of the arm 22c when the ultrasonic image is collected, the setting function 155d identifies the position (coordinates) of the ultrasonic image in the coordinate space S1 of the robot arm device 200 as shown in the lower diagram of FIG. 5. Then, the setting function 155d identifies the positions (coordinates) of the target T1 and the puncture path designated on the ultrasonic image in the coordinate space S1.

[0076] After that, based on the current position (coordinates) of the arm 22a (or / and) arm 22b that holds the puncture needle in the coordinate space S1 and the positions (coordinates) of the identified target T1 and the puncture path in the coordinate space S1, the setting function 155d sets control information for moving the arm 22a (or / and) arm 22b. That is, the setting function 155d sets control information for moving the arm 22a (or / and) arm 22b from the current position to the position for performing the puncture.

[0077] In ultrasonic-guided puncture, various measures are taken, such as changing the body position, changing the respiratory phase, the way of applying the probe, pushing, and waving, in order to obtain an appropriate tumor image or secure an appropriate puncture path. Also, in ultrasonic-guided puncture, measures are taken to reduce artifacts caused by gas in the lungs using artificial pleural effusion or artificial ascites. Even in such various situations, by performing the above-described setting of control information, it is possible to cope with changes in the needle tip position and the position coordinates of the puncture path in the coordinate space of the robot arm device 200, and it is possible to easily perform position correction.

[0078] In addition, in the case of puncture under ultrasonic guidance, reference images such as CT and MRI may be aligned with the ultrasonic image and the puncture may be performed while referring to them. In such a procedure, usually, a position sensor is attached to the ultrasonic probe, and the linkage with the reference image is performed based on the position information of the real-time ultrasonic image. However, in the present application, without attaching a dedicated position sensor to the ultrasonic probe 12, the position information of the arm 22c in the coordinate space of the robot arm device 200 is used. Specifically, the alignment function 155f reads reference images such as ultrasonic images, CT images, and MRI images, and performs alignment with the real-time ultrasonic image. Thereby, the position information on the reference image can be defined in the coordinates in the coordinate space of the robot arm device 200 in synchronization with the position information of the real-time ultrasonic image.

[0079] In this way, when the real-time ultrasonic image and the reference image are aligned, since the position on the reference image can be converted into the position in the coordinate space of the robot arm device 200, the position of the target and the puncture path at the time of puncture can be specified on the reference image. That is, the setting function 155d sets the control information of the puncture needle held by the arm a (or / and the arm 22b) based on the puncture information input with respect to the reference image for which alignment with the ultrasonic image has been performed.

[0080] In addition, when the real-time ultrasonic image and the reference image are aligned, the control function 155a can display the real-time ultrasonic image and the corresponding reference image on the monitor 13. For example, the control function 155a displays the real-time ultrasonic image and the corresponding reference image in parallel.

[0081] In addition, the control function 155a can display the position of the ultrasonic probe 12 and the position of the real-time ultrasonic image on the reference image. Thereby, the operator can move the arm 22c holding the ultrasonic probe 12 using the position of the ultrasonic probe 12 and the position of the real-time ultrasonic image displayed on the reference image.

[0082] For example, the control function 155a causes a marker indicating the position of the ultrasonic probe 12 and the position of the ultrasonic image to be displayed on the reference image. Also, for example, the control function 155a causes a GUI for inputting the distance and angle for moving the ultrasonic probe 12 to be displayed. The operator sets the control information for the arm 22c by moving the marker or inputting numerical values via the input interface 14.

[0083] The scanning control function 155c controls the movement of the ultrasonic probe 12 held by the arm 22c based on the operation information input with respect to the reference image for which alignment with the ultrasonic image has been performed. That is, the scanning control function 155c moves the arm 22 so that an ultrasonic image at a specified position is collected based on the control information set by the operator.

[0084] (Control of Puncturing by a Robot Arm) As described above, the puncture control function 155e controls the movement of the arm 22a (or / and arm 22b) that holds the puncture needle based on the control information set by the setting function 155d. Specifically, the puncture control function 155e moves the arm 22a (or / and arm 22b) to the puncturing position of the puncture needle based on the control information. That is, the puncture control function 155e changes the angle of the arm so that the tip of the puncture needle is placed at the puncturing position and the puncture needle punctures at the angle of the puncture path based on the coordinates of the puncturing position on the body surface of the subject, the coordinates of the position of the tip of the needle at the target, and the coordinates of the puncture path included in the control information.

[0085] Here, the puncture control function 155e can receive a manual operation by the operator on the puncture needle and assist the manual operation on the puncture needle by controlling the arm 22a (or / and arm 22b). That is, the operator can move the arm 22a (or / and arm 22b) by hand.

[0086] Then, the puncture control function 155e moves the puncture needle in the longitudinal direction by controlling the arm. Here, the puncture control function 155e can control the pressure and speed during the insertion and removal of the puncture needle.

[0087] For example, the puncture control function 155e controls the insertion speed of the puncture needle to be constant and controls the insertion speed of the puncture needle according to the change in the insertion pressure of the puncture needle. For example, the puncture control function 155e monitors the pressure applied to the puncture needle while inserting the puncture needle at a constant insertion speed, and when the pressure exceeds the threshold value, controls to increase the insertion speed.

[0088] Also, for example, the puncture control function 155e controls the insertion pressure of the puncture needle to be constant and controls the insertion pressure of the puncture needle according to the change in the insertion speed of the puncture needle. For example, the puncture control function 155e monitors the insertion speed of the puncture needle while inserting the puncture needle at a constant insertion pressure, and when the insertion speed is below the threshold value, increases the allowable value of the pressure applied to the puncture needle.

[0089] Also, for example, the puncture control function 155e can also switch between a constant insertion speed and a constant insertion pressure according to the position from the surface of the subject to the target. For example, the puncture control function 155e controls at a constant insertion speed until it reaches a position that has moved a predetermined distance from the body surface, and controls at a constant insertion pressure from the position where it has moved a predetermined distance from the body surface to the target.

[0090] As described above, the puncture control function 155e can control by appropriately combining conditions such as the insertion speed, the insertion pressure, the threshold values for changing the speed and pressure, and the switching between a constant insertion speed and a constant insertion pressure. Here, the above conditions can be arbitrarily set according to the insertion position of the puncture needle in the subject, the target organ, etc. That is, the puncture control function 155e acquires information regarding the insertion position of the puncture needle and the target organ, reads out the conditions corresponding to the acquired information, and performs the insertion of the puncture needle.

[0091] As described above, the puncture control function 155e can monitor the actual insertion speed and insertion pressure while controlling the insertion speed and insertion pressure of the puncture needle. In this case, the control function 155a can cause the monitor 13 to display the insertion speed and insertion pressure of the puncture needle controlled by the puncture control function 155e and the actually monitored insertion speed and insertion pressure.

[0092] FIG. 6 is a diagram for explaining an example of the process by the control function 155a according to the first embodiment. For example, as shown in FIG. 6, the control function 155a causes the "speed: xxx" and "pressure: yyy" of the puncture needle actually monitored to be displayed together with the ultrasonic image including the puncture needle during puncture with the ultrasonic guide. Note that the control function 155a can also display the set insertion speed and insertion pressure in addition to the actually monitored insertion speed and insertion pressure.

[0093] (Control of Procedures Using Multiple Puncture Needles) The control of puncture by the robotic arm is not limited to the procedure using a single puncture needle, and can also control the procedure using multiple puncture needles. For example, there are procedures that require puncture at multiple locations, such as ablation therapy using a nano knife or Celon needle, and systematic prostate biopsy (a procedure of performing puncture at a plurality of predetermined locations).

[0094] For such procedures, for example, the puncture control conditions regarding the insertion speed, insertion pressure, and position of the puncture needle, and the puncture protocol are registered in the memory 154 in advance, and the scanning control function 155c and the puncture control function 155e execute control according to the puncture control conditions and the puncture protocol. That is, the scanning control function 155c and the puncture control function 155e read out the puncture control conditions and the puncture protocol designated by the operator from the memory 154 and execute control in accordance with the read puncture control conditions and the puncture protocol. Note that the puncture control conditions and the puncture protocol can be edited as appropriate.

[0095] (Manual Puncture) In the above example, the case where the puncture control function 155e controls the arm 22a (or / and arm 22b) to automatically perform the puncture of the puncture needle has been described. However, the arms 22a and 22b can hold the puncture needle so that the puncture needle can be manually inserted. That is, after the puncture control function 155e places the tip of the puncture needle at the puncture position and changes the angle of the arm so that the puncture needle penetrates at the angle of the puncture path, the operator (doctor) can move the puncture needle and insert it into the subject.

[0096] In addition, by holding a puncture adapter that guides the puncture of the puncture needle instead of the puncture needle with the arm 22a (or / and arm 22b), manual puncture can be performed. In such a case, the setting function 155d moves the arm 22a (or / and arm 22b) so that the mounting position of the puncture needle in the puncture adapter is arranged at the puncture position on the body surface and the holding angle of the puncture needle by the puncture adapter coincides with the angle of the puncture path. Set the control information for.

[0097] The puncture control function 155e moves the arm 22a (or / and arm 22b) based on the control information set by the setting function 155d. As a result, the puncture adapter is held in a stable state at a desired position. The operator (doctor) can attach the puncture needle to the puncture adapter held by the arm 22a (or / and arm ②) and insert it himself.

[0098] As described above, according to the first embodiment, the setting function 155d is based on the puncture information input to the ultrasonic image collected by the ultrasonic probe 12 held by the arm 22c, and the puncture needle held by the arm 22a (or / and arm 22b). Set the control information. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can set the control information of the puncture needle using the real-time ultrasonic image immediately before puncture, and can easily realize accurate puncture in puncture support using a robotic arm.

[0099] Also, according to the first embodiment, the scanning control function 155c receives a manual operation by the operator on the ultrasonic probe 12, and controls the arm 22c to assist the manual operation on the ultrasonic probe 12. The setting function 155d sets control information based on the puncture information input to the ultrasonic image collected by the ultrasonic probe 12 held by the arm 22c operated by the manual operation. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables easy collection of an ultrasonic image by the ultrasonic probe held by the arm.

[0100] Also, according to the first embodiment, the scanning control function 155c controls to assist the force applied from the operator to the ultrasonic probe 12. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables reduction of the load on the manual operation by the operator.

[0101] Also, according to the first embodiment, the scanning control function 155c adjusts the position of the ultrasonic probe 12 moved by the manual operation. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can finely adjust the position of the ultrasonic probe 12 and can maintain the ultrasonic probe 12 in a stable state at the adjusted position.

[0102] Also, according to the first embodiment, the puncture control function 155e receives a manual operation by the operator on the puncture needle, and controls the arm 22a (or / and the arm 22b) to assist the manual operation on the puncture needle. Therefore, the ultrasonic diagnostic apparatus 1 according to the first embodiment enables easy movement of the arm 22a (or / and the arm 22b).

[0103] Further, according to the first embodiment, the setting function 155d specifies a position corresponding to the puncture information in the spatial coordinates where the arm 22c and the arm 22a (or / and the arm 22b) operate, and sets control information for the puncture needle based on the specified position. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables accurate puncture at the position specified in the ultrasonic image.

[0104] Further, according to the first embodiment, the alignment function 155f performs alignment between the ultrasonic image and the reference image. The setting function 155d sets control information for the puncture needle held by the arm 22a (or / and the arm 22b) based on the puncture information input with respect to the reference image for which alignment with the ultrasonic image has been performed. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables setting of puncture information using the reference image in consideration of the position of the subject immediately before puncture.

[0105] Further, according to the first embodiment, the alignment function 155f performs alignment between the ultrasonic image and the reference image. The scanning control function 155c controls the movement of the ultrasonic probe 12 held by the arm 22c based on the operation information input with respect to the reference image for which alignment with the ultrasonic image has been performed. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables determination of the position of the ultrasonic image with reference to the information included in the reference image.

[0106] Further, according to the first embodiment, the arm 22a (or / and the arm 22b) holds a puncture adapter that guides the insertion of the puncture needle. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables execution of the manual insertion of the puncture needle in a more stable state.

[0107] Further, according to the first embodiment, the puncture control function 155e controls the pressure and speed during the insertion and removal of the puncture needle. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment enables realization of puncture by a robotic arm corresponding to various procedures.

[0108] Also, according to the first embodiment, the puncture control function 155e controls the insertion speed of the puncture needle to be constant, and controls the insertion speed of the puncture needle according to the change in the insertion pressure of the puncture needle. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can detect changes in the hardness and ease of insertion in the living body, and perform a puncture accordingly.

[0109] Also, according to the first embodiment, the puncture control function 155e controls the insertion pressure of the puncture needle to be constant, and controls the insertion pressure of the puncture needle according to the change in the insertion speed of the puncture needle. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can detect changes in the hardness and ease of insertion in the living body, and perform a puncture accordingly.

[0110] Also, according to the first embodiment, the control function 155a controls to display information on the pressure and speed during the insertion and removal of the puncture needle. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can present the detection results of changes in the hardness and ease of insertion in the living body to the observer.

[0111] Also, according to the first embodiment, the puncture control function 155e controls the movement of the second robotic arm according to the puncture protocol for inserting a plurality of puncture needles. Therefore, the ultrasonic diagnostic apparatus 100 according to the first embodiment can stably perform a difficult procedure that requires skill when conventionally performed manually. As a result, the frequency of performing an effective procedure can be increased.

[0112] (Second Embodiment) In the above-described first embodiment, the case where the puncture needle is inserted automatically or manually has been described. In the second embodiment, the case where the puncture needle is inserted using a virtual puncture device will be described. FIG. 7 is a block diagram showing an example of the configuration of the ultrasonic diagnostic apparatus 100 according to the second embodiment. Here, the ultrasonic diagnostic apparatus 100 according to the second embodiment is different from the first embodiment in that the virtual puncture device 41 is newly connected and the processing content by the puncture control function 155e is different. Hereinafter, the description will be centered around these points.

[0113] As shown in FIG. 7, the virtual puncture device 41 is connected to the apparatus main body 15 via a communication interface. The virtual puncture device 41 is realized by a tactile device and virtually reproduces the puncture pressure of the puncture needle and the stress of the living body. Specifically, the virtual puncture device 41 acquires the pressure applied to the puncture needle detected by the arm 22a (or arm 22b), and operates the tactile device based on the acquired pressure, thereby reproducing the puncture pressure of the puncture needle and the stress of the living body. Then, the virtual puncture device 41 receives an operation related to the insertion of the puncture needle from the operator (physician), and notifies the received operation to the puncture control function 155e. The puncture control function 155e controls the arm 22a (or arm 22b) according to the operation notified from the virtual puncture device 41.

[0114] FIG. 8 is a schematic diagram showing an example of the virtual puncture device 41 according to the second embodiment. For example, as shown in FIG. 8, the virtual puncture device 41 has a puncture needle tool that simulates a puncture needle. The operator (physician) controls the arm 22a (or arm 22b) by gripping and operating the puncture needle tool in the virtual puncture device 41. That is, the operator (physician) controls the arm 22a (or arm 22b) so that the puncture needle is inserted into the subject by pushing down the puncture needle tool as shown in the right figure of FIG. 8. Further, the operator (physician) controls the arm 22a (or arm 22b) so that the puncture needle is removed from the subject by pushing up the puncture needle tool as shown in the left figure of FIG. 8.

[0115] At this time, since the virtual puncture device 41 reproduces the piercing pressure of the puncture needle and the stress of the living body by the tactile device, the operator (doctor) can feel the pressure from the puncture needle and the moving speed of the puncture needle through the sense of touch with the puncture needle tool, and perform the puncture with the same feeling as a normal puncture.

[0116] The puncture control function 155e according to the second embodiment controls the puncture by the arm 22a (or the arm 22b) according to an operation input through a virtual puncture device that reproduces the state at the time of puncture of the puncture needle. Specifically, the puncture control function 155e controls the arm 22a (or the arm 22b) according to the movement of the puncture needle tool in the virtual puncture device 41, and performs the insertion and removal of the puncture needle.

[0117] Here, the puncture control function 155e can accurately reflect the operation of the puncture needle tool by the operator (doctor) to the arm 22a (or the arm 22b). That is, the puncture control function 155e can control the arm 22a (or the arm 22b) so that the actual puncture needle moves by the same amount of movement as the amount of movement of the puncture needle tool by the operator (doctor).

[0118] Furthermore, the puncture control function 155e can also perform various corrections on the operation of the puncture needle tool by the operator (doctor), and control the arm 22a (or the arm 22b) with the corrected amount of movement. For example, the puncture control function 155e can reproduce the operation after correcting the hand tremor of the operator (doctor). Also, for example, the puncture control function 155e can reproduce the operation with the corrected amount of movement with respect to the amount of movement that the operator (doctor) actually moved the puncture needle tool. For example, the puncture control function 155e can control the arm 22a (or the arm 22b) so that the actual puncture needle moves by 1 / 5 (or 2 times) the amount of movement that the operator (doctor) actually moved the puncture needle tool. Note that the correction of the amount of movement is not limited to 1 / 5 and 2 times, and can be set variously.

[0119] As described above, according to the second embodiment, the puncture control function 155e controls the puncture by the arm 22a (or the arm 22b) in response to an operation input via the virtual puncture device 41 that reproduces the state of the puncture needle during puncture. Therefore, the ultrasonic diagnostic apparatus 100 according to the second embodiment enables the puncture by the robotic arm to be realized more intuitively.

[0120] (Other Embodiments) In the above-described embodiment, the case where the information processing apparatus according to the present application is incorporated into the ultrasonic diagnostic apparatus 100 has been described. However, the embodiment is not limited thereto, and for example, the information processing apparatus according to the present application may be incorporated into the robotic arm apparatus 200. In such a case, each of the above-described functions is executed by the processing circuit of the robotic arm apparatus 200.

[0121] Note that the term "processor" used in the above description means, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a circuit such as an application specific integrated circuit (ASIC), a programmable logic device (for example, a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). The processor realizes its functions by reading and executing a program stored in a memory. Note that instead of storing the program in the memory, the program may be directly incorporated into the circuit of the processor. In this case, the processor realizes its functions by reading and executing the program incorporated into the circuit. Note that each processor of the present embodiment is not limited to being configured as a single circuit for each processor, and a plurality of independent circuits may be combined and configured as one processor to realize its functions.

[0122] Note that each component of each device illustrated in the description of the above embodiment is functionally conceptual, and does not necessarily have to be physically configured as illustrated. That is, the specific form of the distribution and integration of each device is not limited to that illustrated, and all or a part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage situations, and the like. Further, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic.

[0123] In addition, the control method described in the above-described embodiment can be realized by executing a prepared control program on a computer such as a personal computer or a workstation. This control program can be distributed via a network such as the Internet. Further, this control program is recorded on a non-transitory computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, a DVD, a USB memory, and a Flash memory such as an SD card memory, and can also be executed by being read from the non-transitory recording medium by a computer.

[0124] As described above, according to the embodiment, in puncture support using a robotic arm, it is possible to easily realize accurate puncture.

[0125] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations of the embodiments can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, as well as in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0126] 100 Ultrasonic diagnostic apparatus 155 Processing circuit 155a Control function 155c Scanning control function 155d Setting function 155e Puncture control function 155f Alignment function

Claims

1. A setting unit that sets control information for a puncture needle held by a second robotic arm based on puncture information input with respect to an ultrasonic image collected by an ultrasonic probe held by a first robotic arm; A puncture control unit that controls the movement of the second robotic arm holding the puncture needle based on the control information; Comprising: The puncture control unit controls pressure and speed during insertion and removal of the puncture needle, switches between a constant insertion speed and a constant insertion pressure according to the position from the surface of the subject to the target, and controls the puncture by the second robotic arm according to an operation input via a virtual puncture device that reproduces the state of the puncture needle during puncture. The virtual puncture device is an information processing device that acquires the pressure applied to the puncture needle detected by the second robotic arm and operates a tactile device based on the acquired pressure to reproduce the insertion pressure of the puncture needle and the stress of the living body.

2. Further comprising a scanning control unit that accepts a manual operation by an operator on the ultrasonic probe and controls the first robotic arm to assist the manual operation on the ultrasonic probe; The setting unit sets the control information based on puncture information input with respect to an ultrasonic image collected by the ultrasonic probe held by the first robotic arm operated by the manual operation. The information processing device according to claim 1.

3. The scanning control unit controls to assist the force applied by the operator to the ultrasonic probe. The information processing device according to claim 2.

4. The scanning control unit adjusts the position of the ultrasonic probe moved by the manual operation. The information processing device according to claim 2 or 3.

5. The puncture control unit accepts a manual operation by an operator on the puncture needle and controls the second robotic arm to assist the manual operation on the puncture needle. The information processing device according to any one of claims 1 to 4.

6. The setting unit identifies the position corresponding to the puncture information in the spatial coordinates in which the first robotic arm and the second robotic arm operate, and sets the control information for the puncture needle based on the identified position. The information processing device according to any one of claims 1 to 5.

7. Further comprising an alignment unit that performs alignment between the ultrasonic image and a reference image The setting unit sets control information for the puncture needle held by the second robotic arm based on the puncture information input with respect to the reference image that has been aligned with the ultrasonic image, according to any one of claims 1 to 6.

8. further comprising an alignment unit that performs alignment between the ultrasonic image and the reference image; The scanning control unit controls the movement of the ultrasonic probe held by the first robotic arm based on the operation information input with respect to the reference image that has been aligned with the ultrasonic image, according to claim 2.

9. The second robotic arm holds a puncture adapter that guides the insertion of the puncture needle, according to any one of claims 1 to 8.

10. The puncture control unit controls the puncture needle at a constant insertion speed until it reaches a position that has moved a predetermined distance from the body surface of the subject, and controls the puncture needle at a constant insertion pressure from when it has moved the predetermined distance from the body surface until it reaches the target, according to any one of claims 1 to 9.

11. The puncture control unit further controls to increase the insertion speed of the puncture needle when the insertion pressure of the puncture needle exceeds a threshold value while controlling the puncture needle at a constant insertion speed, according to claim 1.

12. The puncture control unit further controls to increase the insertion pressure of the puncture needle when the insertion speed of the puncture needle falls below a threshold value while controlling the puncture needle at a constant insertion pressure, according to claim 1.

13. further comprising a display control unit that controls to display information on the pressure and speed during insertion and removal of the puncture needle, according to any one of claims 1 to 12.

14. The puncture control unit controls the movement of the second robotic arm according to a puncture protocol for inserting a plurality of puncture needles, according to any one of claims 1 to 13.

15. the ultrasonic probe, an image generation unit that generates the ultrasonic image based on the ultrasonic waves transmitted and received by the ultrasonic probe, an information processing apparatus according to any one of claims 1 to 14, An ultrasonic diagnostic apparatus comprising:

16. a setting function for setting control information for the puncture needle held by the second robotic arm based on the puncture information input with respect to the ultrasonic image collected by the ultrasonic probe held by the first robotic arm; Based on the control information, control the movement of the second robotic arm that holds the puncture needle, control the pressure and speed during the insertion and removal of the puncture needle, and switch between a constant insertion speed and a constant insertion pressure according to the position from the body surface of the subject to the target. A puncture control function that controls the puncture by the second robotic arm according to the operation input via a virtual puncture device that reproduces the state during the puncture of the puncture needle. Cause the information processing device to execute. The virtual puncture device is a program that acquires the pressure applied to the puncture needle detected by the second robotic arm and operates a tactile device based on the acquired pressure to reproduce the insertion pressure of the puncture needle and the stress of the living body.

Images