Prostate puncture support device, method of operating prostate puncture support device, and program for prostate puncture support device

Abstract

To provide a prostate puncture support device, a method of operating the prostate puncture support device, and a program for the prostate puncture support device, in which the degrees of freedom of a puncture position and angle are secured and the complication of the management of cables of sensors, a magnetic field fluctuation and the like are suppressed.SOLUTION: A prostate puncture system 10 includes a puncture support device 15 that supports puncture during a prostate biopsy. A surgical field image acquisition unit 25 acquires a surgical field image that is captured by a camera 14 positioned outside a body and includes a puncture needle image marker 13 attached to a puncture needle 12; and a position / posture information acquisition unit 21 analyzes the image marker to obtain position / posture information. A puncture information output unit 27 outputs puncture support information of the puncture needle 12 that is based on the position / posture information obtained by the position / posture information acquisition unit 21, and a display control unit 23 superimposes the puncture support information on an image to generate a superimposed image and displays the superimposed image on a display 16.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a prostate puncture assistance device for inserting a puncture needle into the prostate, a method for operating the prostate puncture assistance device, and a program for the prostate puncture assistance device. [Background technology]

[0002] A prostate biopsy is one method for detecting the presence or absence of prostate cancer. A biopsy involves removing a part of the body and conducting a pathological test on the collected cells and tissue to determine whether or not the disease is present. A prostate biopsy involves inserting an ultrasound probe transrectally, and while the condition of the prostate is viewed through ultrasound images, a thin needle (puncture needle) is inserted into the prostate to collect (puncture) tissue. In one test, the needle is typically inserted into more than 10 places to collect tissue.

[0003] During prostate biopsy, if the lesion is small, it can be difficult to see using ultrasound images alone, making it difficult to accurately puncture the lesion. To solve this problem, puncture assistance devices have been developed. Puncture assistance devices include prostate biopsy systems that use surgical robots to assist puncture, as well as MRI-US fusion, which fuses MRI images with ultrasound images and displays them in real time to improve puncture accuracy. [Prior art documents] [Patent documents]

[0004] [Non-Patent Document 1] MRI-US fusion prostate biopsy system (ARIETTA 65 Intuitive Fusion) and others [online] [searched December 13, 2023], Internet<URL:1706596505721_0 products / release / 20230524> Summary of the Invention [Problem to be solved by the invention]

[0005] The encoder-type prostate fusion biopsy system, which is a puncture support device, has the problem that it is necessary to use a template to fix the puncture path of the puncture needle in order to identify the puncture position, which places restrictions on the puncture position and angle.Furthermore, the magnetic sensor-type fusion biopsy system requires sensors to be attached to each surgical instrument, which makes cable management cumbersome.

[0006] The present invention aims to provide a prostate puncture assistance device that ensures freedom of puncture position and angle and reduces the complexity of sensor cable management, a method for operating the prostate puncture assistance device, and a program for the prostate puncture assistance device. [Means for solving the problem]

[0007] The prostate puncture assistance device of the present invention is equipped with a processor that acquires a reference volume image including the prostate gland and a region of interest within the prostate gland, which is acquired preoperatively, an operative field image acquired by a camera positioned outside the body and which includes at least a puncture needle image marker attached to the puncture needle, and an ultrasound image acquired by an ultrasound probe, and outputs puncture assistance information related to the puncture needle to the ultrasound image based on position and orientation information of the puncture needle obtained by analyzing the puncture image marker.

[0008] The processor preferably performs registration between the reference volume image and the ultrasound image based on the position and orientation information of the ultrasound probe.

[0009] It is preferable that the surgical field image further includes a probe image marker attached to the ultrasound probe, and the processor acquires the position and orientation information of the ultrasound probe by analyzing the probe image marker included in the surgical field image.

[0010] It is preferable that the surgical field image further includes a magnetic field generator image marker attached to the magnetic field generator, the ultrasound probe includes a magnetic sensor, and the processor calculates position and orientation information of the ultrasound probe based on position and orientation information obtained by the magnetic sensor of the ultrasound probe and position and orientation information of the magnetic field generator image marker attached to the magnetic field generator.

[0011] It is preferable that the surgical field image further includes a subject image marker attached to the subject's body surface, and the processor corrects the positional deviation between the reference volume image and the ultrasound image by obtaining position and orientation information of the body surface from the subject image marker.

[0012] It is preferable that the processor distinguishes between multiple markers, including a puncture needle image marker attached to the puncture needle, a probe image marker attached to the ultrasound probe, and a subject image marker attached to the body surface, based on the color and / or shape of the image marker.

[0013] It is preferable that the processor distinguishes between multiple markers, including a puncture needle image marker attached to the puncture needle, a magnetic field generator image marker attached to the magnetic field generator, and a subject image marker attached to the body surface, based on the color and / or shape of the image marker.

[0014] The puncture assistance information regarding the puncture needle is preferably a puncture guideline for puncture.

[0015] Preferably, the processor may superimpose, on the ultrasound image, puncture assistance information including a region of interest in the reference volume image and puncture guidelines.

[0016] The region of interest in the acquired reference volume image may be the entire image, or may be a specific region such as a target organ or a surrounding image.

[0017] Preferably, the processor may superimpose puncture support information including the ultrasound image, a region of interest in the reference volume image, and a puncture guideline on the operative field image.

[0018] The ultrasound image may be a 3D ultrasound image, and the processor preferably acquires position and orientation information of the 2D ultrasound image using a probe image marker attached to the ultrasound probe, and generates the 3D ultrasound image based on the acquired position and orientation information.

[0019] Preferably, the processor may superimpose the ultrasound image, the reference volume image, and the puncture guideline on a virtual space in which the target organ is 3D rendered.

[0020] A method for operating a prostate puncture assistance device of the present invention includes a processor, and the processor includes the steps of acquiring a reference volume image including the prostate gland and a region of interest within the prostate gland, which is acquired preoperatively, and an operative field image including at least a puncture needle image marker attached to the puncture needle, which is acquired by a camera located outside the body, and outputting puncture assistance information related to the puncture needle based on position and orientation information of the puncture needle obtained by analyzing the puncture image marker.

[0021] The program for the prostate puncture assistance device of the present invention enables a computer to perform the following functions: acquire a reference volume image including the prostate gland and a region of interest within the prostate gland, which is acquired preoperatively; acquire an operative field image acquired by a camera positioned outside the body and including at least a puncture needle image marker attached to the puncture needle; and output puncture assistance information related to the puncture needle based on position and orientation information of the puncture needle obtained by analyzing the puncture image marker. [Effects of the Invention]

[0022] According to the present invention, the degree of freedom in the puncture position and angle is ensured, and the sensor cable can be easily managed and magnetic field fluctuations can be suppressed. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a schematic diagram of a prostate puncture assistance system. [Figure 2] FIG. 2 is a block diagram showing the functions of the puncture assistance device. [Figure 3]FIG. 1 is an explanatory diagram showing a system configuration 1 of a puncture support device. [Figure 4] FIG. 2 is an explanatory diagram showing a system configuration 2 of the puncture support device. [Figure 5] FIG. 2 is an explanatory diagram showing a system configuration 3 of the puncture support device. [Figure 6] FIG. 1A is an explanatory diagram showing the attachment of a prostate puncture support system to a patient, and FIG. 1B is an explanatory diagram showing a display in which puncture needle position information and biopsy position information are superimposed on a fusion image. [Figure 7] FIG. 10A is an explanatory diagram showing a display in which puncture support information is superimposed on a fusion image, and FIG. 10B is an explanatory diagram showing a display in which puncture support information is superimposed on an operative field image. [Figure 8] FIG. 2 is a block diagram showing the functions of a 3D image generating unit. [Figure 9] FIG. 1A is an explanatory diagram showing the display of a 3D ultrasound image, and FIG. 1B is an explanatory diagram showing the display of a 3D ultrasound image and a fusion image superimposed on a 3D rendered image of the target organ. [Figure 10] 1 is a flowchart showing the sequence of steps involved in a prostate biopsy. DETAILED DESCRIPTION OF THE INVENTION

[0024] As shown in Fig. 1, a prostate puncture system 10 includes an ultrasound probe 11, a puncture needle 12, a puncture needle image marker 13, a camera 14, a puncture assistance device 15, a display 16, and an interface 17. The puncture assistance device 15 is electrically connected to the ultrasound probe 11, the camera 14, the display 16, and the interface 17. These connections are not limited to being wired, but may be wireless. They may also be made via a network.

[0025] To perform a biopsy of suspected cancerous tissue in the prostate, the prostate puncture system 10 inserts an ultrasound probe 11 transrectally into the patient's body to obtain an ultrasound image, and then, while viewing the ultrasound image, samples of prostate tissue are obtained using a puncture needle 12. The sampled prostate tissue is then examined microscopically to confirm the presence or absence of cancer pathologically.

[0026] The prostate puncture system 10 is equipped with a puncture support device 15 that supports puncture during prostate biopsy. The position of the puncture needle is superimposed on the ultrasound image using a puncture needle image marker 13, a camera 14, and the puncture support device 15, and is displayed on a display 16. The interface 17 has a keyboard, mouse, touchpad, microphone, foot pedal, etc., and has the function of accepting input operations such as function settings.

[0027] 2, the puncture support device 15 includes an image acquisition unit 20, a position and orientation information acquisition unit 21, a puncture support unit 22, and a display control unit 23. The puncture support device 15 is provided with a program memory (not shown) that stores a specific program. A control unit (not shown) that constitutes a processor executes the specific program, thereby realizing the functions of the image acquisition unit 20, the position and orientation information acquisition unit 21, the puncture support unit 22, and the display control unit 23.

[0028] The image acquisition unit 20 includes a reference volume image acquisition unit 24, a surgical field image acquisition unit 25, and an ultrasound image acquisition unit 26. The reference volume image acquisition unit 24 acquires a reference volume image including a region of interest within the prostate, acquired before surgery. The reference volume image is obtained by an imaging test performed to predict the presence or absence of prostate cancer. The region of interest is a region that indicates tissue suspected of being prostate cancer. In recent years, MRI tests have been widely used as imaging tests for prostate cancer, but they generate a strong magnetic field. For this reason, if the patient has a metal device such as a pacemaker inside their body, CT tests using X-rays or PET tests are used, in which a diagnostic agent made of glucose labeled with a radioisotope is administered to the patient and images are acquired by detecting the radiation emitted by the diagnostic agent.

[0029] The surgical field image acquisition unit 25 acquires a surgical field image captured by an external camera 14, including the puncture needle image marker 13 attached to the puncture needle 12. The ultrasound image acquisition unit 26 acquires an ultrasound image obtained by an ultrasound probe inserted transrectally into the patient's body. The image is transmitted to the puncture support device 15 via a wired or wireless connection.

[0030] The position and orientation information acquisition unit 21 analyzes the image markers and obtains position and orientation information. The image markers may be attached not only to the puncture needle 12 but also to other surgical tools or the patient's body surface, or may be attached to a device such as a magnetic field generator. There are multiple combinations of image markers, and details will be explained in the system configuration section below.

[0031] The puncture support unit 22 includes a puncture information output unit 27 and a puncture support information superimposition unit 28. The puncture information output unit 27 outputs puncture support information for the puncture needle 12 based on the position and orientation information obtained by the position and orientation information acquisition unit 21. The display control unit 23 aligns the reference volume image and the ultrasound image based on the position and orientation information of the ultrasound probe, which will be described later, to generate a fusion image. In addition, the puncture support information output by the puncture information output unit 27 is superimposed on the image acquired by the image acquisition unit 20 to generate a superimposed image, which is displayed on the display 16. Since there are multiple images to be superimposed, details will be explained in the superimposition pattern section, which will be described later.

[0032] Three system configurations will be described below for a method of outputting puncture support information for the puncture needle 12 using a combination of image markers.

[0033] [System Configuration 1] A case will be described in which the surgical field image acquisition unit 25 acquires a surgical field image including not only the puncture needle image marker 13 but also a probe image marker attached to the ultrasound probe. For example, as shown in FIG. 3, when performing a prostate biopsy on a patient 29, the surgical field image acquisition unit 25 acquires a surgical field image including the puncture needle image marker 13 and a probe image marker 30 attached to the ultrasound probe using the camera 14. Based on the acquired surgical field image, the position and orientation information acquisition unit 21 can acquire position and orientation information of not only the puncture needle 12 but also the ultrasound probe 11. The puncture needle image marker 13 and the probe image marker 30 are markers with symbols or patterns (e.g., an arrangement of multiple black circles and black rectangles) and may be black and white or color. Furthermore, it is preferable that the appearance of the puncture needle image marker 13 and the probe image marker 30 changes depending on the shooting angle (e.g., under perspective conditions, when the markers are viewed from an angle, the gaps between the figures appear narrower than when viewed from the front). The shapes of the puncture needle image marker 13 and the probe image marker 30 may be two-dimensional or three-dimensional, and the puncture needle image marker 13 and the probe image marker 30 themselves may be attached to a two-dimensional plane or a three-dimensional curved surface. Preferably, the puncture needle image marker 13 and the probe image marker 30 are capable of acquiring three-dimensional position and orientation information of the camera 14 by reading the code or pattern of the marker from an image captured by the camera 14.

[0034] With the above configuration, the display control unit 23 aligns the reference volume image and the ultrasound image based on the position and orientation information of the ultrasound probe. Then, a fusion image is generated through the alignment. The position and orientation information of the ultrasound probe may be obtained using a magnetic field generator (described later) and a magnetic field generator image marker attached to the magnetic field generator. The magnetic field generator is used, for example, in a puncture navigation system using a magnetic system, and includes a puncture needle with a magnetic sensor built into the needle tip, an ultrasound probe with a magnetic sensor attached, and a magnetic field generator. By generating a magnetic field using the magnetic field generator, the position of the needle tip containing the sensor and the position of the ultrasound probe can be detected. Furthermore, like the puncture needle image marker, the magnetic field generator image marker can acquire three-dimensional position information using a camera.

[0035] [System Configuration 2] A case will be described in which the surgical field image acquisition unit 25 acquires a surgical field image including not only the puncture needle image marker 13 but also a magnetic field generator image marker attached to the magnetic field generator. For example, as shown in FIG. 4, when a prostate biopsy is performed on a patient 29 using a camera 14, a magnetic field generator 31, and an ultrasonic probe 33 with a magnetic sensor, the surgical field image acquisition unit 25 acquires a surgical field image using the camera 14, including the puncture needle image marker 13 attached to the puncture needle 12 and a magnetic field generator image marker 32 attached to the magnetic field generator. The position and orientation information acquisition unit 21 acquires position and orientation information based on the magnetic field generator image marker 32 included in the acquired surgical field image. Furthermore, by adding the position vector of the magnetic field generator 31 and the position vector of the ultrasonic probe 33 with a magnetic sensor, the position and orientation information acquisition unit 21 acquires position and orientation information of the ultrasonic probe relative to the camera 14. Therefore, the position and orientation information acquisition unit 21 can calculate the position and orientation information of the ultrasonic probe based on the position and orientation information obtained by the magnetic sensor of the ultrasonic probe and the position and orientation information of the magnetic field generator image marker 32 attached to the magnetic field generator. The puncture needle image marker 13 and the magnetic field generator image marker 32 are preferably AR markers.

[0036] [System Configuration 3] The following describes a case where the surgical field image acquisition unit 25 acquires not only the above-described system configuration 1 but also an surgical field image including a subject image marker attached to the body surface of the subject. Similar to the above-described puncture needle image marker and magnetic field generator image marker, the three-dimensional position information of the subject image marker can be acquired using a camera. For example, as shown in FIG. 5 , when a prostate biopsy is performed on a patient 29 using a subject image marker 34 in addition to the system configuration 1, the surgical field image acquisition unit acquires an surgical field image including a puncture needle image marker 13 attached to the puncture needle 12, a probe image marker 30 attached to the ultrasound probe 11, and a subject image marker 34 attached to the body surface of the patient 29 using the camera 14. The position and orientation information acquisition unit 21 acquires position and orientation information of the body surface from the subject image marker based on the acquired surgical field image. Based on the position and orientation information of the body surface, the display control unit 23 can correct positional deviations between the reference volume image and the ultrasound image due to the body movement of the patient 29 in the fusion image generated by the above-described registration. The subject image markers attached to the body surface of the subject are preferably AR markers.

[0037] In the above-described system configurations 1 to 3, the position and orientation information acquisition unit 21 identifies multiple markers, such as the puncture needle image marker attached to the puncture needle, the probe image marker attached to the ultrasound probe, and the subject image marker attached to the body surface, based on the color and / or shape of the image marker. Similarly, the magnetic field generator image marker attached to the magnetic field generator is also identified based on the color and / or shape of the image marker. With the above configuration, the position and orientation of each tool can be acquired simply by attaching a marker to each tool. Furthermore, by acquiring the position and orientation information, it is possible to connect the puncture needle and the ultrasound probe and eliminate the need for a template used to identify the puncture position, thereby eliminating restrictions on the puncture position and angle and enabling free puncture.

[0038] The following three patterns will be explained regarding the superimposition of the puncture support information and the image display pattern.

[0039] [Puncture assistance image pattern 1] The puncture support information related to the puncture needle is preferably a puncture guideline for puncture. The puncture support information superimposing unit 28 superimposes puncture support information including a region of interest in the reference volume image and puncture guidelines on the ultrasound image acquired by the image acquiring unit 20. As shown in FIG. 6(A), the display control unit 23 superimposes a region of interest 38 in the reference volume image and puncture guidelines 39a and 39b on a fusion image 37 obtained by aligning an ultrasound image 35 and a reference volume image 36, and displays the fusion image 37 on the display 16. The region of interest 38 in the reference volume image may be the entire image, or a specific region such as a target organ or a surrounding image may be displayed.

[0040] As shown in FIG. 6B , for example, when performing a prostate biopsy, the puncture support information superimposing unit 28 controls the display 16 to change the location where the puncture needle 12 has advanced to a puncture needle position 40, which is a solid line, when puncturing the puncture guideline 39a and 39b. The puncture support information superimposing unit 28 also controls the display 16 to display biopsy positions 41a and 41b, which are the positions where the biopsy gun's trigger is pulled. The user may change the display of the puncture needle position 40 and the biopsy positions 41a and 41b to any display. The puncture guidelines 39a and 39b are superimposed on the fusion image 37 based on a reference volume image and an anatomical model of the prostate acquired before surgery. In practice, it is preferable that the display of the puncture needle position 40 not only be changed to a solid line but also be changed in color. Although multiple puncture guidelines 39a and 39b are displayed, it is preferable that they be displayed in a way that minimizes the number of punctures.

[0041] Puncture assist image pattern 1 can be implemented in any of the above-mentioned system configurations 1 to 3. Furthermore, system configuration 3 can correct the positional deviation between the reference volume image and the ultrasound image due to the body movement of the patient 29, based on the subject image markers attached to the body surface of the subject. Therefore, by using markers attached to each instrument, system configuration 3 can track the patient's body movement in real time and correct the positional deviation of the fusion image, while facilitating cable management and suppressing the decrease in accuracy due to magnetic field fluctuations caused by metal, thereby enabling the stable display of the fusion image and puncture assist information.

[0042] [Puncture assistance image pattern 2] The puncture support information superimposing unit 28 may superimpose puncture support information, including an ultrasound image acquired by the image acquiring unit 20, a region of interest in the reference volume image, and a puncture guideline 39, on the operative field image captured by the camera 14. For example, when a prostate biopsy is performed on a patient 29 using the prostate puncture system 10 as shown in FIG. 7(A), the puncture support information superimposing unit 28 displays on the display 16 puncture support information, including an ultrasound image 35, a region of interest 38 in the reference volume image, a puncture guideline 39, and a biopsy position 41, on an operative field image 42 capturing an image of the patient 29, the ultrasound probe 11, and the puncture needle 12, as shown in FIG. 7(B). Therefore, the puncture support information can indicate where the puncture needle 12 should be inserted. Furthermore, when moving the puncture needle 12, the user can perform a prostate biopsy while viewing the puncture support information displayed and superimposed on the operative field image 42 captured by the camera 14. Furthermore, like the puncture support image pattern 1, the puncture support image pattern 2 can be implemented in any of the system configurations 1 to 3 described above.

[0043] [Puncture support image pattern 3] 8, the puncture support device 15 may include a 3D image generator 44. The 3D image generator 44 includes a 3D ultrasound image generator 45 and a 3D rendering processor 46.

[0044] The 3D ultrasound image generation unit 45 may acquire position and orientation information of the 2D ultrasound images using a probe image marker attached to the ultrasound probe, and generate a 3D ultrasound image based on the acquired position and orientation information. For example, as shown in Fig. 9(A), the position and orientation information acquisition unit 21 may acquire position and orientation information of multiple 2D ultrasound images using a probe image marker attached to the ultrasound probe, and the 3D ultrasound image generation unit 45 may generate a 3D ultrasound image 47 based on the acquired ultrasound images 35 and the position and orientation information, and display it on the display 16. Note that the 3D ultrasound image may be generated using a 3D ultrasound probe.

[0045] The 3D rendering processing unit 46 may superimpose the ultrasound image, the reference volume image, and the puncture guideline on a virtual space in which the target organ is 3D rendered. For example, as shown in FIG. 9(B), the 3D rendering processing unit 46 may perform 3D rendering of the target organ and superimpose the 3D ultrasound image generated by the 3D ultrasound image generating unit 45, the reference volume image acquired by the image acquiring unit 20, and the puncture guideline superimposed on the 3D ultrasound image by the puncture support information superimposing unit on the 3D rendered virtual space. Furthermore, the display control unit 23 may display the 3D rendering image 48 superimposed on the virtual space on the display 16. The puncture support information includes a region of interest 38 in the reference volume image, a puncture guideline 39, and a biopsy position 41, similar to the puncture support image pattern 2.

[0046] Puncture-assisting image pattern 3 superimposes a probe marker attached to an ultrasound probe or a 3D ultrasound image generated using a 2D ultrasound probe on a virtual space in which the target organ is 3D rendered. Therefore, when a 3D ultrasound image is generated using a plurality of 2D ultrasound images including a probe marker attached to an ultrasound probe, this can be implemented in either of the above-mentioned system configurations 1 or 3. Furthermore, when a 3D ultrasound image is generated using a 3D ultrasound probe, this can be implemented in any of the above-mentioned system configurations 1 to 3.

[0047] A series of processes for supporting a prostate biopsy using the prostate puncture system 10 will be described with reference to the flowchart in Fig. 10. Before a prostate biopsy, a user acquires a reference volume image including a region of interest suspected of being cancerous (step ST100). When performing a prostate biopsy, a camera is used to acquire an operative field image including markers attached to an instrument such as a puncture needle, and an ultrasound image is acquired using an ultrasound probe (step ST110). The markers in the operative field image are analyzed to acquire position and orientation information of the instrument (step ST120). The preoperatively acquired reference volume image and the ultrasound image are aligned to generate a fusion image (step ST130). Based on the acquired ultrasound image and the acquired position and orientation information, a puncture guideline is superimposed on the fusion image and displayed on a display (step ST140). The user then operates the puncture needle while observing the puncture guideline to perform a prostate cancer biopsy (step ST150).

[0048] With the above configuration, the prostate puncture system 10 allows the user to freely move the puncture needle while referring to the puncture support information in a virtual space where the target organ is 3D rendered, thereby enabling highly accurate prostate biopsy. Furthermore, with the prostate puncture system 10, by acquiring position and orientation information of each surgical instrument using markers, there is no need to use a template to connect the puncture needle and the ultrasound probe and fix the position of the puncture needle. This ensures freedom of puncture position and angle and reduces the complexity of sensor cable management and magnetic field fluctuations, allowing the user to perform puncture more freely.

[0049] In the above embodiment, the hardware configuration of processing units that perform various processes, such as the image acquisition unit 20, position and orientation information acquisition unit 21, puncture assistance unit 22, and display control unit 23, and the 3D image generation unit 44, is made up of the following various processors: The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to function as various processing units, a GPU (Graphical Processing Unit), a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor with a circuit configuration designed specifically for performing various processes.

[0050] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Also, multiple processing units may be configured with a single processor. Examples of multiple processing units configured with a single processor include, first, a configuration in which one processor is configured with a combination of one or more CPUs and software, as typified by client or server computers, and this processor functions as multiple processing units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by a system-on-chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0051] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit formed by combining circuit elements such as semiconductor elements, and the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). [Explanation of symbols]

[0052] 10 Prostate Puncture System 11 Ultrasound probe 12 Puncture needle 13. Puncture needle image marker 14 Camera 15 Puncture support device 16 Display 17 Interface 20 Image acquisition unit 21 Position and orientation information acquisition unit 22 Puncture support department 23 Display control unit 24 Reference volume image acquisition unit 25 Surgical field image acquisition unit 26 Ultrasound image acquisition unit 27 Puncture support information output section 28 Puncture support information superimposition unit 29 patients 30 Probe Image Markers 31 Magnetic Field Generator 32 Magnetic Field Generator Image Marker 33 Ultrasonic probe with magnetic sensor 34 Subject Image Marker 35 Ultrasound images 36 Reference Volume Images 37 Fusion Images 38 Areas of Interest 39 Puncture Guidelines 40 Puncture needle position 41 Biopsy Location 42 Surgical field image 44 3D image generation unit 45 3D ultrasound image generation unit 46 3D rendering processing section 47 3D ultrasound images 48 3D rendering images ST100~ST150 steps

Claims

1. a processor; The processor: a reference volume image including the prostate and a region of interest within the prostate, acquired preoperatively; a surgical field image acquired by a camera located outside the body and including at least a puncture needle image marker attached to the puncture needle; an ultrasound image acquired by an ultrasound probe; A prostate puncture assistance device that outputs puncture assistance information related to the puncture needle to an ultrasound image based on position and orientation information of the puncture needle obtained by analyzing the puncture needle image marker.

2. The prostate puncture assistance device according to claim 1 , wherein the processor aligns the reference volume image and the ultrasound image based on position and orientation information of an ultrasound probe.

3. The surgical field image further includes a probe image marker attached to the ultrasound probe, The prostate puncture assistance device according to claim 2 , wherein the processor acquires the position and orientation information of the ultrasound probe by analyzing the probe image marker included in the surgical field image.

4. The surgical field image further includes a magnetic field generator image marker attached to the magnetic field generator; the ultrasonic probe includes a magnetic sensor; 3. The prostate puncture assistance device according to claim 2, wherein the processor calculates the position and orientation information of the ultrasound probe based on the position and orientation information obtained by a magnetic sensor of the ultrasound probe and the position and orientation information of a magnetic field generator image marker attached to the magnetic field generator.

5. The surgical field image further includes a subject image marker attached to a body surface of the subject, The prostate puncture assistance device according to claim 3 , wherein the processor corrects the alignment between the reference volume image and the ultrasound image by acquiring position and orientation information of a body surface from the subject image marker.

6. The prostate puncture support device according to claim 5, wherein the processor identifies the puncture needle image marker attached to the puncture needle, the probe image marker attached to the ultrasound probe, and the subject image marker attached to the body surface based on the color and / or shape of the image marker.

7. The prostate puncture assistance device according to claim 5, wherein the processor identifies the puncture needle image marker attached to the puncture needle, the magnetic field generator image marker attached to the magnetic field generator, and the subject image marker attached to the body surface based on the color and / or shape of the image marker.

8. The prostate puncture assistance device according to claim 1 , wherein the puncture assistance information regarding the puncture needle is a puncture guideline for puncture.

9. The prostate puncture assistance device according to claim 8 , wherein the processor may superimpose puncture assistance information including a region of interest in the reference volume image and the puncture guideline on the ultrasound image.

10. 2. The prostate puncture assistance device according to claim 1, wherein the region of interest in the acquired reference volume image may be the entire image, or a specific region such as a target organ or a peripheral image.

11. The prostate puncture assistance device according to claim 8 , wherein the processor may superimpose puncture assistance information including the ultrasound image, a region of interest in the reference volume image, and the puncture guideline on the operative field image.

12. The ultrasound image may be a 3D ultrasound image; The prostate puncture assistance device according to claim 3 , wherein the processor acquires position and orientation information of a 2D ultrasound image using the probe image marker attached to an ultrasound probe, and generates a 3D ultrasound image based on the acquired position and orientation information.

13. The prostate puncture assistance device according to claim 8 , wherein the processor may superimpose the ultrasound image, the reference volume image, and the puncture guideline on a virtual space in which the target organ is 3D rendered.

14. A method for operating a prostate puncture assistance device, comprising: a reference volume image including the prostate and a region of interest within the prostate, acquired preoperatively; A step of acquiring an operative field image acquired by a camera located outside the body and including at least a puncture needle image marker attached to the puncture needle; and outputting puncture assistance information related to the puncture needle based on position and orientation information of the puncture needle obtained by analyzing the puncture needle image marker.

15. A program for a prostate puncture assistance device On the computer, a reference volume image including the prostate and a region of interest within the prostate, acquired preoperatively; A function of acquiring an image of a surgical field acquired by a camera located outside the body and including at least a puncture needle image marker attached to the puncture needle; and a program for a prostate puncture assistance device for realizing a function of outputting puncture assistance information related to the puncture needle based on position and orientation information of the puncture needle obtained by analyzing the puncture needle image marker.

Images