Surgical system and surgical method

Abstract

A surgical system (1) has an endoscope (3) inserted into a body cavity, capable of acquiring an endoscope image of a surface of an object tissue; an ultrasonic probe (5) inserted into the body cavity, capable of acquiring an ultrasonic tomographic image of the object tissue; a treatment instrument inserted into the body cavity; a display device (11) capable of displaying the ultrasonic tomographic image acquired by the ultrasonic probe; and a control device (20) having a memory and a processor connected to the endoscope, the ultrasonic probe, and the display device, wherein the processor, when the ultrasonic probe is inserted into the body cavity and the ultrasonic tomographic image is acquired, detects the position of the ultrasonic probe from the endoscope image, stores the ultrasonic tomographic image in the memory in correspondence with the position of the ultrasonic probe, in a state where the treatment instrument is inserted into the body cavity, detects the position of the treatment instrument from the endoscope image, and reads out the ultrasonic tomographic image stored in the memory in accordance with the position of the treatment instrument and displays it in the display device.

Description

Technical Field

[0001] This invention relates to surgical systems and surgical methods. Background Technology

[0002] Endoscopic surgical systems are known to be used for treatment by inserting an endoscope and treatment instruments into a patient's body cavity via a cannula (e.g., see Patent Document 1).

[0003] The laparoscope is inserted through one cannula, and treatment instruments are inserted through other cannulas to treat the tissues within the body cavity.

[0004] In this case, before treatment, an ultrasound probe is inserted instead of a treatment instrument to obtain an ultrasound tomographic image of the area to be treated, in order to understand the internal structure of the tissue. After confirming the internal structure using the ultrasound tomographic image, a treatment instrument is inserted instead of the ultrasound probe to perform treatments such as tissue removal.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 8-275958 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] However, when it is necessary to confirm the internal structure of the tissue while performing treatment, it is necessary to repeatedly confirm and treat the internal structure of the tissue by inserting and removing the ultrasonic probe and treatment instruments, which may result in inefficient treatment.

[0010] The purpose of this invention is to provide a surgical system and surgical method that can efficiently confirm the accurate internal structure of the site to be treated without inserting or removing the ultrasound probe and the treatment instrument during treatment based on the treatment instrument.

[0011] Methods for solving problems

[0012] One aspect of the present invention is a surgical system comprising: an endoscope inserted into a body cavity to acquire endoscopic images of the surface of a tissue; an ultrasound probe inserted into the body cavity to acquire ultrasound tomographic images of the tissue; a treatment device inserted into the body cavity; a display device capable of displaying the ultrasound tomographic images acquired by the ultrasound probe; and a control device having a memory and a processor connected to the endoscope, the ultrasound probe, and the display device, wherein, when the ultrasound probe is inserted into the body cavity and acquires the ultrasound tomographic image, the processor detects the position of the ultrasound probe based on the endoscopic image acquired by the endoscope, stores the ultrasound tomographic image corresponding to the position of the ultrasound probe in the memory, and when the treatment device is inserted into the body cavity, detects the position of the treatment device based on the endoscopic image acquired by the endoscope, reads the ultrasound tomographic image stored in the memory based on the detected position of the treatment device, and displays the read ultrasound tomographic image on the display device.

[0013] Another aspect of the present invention is a surgical method comprising: a tomographic image storage step; and a treatment step, wherein in the tomographic image storage step, an endoscope and an ultrasound probe are inserted into a body cavity to obtain an endoscopic image of the surface of a target tissue, and multiple ultrasound tomographic images of the target tissue are obtained by the ultrasound probe; the multiple ultrasound tomographic images are stored in a storage unit in correspondence with the positions of the ultrasound probe detected by processing the endoscopic images obtained at each ultrasound tomographic image; in the treatment step, a treatment device is inserted into the body cavity instead of the ultrasound probe to obtain the endoscopic image of the surface of the target tissue, and the endoscopic image is processed to detect the position of the treatment device; based on the detected position of the treatment device, the ultrasound tomographic image stored in the storage unit is read out and the read-out ultrasound tomographic image is displayed.

[0014] Another aspect of the invention is a surgical system comprising: an endoscope for acquiring endoscopic images of the surface of a tissue; a storage unit for storing multiple ultrasound tomographic images of the tissue acquired by an ultrasound probe inserted into a body cavity, corresponding to the position of the ultrasound probe detected by processing the endoscopic images at the time each ultrasound tomographic image was acquired; a position detection unit for processing the endoscopic images to detect the position of a treatment device inserted into the body cavity in place of the ultrasound probe; a tomographic image readout unit for reading the ultrasound tomographic images stored in the storage unit based on the position of the treatment device detected by the position detection unit; and a display unit for displaying the ultrasound tomographic images read out by the tomographic image readout unit.

[0015] According to this method, multiple ultrasound tomographic images of the target tissue obtained by inserting an ultrasound probe into the body cavity are stored in a storage unit in correspondence with the position of the ultrasound probe detected by processing the endoscopic images obtained when each ultrasound tomographic image is acquired. Furthermore, when treating the target tissue using a treatment instrument inserted into the body cavity, the position of the treatment instrument detected by processing the endoscopic images obtained by the endoscope is detected by a position detection unit.

[0016] Furthermore, based on the detected position of the treatment instrument, the ultrasound tomographic image stored in the storage unit is read out by the tomographic image readout unit and displayed on the display unit. Thus, when the treatment instrument is moved within the body cavity, an ultrasound tomographic image of the target tissue at the position corresponding to the position of the treatment instrument is displayed on the display unit. As a result, the surgeon can confirm the precise internal structure of the treatment site even without using an ultrasound probe when performing treatment with the instrument.

[0017] In the above method, the position of the ultrasonic probe can also be in a direction that intersects with the scanning plane of the ultrasonic waves of the ultrasonic probe.

[0018] According to this structure, ultrasound tomographic images can be stored in a way that allows for easy readout. That is, when an ultrasound probe is positioned on the surface of a tissue and operated, an ultrasound tomographic image along the scanning plane of the ultrasound waves is acquired. By translating the ultrasound probe in a direction intersecting the scanning plane, multiple ultrasound tomographic images arranged in that direction can be acquired. Therefore, by storing the position of the ultrasound probe in only the direction intersecting the scanning plane of the ultrasound waves and corresponding it to the ultrasound tomographic image, the position of the ultrasound tomographic image can be easily determined.

[0019] Alternatively, in the above method, the position of the treatment device can be the position of its front end.

[0020] When using an endoscope to observe the surface of tissue within a body cavity while bringing instruments close to the tissue for treatment, the surgeon is most concerned with the position of the tip of the instrument, and therefore, it is most suitable to read the ultrasound tomographic image at that position.

[0021] According to this structure, when the treatment instrument is moved on the surface of the target tissue, an ultrasonic tomographic image at the position of the tip of the treatment instrument is read and displayed, thus enabling a more accurate identification of the internal structure of the site to be treated.

[0022] Alternatively, in the above method, the position detection unit may set a reference point in the endoscopic image and detect the position of the ultrasonic probe and the position of the treatment device by calculating the distance from the reference point.

[0023] According to this structure, the position of the treatment instrument can be easily detected based on the endoscopic images used during the treatment of the object tissue.

[0024] Furthermore, in the above-described manner, the display unit may display the endoscopic image and the ultrasound tomography image, and an indicator showing the position of the ultrasound tomography image may be overlaid on the endoscopic image.

[0025] Based on this structure, the internal structure of the tissue being represented by the displayed ultrasound tomography image can be clearly identified at a glance by the indicator superimposed on the endoscopic image.

[0026] Alternatively, in the above-described manner, the indicator display may be a straight line along the scanning plane of the ultrasound.

[0027] Based on this structure, the location of ultrasound tomography images can be displayed simply and accurately on endoscopic images.

[0028] Another aspect of the present invention is a surgical method comprising: a tomographic image storage step; and a treatment step, wherein in the tomographic image storage step, an endoscope and an ultrasound probe are inserted into a body cavity to obtain an endoscopic image of the surface of a target tissue, and multiple ultrasound tomographic images of the target tissue are obtained by the ultrasound probe; the multiple ultrasound tomographic images are stored in a storage unit in correspondence with the positions of the ultrasound probe detected by processing the endoscopic images obtained at each ultrasound tomographic image; in the treatment step, a treatment device is inserted into the body cavity instead of the ultrasound probe to obtain the endoscopic image of the surface of the target tissue, and the endoscopic image is processed to detect the position of the treatment device; based on the detected position of the treatment device, the ultrasound tomographic image stored in the storage unit is read out and displayed.

[0029] Invention Effects

[0030] According to the present invention, the following effect is achieved: the accurate internal structure of the site to be treated can be efficiently confirmed without the need to insert or remove the ultrasonic probe and treatment device. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating an application example of a surgical system according to an embodiment of the present invention, showing the state in which an ultrasound probe is inserted into a body cavity.

[0032] Figure 2 It means Figure 1 A block diagram of the surgical system.

[0033] Figure 3 It is in Figure 1 A diagram illustrating the position of the ultrasound scanning plane in endoscopic images obtained from a surgical system.

[0034] Figure 4 It means to substitute Figure 1 A schematic diagram showing the state in which a knife is inserted into a body cavity using an ultrasonic probe.

[0035] Figure 5 It means in Figure 1 An example of an image displayed during the procedure performed with a scalpel in a surgical system.

[0036] Figure 6 It means that when using a knife... Figure 5 An example of an image displayed in a state where the state has been moved.

[0037] Figure 7 It means Figure 5 Other examples of images.

[0038] Figure 8 This is a flowchart illustrating a surgical method according to one embodiment of the present invention.

[0039] Figure 9 Yes Figure 8 The flowchart illustrates the storage phase.

[0040] Figure 10 Yes Figure 8 The flowchart illustrates the handling phase. Detailed Implementation

[0041] Hereinafter, a surgical system 1 and a surgical method according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0042] like Figure 1 As shown, the surgical system 1 of this embodiment is used in a procedure in which an endoscope 3 is inserted into the body cavity via a cannula 2 that penetrates the patient’s body wall B, and the procedure is performed while observing the surface of the liver (the target tissue) C through the endoscope 3.

[0043] In this procedure, an ultrasound probe 5 is inserted into the body cavity via another cannula 4, and while moving it in one direction (in the direction of the arrow) on the surface of liver C, an ultrasound tomographic image of liver C is obtained (see reference). Figure 5 )G2.

[0044] Furthermore, during this procedure, after obtaining the ultrasound tomographic image G2 using the ultrasound probe 5, the ultrasound probe 5 is removed from the cannula 4 and replaced, for example, by inserting a scalpel (disposal instrument) 6 for removing the lesion of liver C into the body cavity. Any disposal instrument other than the scalpel 6 may also be used.

[0045] like Figure 1 As shown, the surgical system 1 of this embodiment includes an endoscope 3, a control device 20, and a display device (display unit) 11. The control device 20 is connected to the endoscope 3 and the ultrasound probe 5.

[0046] like Figure 2 As shown, the control device 20 includes a storage unit 7, a position detection unit 8, a control unit (tomographic image reading unit) 9, and an image processing unit 10.

[0047] Storage unit 7 is a storage device such as a memory. Position detection unit 8, control unit 9, and image processing unit 10 are composed of processor 30. In addition, display unit 11 is a monitor.

[0048] In the storage unit 7, the position of the scanning plane of the ultrasound probe 5 when the ultrasound probe 5 is disposed on the surface of the liver C is stored in relation to the ultrasound tomographic image G2 obtained at that time.

[0049] During the stage of storing the ultrasound tomographic image G2 into the storage unit 7, the position detection unit 8 processes the endoscope image G1 obtained by the endoscope 3 at a predetermined frame rate, thereby sequentially calculating the position of the ultrasound scanning plane as the distance of the position relative to the reference point O within the endoscope image G1.

[0050] In the stage of obtaining ultrasound tomographic images G2, such as Figure 2 As shown by the dashed line, the control unit 9 can be connected to the ultrasonic probe 5. Furthermore, the control unit 9 stores the position of the ultrasonic scanning plane detected by the position detection unit 8 in the storage unit 7, corresponding to the ultrasonic tomographic image G2 acquired at this time.

[0051] For example, such as Figure 3 As shown, the reference point O is set at the center of the endoscopic image G1. Alternatively, more than one feature point can be extracted from the endoscopic image G1, and any position determined for the extracted feature point can be used as the reference point O.

[0052] Additionally, for example, such as Figure 3 As shown, when the ultrasound probe 5 present in the endoscopic image G1 is a rod with a certain width, the scanning plane of the ultrasound is determined as a straight line L extending along the length direction from the center of its width direction. The position of the scanning plane can be determined simply by calculating the distance from the reference point O to the straight line L.

[0053] like Figure 3 As shown, the control unit 9 sets the x-axis in a direction parallel to the aforementioned straight line L, with the set reference point O as the origin, and sets the y-axis in a direction orthogonal to the x-axis.

[0054] Regarding the acquisition of ultrasonic tomographic images G2 using ultrasonic probe 5, the ultrasonic probe 5 is positioned in the width direction, i.e., as... Figure 1 During the movement in the direction intersecting the scanning plane of the ultrasound, as shown by the middle arrow, images are acquired sequentially at a predetermined frame rate. Each time an ultrasound tomographic image G2 is acquired, the distance yn (y-coordinate) from the origin, i.e., the reference point O, to the y-axis of the scanning plane is calculated, and the distance yn is stored in the storage unit 7 in correspondence with the ultrasound tomographic image G2.

[0055] like Figure 4As shown, during the stage where the ultrasound probe 5 is replaced and the knife 6 is inserted into the body cavity to treat the liver C, the position detection unit 8 processes the endoscopic image G1 to detect the position of the knife 6 inserted into the body cavity. Specifically, in the endoscopic image G1 obtained by the endoscope 3, the position of the tip of the knife 6 in the endoscopic image G1 is extracted. Furthermore, as... Figure 5 As shown, in Figure 3 In the coordinate system set in the system, the y-coordinate ym of the extracted front end position of the detection tool 6 is used as the position of the tool 6.

[0056] Furthermore, the position of the blade 6 detected by the position detection unit 8 and the endoscopic image G1 currently acquired by the endoscope 3 are input to the control unit 9. The control unit 9 reads the ultrasonic tomographic image G2 stored in the storage unit 7 corresponding to the input position of the blade 6. Specifically, the ultrasonic tomographic image G2 acquired by the ultrasonic probe 5 at the same position as the position of the blade 6 detected by the position detection unit 8 is read.

[0057] In addition, such as Figure 5 As shown, the control unit 9 transmits the read ultrasound tomographic image G2 to the image processing unit 10. The image processing unit 10 generates a composite image by juxtaposing the ultrasound tomographic image G2 transmitted from the control unit 9 with the current endoscopic image G1 of the surface of the liver C input from the endoscope 3, and transmits it to the display unit 11. The display unit 11 displays the transmitted composite image.

[0058] The surgical method of this embodiment with such configuration will be described below.

[0059] Regarding the surgical method of this embodiment, such as Figure 8 As shown, the endoscope 3 is inserted into the body cavity via the cannula 2 (step S1), and the ultrasound probe 5 is inserted into the body cavity via the cannula 4 (step S2).

[0060] Then, by using the endoscopic image G1 obtained by the endoscope 3 and the ultrasonic tomographic image G2 obtained by the ultrasonic probe 5, the position information of the ultrasonic probe 5 detected by the position detection unit 8 and the ultrasonic tomographic image G2 are matched and stored in the storage unit 7 (tomographic image storage step S3), and the ultrasonic probe 5 is pulled out of the body cavity (step S4).

[0061] Next, in order to cut open the lesion of liver C, the knife 6 is inserted into the body cavity through the cannula 4 (step S5), and the endoscopic image G1 and the ultrasound tomographic image G2 are displayed in association on the display device 11 (treatment step S6), and treatment is performed on liver C. After treatment is completed, the knife 6 is withdrawn from the body cavity (step S7), and the endoscope 3 is withdrawn (step S8), completing the treatment.

[0062] More specifically, in the tomographic image storage step S3, such as Figure 9 As shown, an endoscope image G1 is obtained through the endoscope 3 (step S31), and a reference point O is set at the center position of the obtained endoscope image G1 (step S32).

[0063] Then, an ultrasound tomographic image G2 of the liver C is obtained by ultrasound probe 5 (step S33). Whenever an ultrasound tomographic image G2 is obtained, the position information of ultrasound probe 5 relative to reference point O within the frame is calculated by position detection unit 8, that is, the distance yn from reference point O to the scanning plane in the y-axis direction (step S34).

[0064] Then, the distance yn from the reference point O in the xy coordinate system set by the control unit 9 in the endoscopic image G1 is matched with the ultrasonic tomographic image G2 and stored in the storage unit 7 (step S35). It is determined whether the storage operation is over (step S36). If the storage operation continues, the process from step S31 is repeated. If the storage operation is over, step S4 is executed.

[0065] Furthermore, in processing step S6, such as Figure 10 As shown, an endoscopic image G1 is acquired using the endoscope 3 (step S61), and the position detection unit 8 processes the acquired endoscopic image G1 to detect the tip position of the blade 6 (step S62). Then, the control unit 9 reads the ultrasonic tomographic image G2 corresponding to the tip position of the blade 6 from the storage unit 7 (step S63), and the image processing unit 10 displays the endoscopic image G1 and the ultrasonic tomographic image G2 in association on the display unit 11 (step S64). Then, it is determined whether the treatment action has ended (step S65). If the treatment action continues, the process from step S61 is repeated; if the treatment action has ended, step S7 is executed.

[0066] Therefore, if the position of the tool 6 is detected sequentially using the position detection unit 8 at a predetermined sampling period, then... Figure 5 and Figure 6 As shown, on the endoscopic image G1, whenever the blade 6 moves, the display unit 11 updates and displays an ultrasound tomographic image G2 along the scanning plane passing through the tip of the blade 6. Therefore, it has the advantage that, when making an incision with the blade 6, the accurate internal structure of the incision site can be quickly confirmed even without using the ultrasound probe 5. Thus, it eliminates the need to alternately insert and remove the ultrasound probe 5 and the blade 6 for observation and treatment, allowing for a convenient and accurate incision of the liver C.

[0067] Furthermore, in this embodiment, the image processing unit 10 generates a composite image by arranging the endoscopic image G1 and the ultrasound tomographic image G2 side by side. However, it is also possible to display the endoscopic image G1 and the ultrasound tomographic image G2 on different screens instead.

[0068] Alternatively, in this embodiment, the image processing unit 10 may receive information about the position of the ultrasound computed tomography image G2 from the control unit 9, such as... Figure 7 As shown, a composite image is generated by overlaying the straight line LA (indicator display) representing the scanning plane of ultrasound onto the endoscopic image G1. This has the advantage that the direction of the ultrasound tomographic image G2, displayed alongside the endoscopic image G1, relative to the liver C displayed in the endoscopic image G1 can be clearly visually confirmed via the straight line LA. That is, if a lesion is found in the ultrasound tomographic image G2, the lesion can be reached more accurately by cutting along the straight line LA.

[0069] In this embodiment, the control unit 9 sets an orthogonal coordinate system xy with the center position of the endoscopic image G1 as the reference point O. However, it is also possible to set the y coordinate in any direction intersecting the x-direction. This allows for a unique correspondence between the ultrasonic tomographic image G2 and the tip position of the blade 6.

[0070] Furthermore, by setting the reference point O based on the feature points of the endoscope image G1, even if the endoscope 3 moves during the procedure, the reference point O in the endoscope image G1 before and after the movement can be kept consistent.

[0071] Alternatively, the reference point O in the endoscopic image G1 before and after the movement of the endoscope 3 can be made consistent by detecting the amount of movement of the endoscope 3.

[0072] Therefore, even if the field of view of the endoscope 3 when acquiring ultrasonic tomographic images through the ultrasonic probe 5 is different from the field of view of the endoscope 3 when performing treatment through the knife 6, the ultrasonic tomographic image corresponding to the position of the knife 6 can still be read.

[0073] In addition to the case where the ultrasonic probe 5 and the knife 6, which serves as the treatment instrument, are inserted and removed through the same cannula 4, this method can also be applied to cases where both can be inserted through different cannula 4. Even in this case, it has the advantage that it is not necessary to alternately acquire the ultrasonic tomographic image G2 based on the ultrasonic probe 5 and perform treatment based on the knife 6, which serves as the treatment instrument.

[0074] Label Explanation

[0075] 1: Surgical system;

[0076] 3: Endoscope;

[0077] 5: Ultrasonic probe;

[0078] 6: Knife (processing tool);

[0079] 7: Storage section;

[0080] 8: Position Detection Unit;

[0081] 9: Control unit (tomographic image reading unit);

[0082] 11: Display device (display unit);

[0083] 30: Processor;

[0084] C: Liver (the target tissue);

[0085] G1: Endoscopic image;

[0086] G2: Ultrasonic tomographic image;

[0087] O: Reference point;

[0088] S3: Steps for storing tomographic images;

[0089] S6: Handling steps.

Claims

1. A surgical system having: an endoscope inserted into a body cavity, capable of taking an endoscope image of a surface of an object tissue; an ultrasonic probe inserted into the body cavity, capable of taking an ultrasonic tomographic image of the object tissue; a treatment instrument inserted into the body cavity; a display device capable of displaying the ultrasonic tomographic image taken by the ultrasonic probe; and a control device having a memory and a processor connected to the endoscope, the ultrasonic probe and the display device, wherein the processor detects a position of the ultrasonic probe from the endoscope image taken by the endoscope when the ultrasonic probe is inserted into the body cavity and takes the ultrasonic tomographic image, stores the ultrasonic tomographic image in correspondence with the position of the ultrasonic probe in the memory, detects a position of the treatment instrument from the endoscope image taken by the endoscope in a state where the treatment instrument is inserted into the body cavity, reads out the ultrasonic tomographic image stored in correspondence with the detected position of the treatment instrument stored in the memory, and causes the read out ultrasonic tomographic image to be displayed on the display device.

2. The surgical system according to claim 1, wherein the processor detects a position of the ultrasonic probe in a direction intersecting with a scanning plane of an ultrasonic wave from the endoscope image taken by the endoscope when the ultrasonic probe is inserted into the body cavity and takes the ultrasonic tomographic image.

3. The surgical system according to claim 2, wherein the processor sets a reference point on the endoscope image, and detects the position of the ultrasonic probe from a distance between the reference point and a scanning plane of an ultrasonic wave.

4. The surgical system according to claim 1, wherein the processor detects a position of a tip of the treatment instrument from the endoscope image taken by the endoscope in a state where the treatment instrument is inserted into the body cavity.

5. The surgical system according to claim 4, wherein the processor sets a reference point on the endoscope image, and detects the position of the treatment instrument from a distance between the reference point and a tip of the treatment instrument.

6. The surgical system according to claim 1, wherein the processor sets a reference point on the endoscope image, and detects the positions of the ultrasonic probe and the treatment instrument by calculating a distance between the reference point and the ultrasonic probe and a distance between the reference point and the treatment instrument.

7. The surgical system according to claim 6, wherein the processor causes the endoscope image and the ultrasonic tomographic image to be displayed on the display device, and causes an indication display indicating the position of the ultrasonic probe when the ultrasonic tomographic image displayed on the display device is taken to be displayed on the endoscope image.

8. The surgical system according to claim 7, wherein ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The processor displays a straight line along a scanning plane of the ultrasound waves as the indication on the display device.

9. The surgical system according to claim 1, wherein The processor reads out the ultrasonic tomographic image taken by the ultrasonic probe at the same position as the detected position of the treatment instrument from the memory.

10. An image processing method, comprising: a tomographic image storage step; and an image processing step, In the tomographic image storage step, a plurality of ultrasonic tomographic images of a subject tissue taken by an ultrasonic probe are stored in association with positions of the ultrasonic probe detected by processing an endoscope image of a surface of the subject tissue taken by an endoscope at a time each of the ultrasonic tomographic images is taken in a storage section, In the image processing step, the endoscope image of the surface of the subject tissue is taken, and the position of a treatment instrument is detected by processing the endoscope image, the ultrasonic tomographic image stored in the storage section in association with the detected position of the treatment instrument is read out, and the read-out ultrasonic tomographic image is displayed.

11. The image processing method according to claim 10, wherein The image processing method further includes detecting the position of the ultrasonic probe in a direction intersecting a scanning plane of an ultrasonic wave of the ultrasonic probe based on the endoscope image taken by the endoscope, thereby taking the position of the ultrasonic probe.

12. The image processing method according to claim 11, wherein The image processing method further includes setting a reference point on the endoscope image, and detecting the position of the ultrasonic probe based on a distance between the reference point and an operation plane of the ultrasonic probe, thereby taking the position of the ultrasonic probe.

13. The image processing method according to claim 11, wherein The image processing method further includes setting a reference point on the endoscope image, and taking a distance between the reference point and the treatment instrument as the position of the treatment instrument based on the endoscope image.

14. The image processing method according to claim 11, wherein The image processing method further includes setting a reference point on the endoscope image, and detecting the positions of the ultrasonic probe and the treatment instrument by calculating a distance between the reference point and the ultrasonic probe and a distance between the reference point and the treatment instrument.

15. A surgical system, comprising: an endoscope that takes an endoscope image of a surface of a subject tissue; a storage section that stores a plurality of ultrasonic tomographic images of the subject tissue taken by an ultrasonic probe inserted into a body lumen in association with positions of the ultrasonic probe detected by processing the endoscope image at a time each of the ultrasonic tomographic images is taken; a position detection section that detects a position of a treatment instrument inserted into the body lumen by processing the endoscope image; a tomographic image readout section that reads out the ultrasonic tomographic image stored in the storage section in association with the detected position of the treatment instrument; and a display section that displays the read-out ultrasonic tomographic image. a display section that displays the ultrasonic tomographic image read out by the tomographic image readout section.

16. The surgical system according to claim 15, wherein the position of the ultrasonic probe is a position in a direction intersecting a scanning plane of ultrasonic waves of the ultrasonic probe.

17. The surgical system according to claim 15, wherein the position of the treatment instrument is a position of a distal end of the treatment instrument.

18. The surgical system according to claim 15, wherein the position detection section sets a reference point in the endoscope image, and detects the position of the ultrasonic probe and the position of the treatment instrument by calculating distances of the ultrasonic probe and the treatment instrument from the reference point, respectively.

19. The surgical system according to claim 18, wherein the display section displays the endoscope image and the ultrasonic tomographic image, and displays an indication display indicating the position of the ultrasonic tomographic image on the endoscope image in superposition.

20. The surgical system according to claim 19, wherein the indication display is a straight line along a scanning plane of ultrasonic waves.

Images