Endoscope system, control device, control method, and program

Abstract

To provide an endoscope system, a control device, a control method, and a program that can display an endoscopic image that allows an operator who operates a treatment instrument while observing an endoscopic image to easily and intuitively perform a treatment.SOLUTION: A processor acquires a first position of a first port P1 through which a first treatment instrument operated by an operator is inserted into a subject, a second position of a second port P2 through which a second treatment instrument operated by the operator is inserted into the subject, and a third position of a target of interest T1 inside the subject. On the basis of the acquired first, second, and third positions, an endoscope system rotates an image so that a direction of a second vector V2 in which a first vector V1 indicating a vertical direction of the target of interest T1 as seen from the operator is projected onto a plane C4 orthogonal to a visual axis B of an endoscope and the vertical direction of the image displayed on a display device have a predetermined relative angle.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an endoscope system, a control device, a control method, and a program. [Background technology]

[0002] BACKGROUND ART A surgical robot is known that controls the rotation angle of an insertion portion of a surgical endoscope about its longitudinal axis (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent Application Publication No. 2015 / 0320514 Summary of the Invention [Problem to be solved by the invention]

[0004] The surgical robot in Patent Document 1 uses data generated by modeling the environment inside the abdominal cavity in advance to control the rotation angle of the insertion part of the endoscope. However, modeling the entire environment inside the abdominal cavity requires a huge amount of processing, and control becomes difficult when the region of interest to be treated is placed in an environment that is not anticipated.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an endoscopic system, a control device, a control method, and a program that can display endoscopic images that make it easy for an operator to operate a treatment tool while observing an endoscopic image to perform treatment in a simple and intuitive manner. [Means for solving the problem]

[0006] One aspect of the present invention is an endoscopic system comprising at least one processor, an endoscope, a moving device that holds the endoscope and adjusts the position and attitude of the endoscope based on control signals from the processor, and a display device that displays images acquired by the endoscope, wherein the processor acquires a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject, and based on the acquired first, second, and third positions, rotates the image so that a predetermined relative angle exists between the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the operator onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device.

[0007] Another aspect of the present invention is an endoscopic system comprising at least one processor, an endoscope, a moving device that holds the endoscope and adjusts the position and attitude of the endoscope based on control signals from the processor, and a display device that displays images acquired by the endoscope, wherein the processor acquires a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject, and controls the moving device and / or the endoscope so as to bring the visual axis of the endoscope closer to a line connecting the third position and a fourth position between the first position and the second position.

[0008] Another aspect of the present invention is a control device for an endoscopic system comprising an endoscope, a moving device for holding the endoscope and adjusting the position and attitude of the endoscope, and a display device for displaying images acquired by the endoscope, the control device comprising at least one processor, the processor acquiring a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject, and based on the acquired first, second, and third positions, rotating the image so that a predetermined relative angle is formed between the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the operator onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device.

[0009] Another aspect of the present invention is a control method for an endoscopic system comprising an endoscope, a moving device that holds the endoscope and adjusts the position and attitude of the endoscope, and a display device that displays images acquired by the endoscope, the control method including: acquiring a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject; and rotating the image based on the acquired first, second, and third positions so that a predetermined relative angle is formed between the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the operator onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device.

[0010] Another aspect of the present invention is a control program for an endoscopic system comprising an endoscope, a moving device for holding the endoscope and adjusting the position and attitude of the endoscope, and a display device for displaying images acquired by the endoscope, the control program causing a computer to acquire a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject, and to rotate the image based on the acquired first, second, and third positions so that a predetermined relative angle is formed between the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the operator onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is an overall configuration diagram of an endoscope system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the endoscope system of FIG. 1. [Figure 3] 2 is a schematic diagram illustrating laparoscopic surgery on a patient using the endoscope system of FIG. 1. [Figure 4] 2 is a partial longitudinal cross-sectional view showing an endoscope and a trocar of the endoscope system of FIG. 1. FIG. [Figure 5] 2 is a flowchart showing a control method for the endoscope system of FIG. 1. [Figure 6] FIG. 6 is a schematic diagram illustrating a control method for the endoscope system of FIG. 5. [Figure 7A] FIG. 10 is a diagram in which an endoscopic image when the up-down direction of the imaging element of the endoscope is arranged vertically is superimposed on a second vector. [Figure 7B] 7B is a diagram in which the endoscopic image in FIG. 7A is rotated so that the up-down direction of the imaging element of the endoscope coincides with a second vector. [Figure 8] 1. FIG. 4 is a schematic diagram illustrating a case where an object of interest is changed in the endoscope system of FIG. [Figure 9A] This is an endoscopic image of an object of interest positioned closer to the surgeon than in Figure 7A, and is a diagram in which the endoscopic image when the up and down direction of the imaging element of the endoscope is positioned vertically is superimposed on a second vector. [Figure 9B] 9B is a diagram in which the endoscopic image in FIG. 9A is rotated so that the up-down direction of the imaging element of the endoscope coincides with a second vector. [Figure 10] 6 is a flowchart showing a modification of the control method of FIG. 5. [Figure 11] 6 is a flowchart showing another modified example of the control method of FIG. 5. [Figure 12] 1. FIG. 4 is a diagram illustrating a modified example of the endoscope system of FIG. [Figure 13] FIG. 2 is a diagram illustrating a modified example of the endoscope system of FIG. 1, which includes an oblique-viewing endoscope. [Figure 14] FIG. 2 is a diagram illustrating a modified example of the endoscope system of FIG. 1, which includes an endoscope having a bending portion. DETAILED DESCRIPTION OF THE INVENTION

[0012] An endoscope system 1, a control device 5, a control method, and a program according to an embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the endoscopic system 1 of this embodiment is used in surgery, such as laparoscopic surgery, in which an endoscope 2 and a treatment tool 7 are inserted into the body of a patient (subject) D, and a target area such as an affected area is treated with the treatment tool 7 while the treatment tool 7 is observed through the endoscope 2.

[0013] 1 and 2, the endoscope system 1 includes an endoscope 2, a moving device 3 that holds and moves the endoscope 2, an operating device 4 that is operated by a user, a control device 5 that controls the moving device 3 based on an operating signal from the operating device 4, and a display device 6. The endoscope 2 is a direct-view rigid endoscope that includes a cylindrical lens barrel portion (insertion portion) 2a that extends along a longitudinal axis A and has a visual axis B that coincides with the longitudinal axis A.

[0014] The endoscope 2 is inserted into the body of the patient D together with one or more treatment tools 7, and is equipped with an imaging element 2c that acquires an endoscopic image (image) C including the one or more treatment tools 7. The endoscope 2 transmits the endoscopic image C acquired by the imaging element 2c to the display device 6 via the control device 5. The imaging element 2c is, for example, a three-dimensional camera provided at the base end of the endoscope 2, and acquires a stereo image as the endoscopic image C. The display device 6 is any display such as a liquid crystal display, an organic EL display, or the like. The surgeon E operates the treatment tool 7 while observing the endoscopic image C displayed on the display device 6.

[0015] 3 and 4, the endoscope 2 and the treatment tool 7 are inserted into the body of a patient D, for example, into the abdominal cavity, via a trocar 8. The trocar 8 is a tubular instrument attached to holes P0, P1, and P2 formed in the abdominal wall, and can swing around pivot points near the holes P0, P1, and P2.

[0016] The moving device 3 is a multi-joint robot arm having at least six joints M1 to M6. The moving device 3 is provided with a drive mechanism 3a at the tip thereof, which holds the endoscope 2, as joint M6, for rotating the endoscope 2 around the longitudinal axis A. The position and posture of the endoscope 2 are adjusted three-dimensionally by the operation of the moving device 3. The moving device 3 is provided with angle sensors 3b that detect the angles of each of the joints M1 to M6. The angle sensors 3b are, for example, encoders, potentiometers, or Hall sensors provided at each of the joints M1 to M6.

[0017] 2, the control device 5 includes at least one processor 5a, at least one memory 5b, a storage unit (memory) 5c, an input interface 5d, and an output interface 5e. The control device 5 controls the endoscope 2, the moving device 3, and the endoscopic image C displayed on the display device 6. The control device 5 is also connected to the endoscope 2, the moving device 3, the operation device 4, and the display device 6 via the input interface 5d and the output interface 5e, and transmits and receives the endoscopic image C, signals, etc. via the input interface 5d and the output interface 5e.

[0018] The storage unit 5c is a computer-readable non-transitory recording medium, such as a hard disk drive, an optical disk, or a flash memory. The storage unit 5c stores a control program that causes the processor 5a to execute a control method, which will be described later, and data necessary for processing by the processor 5a. The storage unit 5c also stores link lengths, which are the lengths of the links between the joints M1 to M6 of the moving apparatus 3.

[0019] The processor 5a executes a control method described below in accordance with a control program loaded from the storage unit 5c into a memory 5b such as a RAM (Random Access Memory). A part of the processing described below executed by the processor 5a may be realized by dedicated logic circuits or hardware such as an FPGA (Field Programmable Gate Array), an SoC (System-On-A-Chip), an ASIC (Application Specific Integrated Circuit), or a PLD (Programmable Logic Device).

[0020] The processor 5a may control the moving device 3 in either a follow-up mode or a stationary mode. A user such as a surgeon can select either the stationary mode or the follow-up mode using a user interface (not shown) provided on the control device 5.

[0021] The stationary mode is a mode in which the endoscope 2 is maintained at a fixed position regardless of the position of the treatment tool 7. The tracking mode is a mode in which the processor 5a controls the moving device 3 based on the position of the treatment tool 7, thereby causing the endoscope 2 to automatically track the treatment tool 7. For example, the processor 5a acquires the position of the tip of the treatment tool 7 by stereo measurement using the endoscopic image C, and controls the moving device 3 to move the endoscope 2 so that the tip of the treatment tool 7 is positioned within the endoscopic image C.

[0022] In general, the moving device 3 changes the position and posture of the endoscope 2 while maintaining the up-down direction of the imaging element 2c so that the up-down direction of the endoscopic image C acquired by the endoscope 2 does not change. In the endoscope system 1 according to this embodiment, the movement device 3 changes the position and posture of the endoscope 2 while rotating the endoscope 2 around its longitudinal axis A in accordance with a control method that will be described below.

[0023] Next, a control method for the endoscope system 1 according to this embodiment will be described below with reference to the drawings. As shown in FIG. 5, the processor 5a first obtains the three-dimensional positions of holes (ports) P0, P1, and P2 formed in the abdominal wall for attaching the trocar 8 (step S1).

[0024] Trocars 8 are attached to a camera port P0 for inserting an endoscope 2 and to first and second ports P1 and P2 for inserting two treatment tools 7. The first port P1 is a hole to which a trocar 8 is attached for inserting a treatment tool (first treatment tool) 7 operated by an operator E with his right hand. The second port P2 is a hole to which a trocar 8 is attached for inserting a treatment tool (second treatment tool) 7 operated by an operator E with his left hand. In Figure 6, plane C1 is shown, for convenience, as a schematic horizontal plane representing the abdominal wall on which the first port P1 and the second port P2 are located.

[0025] The three-dimensional position of the camera port P0 and the three-dimensional positions of the first and second ports P1 and P2 (first and second positions) are obtained by measurement or numerical input. Each three-dimensional position may be obtained from angle information of each joint M1 to M6 of the movable device 3 obtained by the angle sensor 3b, for example, when a movable device 3 having a probe (not shown) at its tip is operated and the probe is brought into contact with each port P0, P1, and P2.

[0026] Next, the processor 5a acquires the three-dimensional position (third position) of the target of interest T1 in the subject in the abdominal cavity (step S2). The target of interest T1 is, for example, the tip point of any treatment tool 7 in the endoscopic image C acquired by the endoscope 2, or a point located at the center of the endoscopic image C. The three-dimensional position of the target of interest T1 is calculated based on the link lengths and angles of the joints M1 to M6 of the moving device 3 when the endoscope 2 is inserted through the camera port P0, and the distance to the target of interest T1 calculated from the endoscopic image C. In FIG. 6, plane C2, the subject on which the target of interest T1 is located, is shown schematically as a horizontal plane for convenience.

[0027] Then, the processor 5a calculates a first vector V1 indicating the up-down direction of the target of interest T1 as seen by the surgeon E based on the acquired first, second, and third positions (step S3). The first vector V1 is defined in a direction perpendicular to a plane C3 that includes the first, second, and third positions, starting from the third position as shown in Fig. 6. The magnitude of the first vector V1 may be arbitrary.

[0028] Furthermore, a third vector V3 indicating the line of sight of the surgeon E is defined on a line L1 connecting the target of attention T1 with the first port P1 and the second port P2 on the plane C3 described above, for example, the central position (fourth position) P3. That is, the head H of the surgeon E is positioned between the treatment tool 7 held in the right hand and the treatment tool 7 held in the left hand, and the height of the eyes is set to be approximately on an extension of the plane C3. The magnitude of the third vector V3 may also be arbitrary.

[0029] The processor 5a also calculates the direction of a second vector V2 obtained by projecting the first vector V1 onto a plane C4 perpendicular to the visual axis B of the endoscope 2 (step S4). The visual axis B of the endoscope 2 is defined on a straight line L2 connecting the three-dimensional position of the camera port P0 and the three-dimensional position of the target of interest T1.

[0030] 7A shows an endoscopic image C acquired while maintaining the up-down direction of the image sensor 2c in the vertical direction, superimposed on the second vector V2. In the figure, "Z" schematically indicates the subject within the abdominal cavity. The second vector V2 may or may not be displayed by the display device 6.

[0031] The third vector V3 indicating the line of sight of the surgeon E extends along the straight line L1 connecting the central position P3 on the abdominal wall and the target of interest T1 inside the body, and is inclined with respect to the horizontal planes C1 and C2. Therefore, the first vector V1 is inclined with respect to the vertical direction when viewed from a direction other than the direction along the straight line L1, and the second vector V2 projected onto the plane C4 orthogonal to the straight line L2 along the visual axis B that is not parallel to the straight line L1 is also inclined with respect to the vertical direction.

[0032] The processor 5a sends a control signal to the drive mechanism 3a to rotate the endoscope 2 around the longitudinal axis A so that the direction of the second vector V2 coincides with the up-down direction of the endoscopic image C displayed on the display device 6 (the relative angle becomes 0°) (step S5). As a result, as shown in Fig. 7B, the endoscopic image C displayed on the display device 6 rotates around the center of the image in the direction of the arrow, and the second vector V2 is directed vertically upward.

[0033] Then, the processor 5a determines whether or not to end the process (step S6), and if not, repeats the steps from step S2. Thus, according to the endoscopic system 1, control device 5, control method and program of this embodiment, the endoscopic image C is rotated so that the direction of the second vector V2 matches the vertical direction of the endoscopic image C displayed on the display device 6.

[0034] Therefore, the up-down direction of the endoscopic image C displayed on the display device 6 can be aligned with the up-down direction of the surgeon E who operates the treatment tool 7 while checking the endoscopic image C on the display device 6. This allows the surgeon E to intuitively operate the treatment tool 7. For example, when the surgeon E intuitively operates the tip of the treatment tool 7 so as to move it straight up and down, the treatment tool 7 can also be moved up and down within the endoscopic image C displayed on the display device 6.

[0035] Furthermore, in the tracking mode, when the treatment tool 7 is moved, the processor 5a controls the movement device 3 to move the endoscope 2 so that the tip of the treatment tool 7 is positioned, for example, at the center of the endoscopic image C. This changes the first vector V1 and the second vector V2, so the processor 5a operates the drive mechanism 3a so that the up-down direction of the endoscopic image C displayed on the display device 6 always coincides with the direction of the second vector V2.

[0036] For example, as shown in FIG. 8, a case will be described in which the distal end of the treatment tool 7 is moved from a target of interest T1, which is located relatively far from the first port P1 and the second port P2 in the horizontal direction, to a target of interest T2, which is located relatively close. In an endoscopic image C capturing the target of interest T1, as shown in FIG. 8, the angle of plane C3 relative to horizontal planes C1 and C2 is relatively small, and therefore the angle of the second vector V2 relative to the vertical direction of the captured endoscopic image C is relatively small. In contrast, in an endoscopic image C capturing the target of interest T2, which is located relatively close to the first port P1 and the second port P2 in the horizontal direction, the angle of plane C3 relative to planes C1 and C2 is relatively large. Therefore, the angle of the second vector V2 relative to the vertical direction of the captured endoscopic image C is relatively large.

[0037] 7A and 9A, if the endoscopic image C acquired while maintaining the up-down direction of the image sensor a in the vertical direction is displayed on the display device 6 as is, the up-down direction perceived by the surgeon E relative to the up-down direction of the endoscopic image C will vary for each of the targets of attention T1 and T2. This makes it difficult for the surgeon E to accurately operate the treatment tool 7. According to this embodiment, as shown in FIGS. 7B and 9B, the up-down direction of the endoscopic image C displayed on the display device 6 is always aligned with the direction of the second vector V2, which has the advantage that the surgeon E can always intuitively and accurately operate the treatment tool 7 even if the targets of attention T1 and T2 are changed.

[0038] In this embodiment, the processor 5a operates the drive mechanism 3a of the moving device 3 to rotate the endoscope 2 about the longitudinal axis A, thereby rotating the angle of view of the endoscopic image C acquired by the image sensor 2c (step S5). Alternatively, as shown in Fig. 10, the processor 5a may rotate the endoscopic image C acquired by the image sensor 2c about the image center by image processing (step S7). This has the advantage that the endoscope 2 is not physically rotated, so there is no need to provide the drive mechanism 3a in the moving device 3, and the endoscopic system 1 can be configured simply.

[0039] In this embodiment, a first vector V1 perpendicular to a plane C3 including the positions of the first port P1, the second port P2, and the target of interest T1 is defined, and the rotation angle of the endoscopic image C is calculated based on a second vector V2 obtained by projecting the first vector V1 onto a plane C4 perpendicular to the longitudinal axis A of the endoscope 2. Alternatively, as shown in Fig. 11, when a third vector V3, which is the line of sight of the surgeon E, rotates around the vertical axis due to a change in the targets of interest T1 and T2, the rotation angle θ yaw function f(θ yaw ) The rotation angle θ roll The endoscopic image C may be rotated by only one of the angles.

[0040] Specifically, the processor 5a defines a third vector V3 in the line of sight of the surgeon E, and calculates a rotation angle θ about the vertical axis of the vector obtained by projecting the third vector V3 onto the horizontal plane C1 due to changes in the targets of interest T1 and T2. yaw Calculate. Then, for example, the angle θ roll Calculate. θ roll =f(θ yaw )=Kθ yaw where K is a constant. According to this, it is not necessary to calculate the first vector V1 and the second vector V2, and the rotation angle θ of the endoscopic image C can be obtained by a simple calculation. roll This has the advantage of requiring less calculation.

[0041] Furthermore, in this embodiment, the endoscopic image C is rotated so that the second vector V2 coincides with the up-down direction of the endoscopic image C displayed on the display device 6. Alternatively, the endoscopic image C may be rotated so that the second vector V2 forms a predetermined angle with respect to the up-down direction of the endoscopic image C displayed on the display device 6. Depending on the surgeon E, it may be easier to operate the treatment tool 7 if the direction of the second vector V2 does not coincide with the up-down direction of the endoscopic image C. Therefore, it is sufficient if the surgeon E's preferred angle can be set as at least one of the preset value and the offset value.

[0042] The positions of the first port P1, the second port P2, and the camera port P0 may be stored in advance in the storage unit 5c in association with the type of procedure to be performed. The processor 5a may acquire the position information by having a user, including the surgeon E, input the type of procedure to be performed using the user interface 4a and reading out the corresponding position information from the storage unit 5c.

[0043] 12, the endoscope system 1 may include a sensor 9 such as a gyro sensor or a position sensor attached to the head H of the surgeon E, and the processor 5a may adjust the tilt of the first vector V1 based on the tilt of the head H detected by the sensor 9. Since the up-and-down direction perceived by the surgeon E varies depending on the left-right tilt α of the head H, the first vector V1 may be calculated taking the tilt α into account. In this way, even if the surgeon E tilts his / her head H during surgery, the up-and-down direction of the endoscopic image C displayed on the display device 6 can be made to match the up-and-down direction perceived by the surgeon E.

[0044] Furthermore, in this embodiment, a direct-view rigid endoscope in which the visual axis B coincides with the direction of the longitudinal axis A of the insertion section 2a has been exemplified as the endoscope 2. Alternatively, an oblique-view endoscope in which the visual axis B is inclined with respect to the longitudinal axis A, or an endoscope 2 equipped with a bending section 2b at the tip of the insertion section 2a that can change the direction of the visual axis B, may be used. In this case, the processor 5a may control the movement device 3 and / or the endoscope 2 so as to bring the direction of the visual axis B closer to the direction of the third vector V3.

[0045] That is, the case where the endoscope 2 is an oblique type will be described with reference to Fig. 13. When the endoscope 2 is an oblique type, even if the endoscope 2 is inserted into the body from a direction different from the straight line L1, the targets of interest T1 and T2 can be positioned within the field of view with the tip of the endoscope 2 positioned on the straight line L1.

[0046] The processor 5a controls the moving device 3 so that the visual axis B is approximately parallel to the line L1 and the distance thereto is smaller than a predetermined threshold. In particular, it is preferable to control the moving device 3 so that the visual axis B coincides with the line L1. This causes the up-down direction of the endoscopic image C acquired by the image sensor 2c to approximately coincide with the up-down direction perceived by the surgeon E, eliminating the need for a process to rotate the endoscopic image C. This also allows the surgeon E to intuitively operate the treatment tool 7. Note that, after controlling the moving device 3 so that the visual axis B coincides with the line L1, if the visual axis B and the line L1 do not completely coincide, the endoscopic image C may be rotated based on the acquired first, second, and third positions so that a predetermined relative angle is formed between the direction of a second vector V2 obtained by projecting a first vector V1, which indicates the up-down direction of the target of interest as seen by the surgeon E, onto a plane perpendicular to the visual axis B, and the up-down direction of the endoscopic image C displayed on the display device 6.

[0047] The same applies when the endoscope 2 has a bending portion 2b. In this case, as shown in Fig. 14, the processor 5a controls the movement device 3 and / or the bending portion 2b of the endoscope 2 so that the visual axis B is approximately parallel to the straight line L1 and the distance therebetween is smaller than a predetermined threshold. This makes it possible to easily align the up-down direction of the endoscopic image C acquired by the image sensor 2c with the up-down direction perceived by the surgeon E, without requiring processing to rotate the endoscopic image C. This also allows the surgeon E to intuitively operate the treatment tool 7.

[0048] Although the embodiments of the present invention and their modifications have been described above, the scope of the present invention is not limited to these, and various improvements are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0049] 1. Endoscopy system 2 Endoscopy 2a Lens tube part (insertion part) 3. Mobile Devices 3a Drive mechanism 5. Control device 5a processor 5c Memory 6 Display device 7 Treatment tool (first treatment tool, second treatment tool) 9 Sensors A Longitudinal axis B visual axis C Endoscopic image (image) C4 plane D Patient (subject) E Operator H head P0 Camera Port (3rd Port) P1 First port P2 Second port T1 Featured V1 First Vector V2 second vector V3 Third Vector α slope

Claims

1. at least one processor; An endoscope and a moving device that holds the endoscope and adjusts the position and attitude of the endoscope in response to a control signal from the processor; a display device that displays an image acquired by the endoscope, the processor: acquiring a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject; An endoscopic system that rotates the image based on the acquired first, second, and third positions so that the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the surgeon onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device form a predetermined relative angle.

2. The endoscopic system of claim 1 , wherein the processor defines the first vector in a direction perpendicular to a plane containing the first, second, and third positions.

3. The endoscope system according to claim 1 , wherein the predetermined relative angle is 0°.

4. The endoscope system according to claim 1 , wherein the predetermined relative angle is at least one of a preset value or an input offset value.

5. the endoscope includes an insertion section extending along a longitudinal axis; the moving device includes a drive mechanism that rotates the endoscope around the longitudinal axis; The endoscope system according to claim 1 , wherein the processor rotates the image by transmitting the control signal that activates the drive mechanism.

6. The endoscope system according to claim 1 , wherein the processor rotates the image by image processing.

7. the processor: calculating a third vector connecting the third position and a fourth position between the first position and the second position on the plane; The endoscope system according to claim 2 , wherein when the third position changes, the image is rotated by an angle calculated as a function of the angular change of the third vector projected onto a horizontal plane.

8. the endoscope includes an insertion section extending along a longitudinal axis; at least one memory; the memory stores the first position, the second position, and the position of a third port into which the insertion section is inserted; The endoscope system according to claim 1 , wherein the processor acquires the first position, the second position, and the position of the third port from the memory.

9. a sensor for detecting the tilt of the head of the surgeon; The endoscope system according to claim 1 , wherein the processor adjusts the tilt of the first vector based on the tilt detected by the sensor.

10. The endoscope is an oblique-viewing endoscope or an endoscope having a bending portion capable of changing the direction of a visual axis, The endoscopic system of claim 1, wherein the processor controls the moving device and / or the endoscope so as to bring the visual axis of the endoscope closer to a straight line connecting the third position and a fourth position between the first position and the second position.

11. A control device for an endoscope system including an endoscope, a moving device that holds the endoscope and adjusts the position and attitude of the endoscope, and a display device that displays an image acquired by the endoscope, at least one processor; The processor: acquiring a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject; A control device for an endoscopic system that rotates the image based on the acquired first, second, and third positions so that the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the surgeon onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device form a predetermined relative angle.

12. A control method for an endoscope system including an endoscope, a moving device that holds the endoscope and adjusts the position and attitude of the endoscope, and a display device that displays an image acquired by the endoscope, comprising: acquiring a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject; a control method for an endoscopic system, comprising: rotating the image based on the acquired first, second, and third positions so that a predetermined relative angle is formed between the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the surgeon onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device.

13. A control program for an endoscope system including an endoscope, a moving device that holds the endoscope and adjusts the position and attitude of the endoscope, and a display device that displays an image acquired by the endoscope, acquiring a first position of a first port for inserting a first treatment tool operated by an operator into a subject, a second position of a second port for inserting a second treatment tool operated by the operator into the subject, and a third position of an object of interest within the subject; A control program for an endoscopic system that causes a computer to execute, based on the acquired first, second, and third positions, rotating the image so that a predetermined relative angle is formed between the direction of a second vector obtained by projecting a first vector indicating the up-down direction of the object of interest as seen by the surgeon onto a plane perpendicular to the visual axis of the endoscope and the up-down direction of the image displayed on the display device.

Images