Endoscopic system, operation method of the endoscopic system, and procedural support program

Abstract

This endoscope system comprises: an electric scope 3 equipped with a camera for capturing an observation target; a robot arm 5 that changes the position and posture of the electric scope 3 while supporting the electric scope 3; a curved portion with which the viewing direction of the electric scope 3 can be changed; an auxiliary storage device 27 that stores a plurality of sets of library data for reproducing endoscopic views associated with respective procedure scenes; and at least one curvature controller 23 and position / posture controller 25. The library data includes at least one relative parameter relating to the relative position and posture between the camera and the observation target, and after calling library data associated with a certain procedure scene from the auxiliary storage device 27, the curvature controller 23 and the position / posture controller 25 control at least one of the curved portion and the robot arm 5 on the basis of the called library data.

Description

Technical Field

[0001] The present invention relates to an endoscope system, a procedure support method, and a procedure support program.

Background Art

[0002] Conventionally, techniques for reducing the labor of procedure operations by an operator have been known (see, for example, Patent Documents 1 and 2). The technique described in Patent Document 1 records the angle information of the joints of a holder, and reversely reproduces the angles of the joints of the holder in time series based on the angle information, thereby returning the endoscope to its original position and orientation. The technique described in Patent Document 2 controls the functions of an energy treatment tool or the like during a surgical procedure based on a machine learning model trained using surgical procedure state and procedure type, etc. included in surgical tool image data or images of anatomical structures.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, using only the angle information of the joints of a holder as in the technique described in Patent Document 1 has a problem that it is difficult to cope with cases where the position and orientation of the endoscope change according to the procedure steps, such as when performing procedures at different locations. Also, in a method of controlling a surgical procedure or controlling the ON / OFF or strength of functions of an energy treatment tool or the like according to the procedure state as in the technique described in Patent Document 2, it is difficult to perform complicated controls such as peeking from the direction the operator wants by changing the position and orientation of the endoscope.

[0005] The present invention has been made in view of the above circumstances, and aims to provide an endoscopic system, a procedure support method, and a procedure support program that reduce the workload of the operator and enable smoother procedures by providing the operator with an observation field of view corresponding to the procedure steps and the object of observation. [Means for solving the problem]

[0006] To achieve the above objective, the present invention provides the following means. A first aspect of the present invention comprises an endoscope equipped with an imaging optical system for photographing an object to be observed, an electric arm for changing the position and orientation of the endoscope while supporting the endoscope, a field of view direction changing unit capable of changing the field of view direction of the endoscope, a storage device for storing a plurality of library data for realizing an endoscope field of view associated with each procedure scene, and at least one processor, wherein the library data includes at least one relative parameter relating to the relative position and orientation between the imaging optical system and the object to be observed. The endoscope comprises a long section at its tip on which the imaging optical system is located, the field of view direction changing section is a curved section capable of changing the angle of the imaging optical system in the endoscope, and the library data includes parameters relating to the orientation of the imaging optical system on the base coordinates, which are determined by the sum of the inclination of the longitudinal axis of the long section on the base coordinates and the amount of curvature of the curved section. The endoscopic system is configured such that the processor retrieves library data associated with any procedure scene from the storage device, and then controls at least one of the field of view direction changing unit and the electric arm based on the retrieved library data.

[0007] According to this embodiment, the processor activates to retrieve library data from storage to realize an endoscopic field of view associated with any procedure scene. The field of view direction changing unit and the motorized arm are then controlled based on the retrieved library data. In this case, since each library data includes relative parameters relating to the relative position and orientation between the endoscope's imaging optical system and the object being observed, the operator can be provided with an endoscopic field of view that is appropriate for both the procedure scene and the object being observed. This reduces the operator's workload and allows for smoother procedures.

[0008] In the above embodiment of the endoscopic system, the processor may identify the procedure scene by processing image information acquired by the imaging optical system, and then retrieve the library data associated with the identified procedure scene from the storage device.

[0009] In this configuration, the field of view direction change unit and the motorized arm are controlled based on the procedure scene identified by image processing by the processor. Therefore, the operator does not need to identify the procedure scene from the image information, further reducing the operator's workload.

[0010] In the above embodiment of the endoscope system, the relative parameter may be distance information between the imaging optical system and the object being observed, and the processor may control at least one of the field of view direction changing unit and the electric arm to match the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, with the distance information.

[0011] This configuration allows the endoscope's imaging optical system to be positioned at a predetermined distance relative to the object being observed, depending on both the procedure and the object being observed. This enables the object to be displayed at the desired distance.

[0012] In the above embodiment of the endoscope system, the relative parameter may be orientation information of the imaging optical system with respect to the object being observed, and the processor may control at least one of the field of view direction changing unit and the electric arm to match the orientation information of the imaging optical system with respect to the object being observed, which is calculated from the image information acquired by the imaging optical system.

[0013] This configuration allows the endoscope's imaging optical system to be positioned in a preset orientation relative to the object being observed, depending on both the procedure and the object being observed. Because the orientation of the imaging optical system is determined by the posture of the organ or other observed tissue, even if there is variation in individual differences, the preset endoscopic field of view can be reproduced more easily and accurately. Furthermore, even if the posture or orientation of the observed tissue changes due to the assistant's forceps manipulation, the preset endoscopic field of view can be easily obtained.

[0014] In the above embodiment of the endoscope system, the relative parameter may be inclination information of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, around the optical axis, and the processor may control at least one of the field of view direction changing unit and the electric arm to match the inclination of the elongated object being observed, which is calculated from the image information acquired by the imaging optical system, around the optical axis of the imaging optical system with the inclination information.

[0015] This configuration allows the endoscopic field of view to be pre-configured to show the direction in which the intestines, blood vessels, etc., extend, depending on both the procedure and the object being observed. This makes it easier to obtain an endoscopic field of view that is more accommodating to individual differences in patients.

[0016] In the above embodiment of the endoscopic system, the field of view direction changing section may be a bending section capable of changing the angle of the imaging optical system in the endoscope. In this case, the endoscope may have a long section at its tip where the imaging optical system is located, and the library data may include parameters relating to the orientation of the imaging optical system on the base coordinates, which are determined by the sum of the inclination of the longitudinal axis of the long section on the base coordinates and the amount of curvature of the bending section.

[0017] This configuration allows the orientation of the imaging optical system to be aligned with the base coordinates. As a result, even if the position of the observed tissue changes or the tilt of the long end of the endoscope changes due to individual differences, the orientation of the imaging optical system does not change, allowing for easy and accurate reproduction of a pre-set endoscopic field of view.

[0018] The endoscope system according to the above embodiment is an oblique viewer having a lens barrel that houses the imaging optical system, and the optical axis of the imaging optical system is positioned inclined with respect to the central axis of the lens barrel, and the field of view direction changing unit may change the angle of the lens barrel around the central axis. By changing the angle around the central axis of the endoscope tube, the field of view of the oblique endoscope, which is at a constant angle with respect to the central axis of the endoscope tube, changes around the central axis of the endoscope tube. This allows the field of view of the oblique endoscope to be switched to an oblique upward or oblique downward direction, etc. As a result, the field of view of the oblique endoscope can be directed towards the desired observation target simply by rotating the endoscope tube around its central axis. Therefore, even when an oblique endoscope is used as an endoscope, the observation target specified by the operator can be easily brought into the endoscopic field of view.

[0019] The endoscope system according to the above embodiment is a straight-viewing endoscope having a barrel section that houses the imaging optical system, and the central axis of the barrel section and the optical axis of the imaging optical system are aligned, and the field of view direction changing section may change the angle around a rotation axis perpendicular to the central axis of the barrel section in accordance with the change in the position of capturing the object of observation on the display screen that displays the image information acquired by the imaging optical system. The field of view direction changing unit changes the angle around the rotation axis perpendicular to the central axis of the endoscope tube in accordance with the change in the position where the object of observation is captured on the display screen. As a result, the field of view direction of the straight-viewing endoscope is changed to the orientation in which the object of observation is captured at the changed position on the display screen. For example, if the position where the object of observation is captured is changed to the edge of the display screen, the field of view direction of the straight-viewing endoscope is changed to the orientation in which the object of observation is captured at the edge of the field of view, thereby creating an endoscopic field of view at an angle in which the object of observation is peered into. Therefore, even when a straight-viewing endoscope is used as the endoscope, the object of observation specified by the operator can be easily placed in the endoscopic field of view.

[0020] The endoscope system according to the above aspect may be configured such that the processor processes the image information acquired by the imaging optical system to recognize the observation target. With this configuration, there is no need for the operator to identify the observation target from the image information, and the labor of the operator can be further reduced.

[0021] The endoscope system according to the above aspect may include an input unit that allows a user to specify the observation target based on the image information acquired by the imaging optical system. With this configuration, the operator can select a desired observation target.

[0022] A second aspect of the present invention is The processor, a step of calling, from a storage device, the library data associated with an arbitrary procedure scene among a plurality of library data for realizing an endoscope view associated with each procedure scene, and a step of changing at least one of the viewing direction, position, and orientation of the endoscope that photographs the observation target based on the called library data, wherein each said library data includes at least one relative parameter regarding the relative position and orientation between the imaging optical system of the endoscope and the observation target The endoscope system includes an elongated section at its tip on which the imaging optical system is located, and a curved section on which the angle of the imaging optical system can be changed, wherein the field of view of the endoscope is changed by changing the angle of the imaging optical system with the curved section, and the library data includes parameters relating to the orientation of the imaging optical system on the base coordinates, which are determined by the sum of the inclination of the longitudinal axis of the elongated section on the base coordinates and the amount of curvature of the curved section. is.

[0023] According to the above aspect How to operate an endoscope system is The aforementioned processor, After identifying the procedure scene by processing the image information acquired by the imaging optical system, the library data associated with the identified procedure scene may be called from the storage device.

[0024] According to the above aspect How to operate an endoscope system is such that the relative parameter is distance information between the imaging optical system and the observation target, The aforementioned processor, and by changing at least one of the viewing direction, position, and orientation of the endoscope, the distance between the imaging optical system and the observation target calculated from the image information acquired by the imaging optical system may be made to match the distance information.

[0025] Related to the above-mentioned aspect How to operate an endoscope system The relative parameter is the orientation information of the imaging optical system with respect to the object being observed. The aforementioned processor, The orientation of the imaging optical system relative to the object being observed, calculated from the image information acquired by the imaging optical system, may be made to match the orientation information by changing at least one of the field of view direction, position, and orientation of the endoscope.

[0026] Related to the above-mentioned aspect How to operate an endoscope system The relative parameter is the inclination information of the observable object, which is elongated and extends in a direction perpendicular to the optical axis of the imaging optical system, about the optical axis. The aforementioned processor, The inclination of the elongated object being observed around the optical axis of the imaging optical system, calculated from the image information acquired by the imaging optical system, may be made to match the inclination information by changing at least one of the field of view direction, position, and orientation of the endoscope.

[0027] Related to the above-mentioned aspect How to operate an endoscope system The endoscope may also be provided with a bending section that can change the angle of the imaging optical system, and the field of view of the endoscope may be changed by changing the angle of the imaging optical system with the bending section. In that case, the endoscope may be provided with a long section at its tip where the imaging optical system is located, and the library data may include a parameter relating to the orientation of the imaging optical system on the base coordinates, which is determined by the sum of the inclination of the longitudinal axis of the long section of the endoscope on the base coordinates and the amount of curvature of the bending section.

[0028] Related to the above-mentioned aspect How to operate an endoscope system The endoscope is an oblique viewer having a barrel portion that houses the imaging optical system, and the optical axis of the imaging optical system is positioned at an inclination with respect to the central axis of the barrel portion, and the field of view of the endoscope may be changed by changing the angle of the barrel portion around the central axis.

[0029] Related to the above-mentioned aspect How to operate an endoscope systemThe endoscope is a straight-viewing endoscope having a barrel section that houses the imaging optical system, and the central axis of the barrel section and the optical axis of the imaging optical system are aligned. The field of view of the endoscope may be changed by changing the angle around a rotation axis perpendicular to the central axis of the barrel section in accordance with a change in the position of the object being observed on the display screen that displays the image information acquired by the imaging optical system.

[0030] Related to the above-mentioned aspect How to operate an endoscope system The image information acquired by the imaging optical system is The aforementioned The object being observed may be recognized by being processed by a processor. Related to the above-mentioned aspect How to operate an endoscope system The object to be observed may be specified by the user based on the image information acquired by the imaging optical system.

[0031] A third aspect of the present invention involves causing a computer to perform the following steps: retrieve from a storage device the library data associated with an arbitrary procedure scene from among a plurality of library data for realizing an endoscopic field of view associated with each procedure scene; and change at least one of the field of view direction, position, and orientation of the endoscope that photographs the object of observation based on the retrieved library data, wherein each library data includes at least one relative parameter relating to the relative position and orientation between the imaging optical system of the endoscope and the object of observation. Furthermore, the endoscope comprises a long section at its tip on which the imaging optical system is located, and a curved section on which the angle of the imaging optical system can be changed, and by changing the angle of the imaging optical system with the curved section, the field of view of the endoscope is changed, and the library data includes parameters relating to the orientation of the imaging optical system on the base coordinates, which are determined by the sum of the inclination of the longitudinal axis of the long section on the base coordinates and the amount of curvature of the curved section. This is a procedural support program.

[0032] The procedure support program according to the above embodiment may also cause a computer to perform the steps of identifying the procedure scene by processing image information acquired by the imaging optical system, and retrieving the library data associated with the identified procedure scene from the storage device.

[0033] In the above embodiment, the procedural support program may be configured such that the relative parameter is distance information between the imaging optical system and the object being observed, and the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, is matched to the distance information by changing at least one of the field of view direction, position, and orientation of the endoscope.

[0034] In the above embodiment, the procedural support program may be configured such that the relative parameter is orientation information of the imaging optical system relative to the object of observation, and the orientation of the imaging optical system relative to the object of observation, calculated from the image information acquired by the imaging optical system, is matched to the orientation information by changing at least one of the field of view direction, position, and posture of the endoscope.

[0035] The procedure support program according to the above embodiment may be configured such that the relative parameter is inclination information of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, around the optical axis, and the inclination of the elongated object being observed, which is calculated from the image information acquired by the imaging optical system, around the optical axis of the imaging optical system is matched to the inclination information by changing at least one of the field of view direction, position, and orientation of the endoscope.

[0036] The procedure support program according to the above embodiment may also include a bending section in the endoscope that can change the angle of the imaging optical system, and the field of view of the endoscope is changed by changing the angle of the imaging optical system with the bending section. In this case, the endoscope may include a long section at its tip on which the imaging optical system is located, and the library data may include a parameter relating to the orientation of the imaging optical system on the base coordinates, which is determined by the sum of the inclination of the longitudinal axis of the long section on the base coordinates and the amount of curvature of the bending section.

[0037] The procedure support program according to the above embodiment is an oblique endoscope having a barrel portion that houses the imaging optical system, and the optical axis of the imaging optical system is positioned inclined with respect to the central axis of the barrel portion, and the field of view direction of the endoscope may be changed by changing the angle of the barrel portion around the central axis.

[0038] The procedure support program according to the above embodiment is a straight-viewing endoscope having a barrel section that houses the imaging optical system, and the central axis of the barrel section and the optical axis of the imaging optical system are aligned. The program may also change the field of view of the endoscope by changing the angle around a rotation axis perpendicular to the central axis of the barrel section in accordance with a change in the position of the object being observed on a display screen that displays image information acquired by the imaging optical system.

[0039] The procedure support program according to the above embodiment may also cause a computer to perform the step of recognizing the object to be observed by processing the image information acquired by the imaging optical system. In the above embodiment, the procedure support program may also be configured such that the user specifies the object to be observed based on the image information acquired by the imaging optical system. [Effects of the Invention]

[0040] According to the present invention, by providing the operator with an observation field of view that corresponds to the procedure and the object being observed, the effort required of the operator is reduced and the procedure can be performed more smoothly. [Brief explanation of the drawing]

[0041] [Figure 1] This is a schematic diagram of an endoscope system according to the first embodiment of the present invention. [Figure 2] This diagram illustrates an example of the schematic configuration and relative parameters of an electric scope. [Figure 3] There is a diagram illustrating the control method using a control device. [Figure 4]This is a flowchart illustrating a procedure support method and a procedure support program according to the first embodiment of the present invention. [Figure 5] This diagram illustrates the relationship between the tilt of the motorized scope and the orientation of the camera. [Figure 6] This diagram illustrates the orientation of the imaging optical system as viewed from the base coordinates. [Figure 7] This diagram illustrates the relationship between the posture of the object being observed and the orientation of the camera. [Figure 8] This diagram illustrates the orientation of the imaging optical system relative to the object being observed. [Figure 9] This diagram illustrates the relationship between the direction in which a long object being observed moves and the tilt of the camera around its optical axis. [Figure 10] This figure shows an example of an endoscopic field of view where the direction in which the elongated object being observed is moving is perpendicular to the monitor screen. [Figure 11] This diagram illustrates how to adjust the roll angle of an electric scope to match the direction in which a long object being observed is moving. [Figure 12] This diagram illustrates how to adjust the roll angle of the motorized endoscope to match the patient's or operating table's position. [Figure 13A] This figure shows the relationship between the coordinates of the operating table or patient and the base coordinates under normal circumstances. [Figure 13B] This diagram shows the relationship between the coordinates of a tilted operating table or patient and the uncalibrated base coordinates. [Figure 13C] This diagram shows the relationship between the coordinates of the tilted operating table or patient and the calibrated base coordinates. [Figure 14A] This figure shows the relationship between the coordinates of the operating table or patient and the base coordinates under normal circumstances. [Figure 14B] This figure shows the relationship between the coordinates of a tilted operating table or patient and the orientation of the motorized endoscope when the library data has not been calibrated. [Figure 14C] This figure shows the relationship between the tilted operating table or patient's coordinates and the orientation of the motorized endoscope when calibrated with library data. [Figure 15] This diagram illustrates how the main controller switches library data in response to AI-based analysis of the procedure scene. [Figure 16] This diagram illustrates how the main controller switches library data in response to the surgeon's identification of a procedure scene. [Figure 17] This diagram illustrates how the main controller switches library data according to the operator's instructions. [Figure 18] This diagram illustrates how the main controller switches library data depending on the treatment location on the patient. [Figure 19] This diagram illustrates the treatment position with the insertion point in the patient as the origin. [Figure 20] This diagram illustrates other treatment locations on the patient, with the insertion point as the origin. [Figure 21] This diagram illustrates how to control an endoscope system when using roll angles around the optical axis as library data. [Figure 22] There is a diagram illustrating how to control the endoscopic system when creating an overhead view of the endoscopic field of view. [Figure 23] This is a schematic diagram of the oblique viewer of an endoscope system according to a second embodiment of the present invention. [Figure 24] There is a diagram illustrating the control method using a control device. [Figure 25] This diagram illustrates how the field of view of an oblique-viewing microscope changes around the longitudinal axis of the microscope tube. [Figure 26] This diagram illustrates the upward-looking field of view of an oblique-viewing microscope. [Figure 27] This diagram illustrates the field of view when the curved section of an electric scope is bent approximately 30 degrees upwards. [Figure 28] This diagram illustrates the downward field of view of an oblique-viewing microscope. [Figure 29] This diagram illustrates the field of view when the curved section of an electric scope is bent approximately 30 degrees downwards. [Figure 30] This diagram illustrates how the camera's viewing axis rotates around its axis. [Figure 31] This is a schematic diagram of the direct viewing endoscope of an endoscope system according to the third embodiment of the present invention. [Figure 32] There is a diagram illustrating the control method using a control device. [Figure 33] This diagram illustrates how the camera's viewing axis rotates around its axis. [Figure 34] This figure shows the object being observed positioned in the center of the endoscopic field of view. [Figure 35] This diagram shows the object of observation positioned in the center of the screen. [Figure 36] This diagram shows the object of observation captured at the edge of the field of view. [Figure 37] This diagram shows how previously hidden areas become visible through the endoscopic field of view at the angle from which the object is being observed. [Modes for carrying out the invention]

[0042] [First Embodiment] An endoscope system, a procedure support method, and a procedure support program according to the first embodiment of the present invention will be described below with reference to the drawings. The endoscopic system 1 according to this embodiment, as shown in Figure 1, includes an electric scope (endoscope) 3 for acquiring images inside the patient's body cavity, a robotic arm (electric arm) 5 for supporting the electric scope 3, and a control device 7 for controlling the electric scope 3 and the robotic arm 5. In Figure 1, reference numeral 9 indicates a video system center, and reference numeral 11 indicates a monitor. The video system center 9 is connected to the electric scope 3 and the control device 7. The monitor 11 is connected to the control device 7.

[0043] As shown in Figure 2, the electric scope 3 comprises a long insertion section (long section) 13 inserted into the patient's body cavity, a camera (imaging optical system) 15 provided at the tip of the insertion section 13, a bending section (field of view direction changing section) 17 that changes the inclination angle of the camera 15's endoscopic field of view with respect to the longitudinal axis of the insertion section 13, and a bending motor that drives the bending section 17 and a roll motor that rolls the insertion section 13 around its longitudinal axis (neither of which are shown).

[0044] Camera 15 is composed of, for example, at least one lens and an imager such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor). Camera 15 may be a monocular camera or a stereo camera. Furthermore, the motorized scope 3 has a distance measuring function that measures the distance to the object being observed within the field of view of the camera 15. A known mechanism can be used to realize the distance measuring function.

[0045] The robot arm 5 is, for example, a motorized holder for a general-purpose 6-axis articulated robot that holds the motorized scope 3 in a movable position. The robot arm 5 is equipped with a motor (not shown) for each joint that operates the respective joint.

[0046] The control device 7 is implemented, for example, by a dedicated or general-purpose computer. Specifically, as shown in Figures 1 and 3, the control device 7 includes a main controller (processor) 21 such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), a field of view controller (processor) 23, and a position and orientation controller (processor) 25, a main memory (not shown) such as RAM (Random Access Memory) used as the working area for each of these controllers 21, 23, and 25, and an auxiliary memory (storage device) 27.

[0047] The auxiliary storage device 27 is a computer-readable non-temporary storage medium such as an SSD (Solid State Drive) or HDD (Hard Disk Drive). The auxiliary storage device 27 stores a procedure support program that causes each controller 21, 23, and 25 to execute processing, and multiple library data for realizing the endoscopic field of view of the motorized scope 3 associated with each procedure scene. The main storage device and the auxiliary storage device 27 may be connected to the control device 7 via a network.

[0048] The procedure support program, for example as shown in Figure 4, processes image information acquired by the motorized scope 3 to perform the following steps: recognizing the object of observation S captured by the camera 15 and identifying the procedure scene (steps S2, S6); retrieving library data associated with the identified procedure scene from the auxiliary storage device 27 (steps S3, S4); and changing at least one of the angle of the camera 15, the position and posture of the motorized scope 3 based on the retrieved library data (steps S4, S8), which are performed by the controllers 21, 23, and 25 of the control device 7.

[0049] The main controller 21 includes a capture board 29 that acquires endoscopic images from the video system center 9, and a graphics board 31 that outputs endoscopic images and status signals. By processing the endoscopic images, the main controller 21 recognizes the object of observation S on the endoscopic image and identifies the procedure scene in that endoscopic image. The main controller 21 then retrieves library data associated with the identified procedure scene from the auxiliary storage device 27.

[0050] The field of view controller 23 is connected to the motorized scope 3 and sends bending motion commands to the motorized scope 3 while receiving bending angle information from the motorized scope 3. The position and orientation controller 25 is connected to the motorized scope 3, the robot arm 5, and the voice recognition unit 33. The position and orientation controller 25 sends endoscopic operation commands to the motorized scope 3, while receiving input from the motorized scope 3 regarding the amount of rotation of the insertion section 13 around its longitudinal axis. The position and orientation controller 25 also sends endoscopic operation commands to the robot arm 5, while receiving signals from the robot arm 5 regarding its position and state (orientation).

[0051] Furthermore, the control device 7 is connected to various user interfaces (UIs), including a headset (input unit) 35, a hand switch (input unit) 37, and a foot switch (input unit) 39. The headset 35 allows the operator to input endoscopic operation commands and operation switching commands by voice. The endoscopic operation commands and operation switching commands input from the headset 35 are sent to the position and posture controller 25 via the voice recognition unit 33.

[0052] The hand switch 37 is attached to, for example, a treatment instrument, allowing the operator to input endoscopic operation commands by operating it with their hand. The endoscopic operation commands input from the hand switch 37 are sent to the position and posture controller 25 via the voice recognition unit 33. The foot switch 39 allows the operator to input endoscopic operation commands and operation switching commands by operating it with their feet. The endoscopic operation commands and operation switching commands input from the foot switch 39 are sent to the main controller 21.

[0053] Multiple library data stored in the auxiliary storage device 27 include at least one relative parameter relating to the relative position and orientation between the camera 15 of the motorized scope 3 and the object of observation S captured by the camera 15. Examples of relative parameters include scope axis roll angle information, distance information between the camera 15 and the object of observation S, and curvature amount information of the curved portion 17 of the motorized scope 3.

[0054] The scope axis roll angle information is the roll angle of the insertion section 13 of the motorized scope 3 from its initial state around the longitudinal axis, as shown in Figure 2. The distance information between the camera 15 and the observation target S is the distance between the camera 15 and the observation target S in the direction along the visual axis of the camera 15. The curvature amount information of the bending section 17 is the orientation of the camera 15 with respect to the scope axis of the motorized scope 3. Hereinafter, this information will be referred to as roll angle information, distance information, and curvature amount information. In other words, the auxiliary storage device 27 has endoscopic fields of view suitable for each procedure scene pre-set for each procedure scene, and at least one of the roll angle information, distance information, and curvature amount information necessary to realize each of these endoscopic fields of view is stored in association with each procedure scene.

[0055] The control device 7 retrieves roll angle information, distance information, and / or curvature amount information associated with any procedure scene from the auxiliary storage device 27, and then controls at least one of the curvature section 17 of the motorized scope 3 and the robot arm 5 based on each of these retrieved pieces of information.

[0056] For example, when the control device 7 retrieves distance information from the auxiliary storage device 27, it controls at least one of the curved portion 17 of the motorized scope 3 and the robot arm 5 to match the actual distance between the camera 15 and the object being observed S, which is calculated from the image information of the object being observed S, with the retrieved distance information.

[0057] Specifically, as shown in Figure 3, the main controller 21 compares the current distance measurement to the observation target S with the distance information retrieved from the auxiliary storage device 27 to calculate a trajectory that matches the distance between the camera 15 and the observation target S to the distance information. The current distance measurement to the observation target S is measured by the distance measuring function of the motorized scope 3. Then, the main controller 21 distributes the commands to be joint control commands for the robot arm 5 and bending angle commands for the motorized scope 3, and inputs the joint control commands to the position and orientation controller 25 and the bending angle commands to the field of view controller 23.

[0058] The position and orientation controller 25, following joint control commands, uses the inverse kinematics of the robot arm 5 to determine the amount of drive required for each joint to match the distance between the camera 15 and the observation target S with distance information. The determined amount of drive for each joint is input to each motor of the robot arm 5 as an angle command (endoscopic motion command) for each joint.

[0059] Meanwhile, the field of view controller 23 determines the curvature angle of the curvature section 17 necessary to match the distance between the camera 15 and the observation target S with the distance information, according to the curvature angle command. The determined curvature angle of the curvature section 17 is input to the motor of the motorized scope 3 as a motor angle command (curvature operation command).

[0060] Each joint of the robot arm 5 operates according to its respective angle command, and the bending section 17 of the electric scope 3 bends according to the motor angle command, thereby obtaining an endoscopic field of view in which the distance between the camera 15 and the observation target S matches the distance information.

[0061] Furthermore, for example, when the control device 7 retrieves curvature amount information from the auxiliary storage device 27, it controls at least one of the bending portion 17 of the motorized scope 3 and the robot arm 5 to match the orientation of the camera 15 relative to the observation target S, which is calculated from the image information of the observation target S, with the retrieved curvature amount information.

[0062] Specifically, the main controller 21 compares the current curvature of the curved section 17 with the curvature information retrieved from the auxiliary storage device 27 to calculate a trajectory that will match the curvature of the curved section 17 to the curvature information. Then, the main controller 21 distributes the commands to be joint control commands for the robot arm 5 and curvature angle commands for the motorized scope 3, and inputs the joint control commands to the position and orientation controller 25 and the curvature angle commands to the field of view controller 23.

[0063] The position and orientation controller 25, following joint control commands, uses the inverse kinematics of the robot arm 5 to determine the amount of drive required for each joint to match the curvature of the curved section 17 with the curvature amount information. The determined amount of drive for each joint is input to each motor of the robot arm 5 as an angle command (endoscopic motion command) for each joint.

[0064] Meanwhile, the field of view controller 23 determines the curvature angle of the curved section 17 necessary to match the amount of curvature of the curved section 17 with the curvature amount information, in accordance with the curvature angle command. The determined curvature angle of the curved section 17 is input to the motor of the electric scope 3 as a motor angle command (curvature operation command).

[0065] As each joint of the robot arm 5 operates according to its respective angle command, and the bending section 17 of the electric scope 3 bends according to the motor angle command, an endoscopic field of view is obtained in which the orientation of the camera 15 relative to the observation target S matches the amount of bending information.

[0066] Furthermore, for example, if the control device 7 retrieves roll angle information from the auxiliary storage device 27, it controls the robot arm 5 to match the inclination of the object being observed S around the optical axis of the camera 15, which is calculated from the image information of the object being observed S, with the retrieved roll angle information.

[0067] Specifically, the main controller 21 compares the current roll angle of the scope axis with the roll angle information to determine the amount of position and attitude change around the pivot point required to match the roll angle of the scope axis to the roll angle information. The determined amount of position and attitude change around the pivot point is input to the position and attitude controller 25 as a position and attitude command. The position and attitude command includes, for example, the roll angle, pitch angle, yaw angle, and advance / reverse amount of the motorized scope 3.

[0068] The position and orientation controller 25 determines the amount of drive required for each joint to match the roll angle of the scope axis to the roll angle information by using the inverse kinematics of the robot arm 5 in accordance with the position and orientation command. The determined amount of drive for each joint is input to each motor of the robot arm 5 as an angle command (endoscopic motion command) for each joint. As each joint of the robot arm 5 operates according to its respective angle command, an endoscopic field of view is obtained in which the tilt of the observation target S around the optical axis of the camera 15 matches the roll angle information.

[0069] When the position and orientation controller 25 controls the robot arm 5, it uses the forward kinematics of the robot arm 5 to calculate Euler angles (roll, pitch, yaw) based on the angles of each joint. This allows the current roll angle of the scope axis to be calculated. The calculated current roll angle of the scope axis is stored in the main controller 21.

[0070] The field of view controller 23 controls the bending section 17 of the motorized scope 3 and calculates the current curvature of the bending section 17 by converting the motor angle of the bending section 17 into a curvature amount. The calculated current curvature amount is stored in the main controller 21.

[0071] Next, the operation of the endoscopic system 1, the procedure support method, and the procedure support program with the above configuration will be explained with reference to the flowchart in Figure 4. In order to support the operator's procedure using the endoscopy system 1, procedure support method, and procedure support program according to this embodiment, first, the motorized scope 3 is inserted into the patient's body (step S1). Then, the camera 15 of the motorized scope 3 captures the observation target S, thereby acquiring image information. The acquired image information is input to the main controller 21 of the control device 7 via the video system center 9.

[0072] Next, the main controller 21 processes the input image information, thereby recognizing the object of observation S and identifying the procedure scene (step S2). Then, the main controller 21 retrieves library data associated with the identified procedure scene from the auxiliary storage device 27 (step S3), and then creates an endoscopic field of view for deployment (step S4). Deployment refers to operations such as applying tension to surrounding tissue by pulling it with an assistant's forceps to make it easier to cut or visualize the tissue.

[0073] For example, if the library data retrieved from the auxiliary storage device 27 is distance information, the main controller 21 determines joint control commands and bending angle commands to match the distance between the camera 15 and the observed object S with the distance information, based on the current distance measurement between the camera 15 and the observed object S.

[0074] Next, the position and orientation controller 25 determines the amount of drive for each joint of the robot arm 5 based on the joint control command. Similarly, the field of view controller 23 determines the curvature angle of the curvature section 17 based on the curvature angle command. Then, as each joint of the robot arm 5 and the curvature section 17 of the motorized scope 3 operate according to their respective commands, an endoscopic field of view is created in which the distance between the camera 15 and the observation target S matches the distance information.

[0075] Furthermore, for example, if the library data retrieved from the auxiliary storage device 27 is curvature amount information, the main controller 21 determines joint control commands and curvature angle commands to match the curvature amount of the curvature portion 17 with the curvature amount information, based on the current curvature amount of the curvature portion 17 and the curvature amount information.

[0076] Next, the position and orientation controller 25 determines the amount of drive for each joint of the robot arm 5 based on the joint control command. Similarly, the field of view controller 23 determines the curvature angle of the curvature section 17 according to the curvature angle command. Then, as each joint of the robot arm 5 and the curvature section 17 of the motorized scope 3 operate according to their respective commands, an endoscopic field of view is created in which the orientation of the camera 15 relative to the observation target S matches the curvature amount information.

[0077] Furthermore, for example, if the library data retrieved from the auxiliary storage device 27 is roll angle information, the main controller 21 determines a position and attitude command around the pivot point to match the roll angle of the scope axis to the roll angle information, based on the current roll angle of the scope axis and the roll angle information.

[0078] Next, the position and orientation controller 25 determines the amount of drive for each joint based on the position and orientation command. Then, as each joint of the robot arm 5 operates according to its respective angle command, an endoscopic field of view is created in which the tilt of the observation target S around the optical axis of the camera 15 matches the roll angle information. Next, the assistant performs forceps manipulation and dissection operations within the created endoscopic field of view (Step S5).

[0079] Next, when the operator proceeds to the actual procedure, the main controller 21 processes the newly acquired image information from the camera 15, thereby recognizing the object of observation S and identifying the procedure scene (step S6). Then, the main controller 21 retrieves library data associated with the identified procedure scene from the auxiliary storage device 27 (step S7), and based on the retrieved library data, the motorized scope 3 and robotic arm are controlled to create an endoscopic field of view for the procedure (step S8). The method for creating the field of view to be used during the procedure is the same as the method for creating the endoscopic field of view for deployment, so the explanation is omitted. Once the endoscopic field of view for the procedure is created, the operator begins tracking the treatment instrument within the created endoscopic field of view (Step S9).

[0080] As described above, according to the endoscope system 1, procedure support method, and procedure support program of this embodiment, when the control device 7 is activated, library data associated with any procedure scene is retrieved from the auxiliary storage device 27, and then at least one of the robot arm 5 and the bending portion 17 of the motorized scope 3 is controlled based on that library data. In this case, since each library data includes relative parameters relating to the relative position and posture between the camera 15 of the motorized scope 3 and the object of observation S, the operator can be provided with an endoscopic field of view that corresponds to both the procedure scene and the object of observation S. Therefore, the operator's workload can be reduced and the procedure can be performed more smoothly.

[0081] In this embodiment, the example described shows a case where one of the distance information, curvature amount information, and roll angle information is retrieved as library data from the auxiliary storage device 27. Alternatively, multiple library data may be retrieved simultaneously.

[0082] For example, when distance information, curvature amount information, and roll angle information are retrieved as library data from the auxiliary storage device 27, the position and attitude controller 25 determines the amount of drive for each joint necessary to match the distance between the camera 15 and the observation target S, the curvature amount of the curvature section 17, and the roll angle of the scope axis to the distance information, curvature amount information, and roll angle information, respectively, based on the joint control command and the position and attitude command.

[0083] Furthermore, the field of view controller 23 determines the curvature angle of the curved section 17 necessary to match the distance between the camera 15 and the observation target S, and the amount of curvature of the curved section 17, to the distance information and curvature amount information, respectively, based on the curvature angle command.

[0084] Next, each joint of the robot arm 5 operates according to the respective angle commands, and the bending section 17 of the electric scope 3 bends according to the motor angle command, thereby creating an endoscopic field of view in which the distance between the camera 15 and the object of observation S, the orientation of the camera 15 relative to the object of observation S, and the tilt of the object of observation S around the optical axis of the camera 15 correspond to the distance information, the amount of curvature information, and the roll angle information, respectively.

[0085] Once the distance between the camera 15 and the observation target S, the curvature of the motorized scope 3, and the roll angle of the scope axis are determined, the position and orientation of the motorized scope 3 are uniquely determined relative to the observation target S. Therefore, there is no need to register data for each patient in advance, and the same physical quantities can be used to create endoscopic views even for different patients.

[0086] This embodiment can be modified as follows. In this embodiment, the main controller 21 recognizes the observation target S by processing the endoscopic image. Alternatively, for example, the operator may specify the observation target using a user interface (UI) such as a headset 35, hand switch 37, and foot switch 39. Similarly, instead of the main controller 21 identifying the procedure scene, the operator may identify the procedure scene.

[0087] Furthermore, in this embodiment, the amount of curvature of the curved portion 17 of the motorized scope 3 was illustrated as library data. In this case, for example, as shown in Figure 5, even if the amount of curvature of the curved portion 17 is the same, the angle at which the camera 15 looks up or down relative to the observation target S changes depending on the tilt of the motorized scope 3.

[0088] Therefore, instead of curvature information, for example, as shown in Figure 6, information indicating the orientation of the camera 15 as seen from a base coordinate system with the center of the robot arm 5's base as the origin, that is, the orientation of the viewing axis relative to the base coordinate system, may be adopted as library data. Hereinafter, this information will be referred to as orientation information as seen from the base coordinate system.

[0089] The orientation of camera 15 relative to the base coordinates is determined by the sum of the tilt of the motorized scope 3 relative to the base coordinates and the amount of curvature of the bending section 17. In a given procedure scene, by matching the orientation of camera 15 relative to the base coordinates with the orientation information relative to the base coordinates associated with that procedure scene, it is possible to prevent the orientation of camera 15 from changing even if the tilt of the motorized scope 3 changes in accordance with changes in the position of biological tissue due to individual differences, etc. Therefore, the endoscopic field of view at the time of setting registration can be reproduced more easily and accurately.

[0090] Furthermore, if the library data includes information on the amount of curvature of the curved section 17 and the orientation of the visual axis, then, for example, as shown in Figure 7, if the posture of the object of observation S changes due to the operation of the assistant's forceps, the angle at which the camera 15 looks through the object of observation S may change relatively.

[0091] Therefore, as library data, for example, as shown in Figure 8, information indicating the orientation of the camera 15 relative to the observation target S, that is, the orientation of the visual axis as seen from the coordinates of the observation target S, may be adopted. Hereinafter, this information will be referred to as orientation information as seen from the target. In a given procedure scene, by matching the orientation of the camera 15 as seen from the observation target S with the orientation information as seen from the target associated with that procedure scene, the orientation of the camera 15 is determined according to the posture of the organ or living tissue. Therefore, even if the posture or orientation of the living tissue changes due to the assistant's forceps manipulation, etc., the same endoscopic field of view, i.e., the viewing angle, can be maintained as at the time of setting registration.

[0092] Furthermore, if scope axis roll angle information is adopted as library data, the tilt of the camera 15 around the optical axis in the direction in which the elongated observation target S, such as the intestine or blood vessel, is traveling may be recognized, for example, as shown in Figure 9. Then, as roll angle information, a parameter may be set to orient the monitor 11 screen horizontally or vertically in the direction in which the intestine or blood vessel is traveling.

[0093] In actual procedures, for example, as shown in Figure 10, the field of view is adjusted so that the direction in which the intestine or blood vessels run, i.e., the direction in which the dissection line extends, is oriented horizontally or vertically relative to the screen of the monitor 11. By controlling the direction in which the intestine or blood vessels run, etc., is oriented horizontally or vertically relative to the screen of the monitor 11, an endoscopic field of view that is more responsive to individual differences in patients can be obtained. In Figure 10, the symbol A represents the intestine, the symbol B represents nerve bundles, the symbol C represents the IMA (Inferior Mesenteric Artery), and the symbol D represents the dissection line.

[0094] In this case, for example, as shown in Figure 11, the direction in which the elongated observation target S, such as the intestine or blood vessel, travels is approximated by a curve, and then the average angle of these inclinations is calculated by calculating the inclinations of multiple points. The roll angle of the electric scope 3 and the curvature angles of the curved section 17 in the UD (UP DOWN) direction and LR (LEFT RIGHT) direction may be adjusted to match the average angle.

[0095] Furthermore, by pre-inputting or detecting the coordinates of the patient or operating table, the roll angle of the motorized scope 3 may be aligned with the Z-axis direction (usually vertically upward) relative to the coordinates of the patient or operating table, for example, as shown in Figure 12.

[0096] Furthermore, in this embodiment, the library data or base coordinates may be calibrated according to the tilt of the patient themselves or the tilt of the operating table on which the patient lies. For example, as shown in Figures 13A-13C, the base coordinates may be corrected by detecting the tilt of the patient or the operating table and then adding the detected tilt to the base coordinates. Alternatively, as shown in Figures 14A-14C, the library data may be corrected by detecting the tilt of the patient or the operating table and then adding the detected tilt to the library data. In Figures 13A-13C and 14A-14C, the symbol E indicates the operating table or the patient.

[0097] Furthermore, in this embodiment, the procedure flow may be programmed in advance, and the system may switch to library data associated with each procedure scene according to the treatment process, i.e., the procedure scene. In this case, for example, as shown in Figure 15, the main controller 21 may switch the library data by determining the treatment process using AI (Artificial Intelligence).

[0098] The procedure steps refer to, for example, the anatomical location being treated or the surgeon's actions such as incisions. AI can also be used to estimate the procedure steps based on the anatomical location and the surgeon's actions such as incisions. Possible surgeon actions include severing specific blood vessels, controlling bleeding, clipping blood vessels, compressing organs, and fluorescence observation.

[0099] Furthermore, the procedure steps may be recognized by the operator. For example, the operator may identify the procedure steps using a UI (User Interface) such as voice commands or button operations. In this case, for example, as shown in Figure 16, the main controller 21 may switch library data when the operator identifies the procedure steps using the UI.

[0100] Alternatively, as shown in Figure 17, for example, the main controller 21 may switch to library data associated with a specified procedure scene based on the operator's instructions such as "move to the next procedure," "return to the previous procedure," or "move to the IMA dissection procedure."

[0101] Furthermore, as the procedure progresses, the position of the camera 15 relative to the patient changes, so the library data may be switched based on, for example, the position and orientation of the camera 15 relative to the patient. As shown in Figures 18 and 19, for example, if the insertion point is defined as the origin, the patient's head side as 0 degrees, and the patient's tail side as 180 degrees, the main controller 21 may set the library data associated with the "scene of entering the retrorectal space" when the orientation of the camera 15 is less than threshold A, which indicates the procedure within the triangular frame in Figure 19 (for example, the orientation of the camera 15 is between 210 and 150 degrees). In Figure 19, the symbol F indicates the patient. The same applies to Figure 20.

[0102] Furthermore, as shown in Figures 18 and 20, when the orientation of camera 15 is greater than or equal to threshold A and less than threshold B, which indicates the treatment within the triangular frame in Figure 20 (for example, when the orientation of camera 15 is between 150 and 120 degrees), the main controller 21 may switch to the library data associated with the "first half of the internal approach".

[0103] The main controller 21 may use biological tissue recognized by AI-based tissue recognition as the observation target S. By pre-training tissues such as blood vessels or organs using machine learning, the main controller 21 can automatically set a trained tissue as the observation target S when it appears in the image.

[0104] Furthermore, the main controller 21 may recognize a biological tissue specified by the operator using the UI as the observation target S. In this case, the operator may specify the biological tissue to be observed as the observation target S by using a UI such as a touch panel, or the operator may use the memorized biological tissue as the observation target S by memorizing a point on the biological tissue pointed to by the tip of a forceps or the like. In this way, since there is no need to learn in advance, various biological tissues can be easily set as the observation target S.

[0105] Furthermore, in this embodiment, the scope axis roll angle was used as an example of library data, but instead, for example, the roll angle around the optical axis of the camera 15 on the tip side of the curved section 17, i.e., around the visual axis, may be used. In this case, for example, as shown in Figure 21, the position and attitude controller 25 may calculate the Euler angles (roll, pitch, yaw) based on the angles of each joint of the robot arm 5 by using the forward kinematics of the end effector of the motorized scope 3 in addition to the forward kinematics of the robot arm 5.

[0106] Furthermore, this embodiment can also be applied when performing an unfolding operation after creating an overhead endoscopic view by retracting the motorized scope 3 to near the patient's insertion point, for example, near the trocker. In this case, the library data may include, for example, information on the amount of curvature of the bending section 17 when the curvature amount is zero, information on the advancement and retraction amount of the scope axis when the advancement and retraction amount of the motorized scope 3 is zero, and information on the roll angle of the scope axis when the roll angle is a desired value, as shown in Figure 22. The state in which the curvature amount of the bending section 17 is zero is when the bending section 17 is oriented in the longitudinal axis direction of the insertion section 13. The state in which the advancement and retraction amount of the motorized scope 3 is zero is when the motorized scope 3 is positioned near the patient's insertion point.

[0107] Through control based on such library data, after the curvature of the bending section 17 is returned to zero, the advance and retraction range of the electric scope 3 is reduced to zero, i.e., to the withdrawal limit, and the roll angle of the electric scope 3 is adjusted. By retracting the electric scope 3 to near the trocker with the bending section 17 extending straight along the longitudinal axis of the insertion section 13, a completely overhead endoscopic view can be created. In this case, the position and orientation controller 25 may calculate the current advance and retraction range of the electric scope 3 based on the angles of each joint of the robot arm 5 by using the forward kinematics of the robot arm 5. The calculated current advance and retraction range of the electric scope 3 is stored in the main controller 21.

[0108] [Second Embodiment] Next, an endoscopy system, a procedure support method, and a procedure support program according to a second embodiment of the present invention will be described. The endoscope system 1 according to this embodiment differs from the first embodiment in that, instead of the motorized scope 3, it includes an oblique viewer (endoscope) 41 that does not have a bending section 17, as shown in Figure 23. Hereinafter, parts that share the same configuration as the endoscope system 1, procedure support method, and procedure support program according to the first embodiment will be denoted by the same reference numerals and their descriptions will be omitted.

[0109] The oblique viewing microscope 41 comprises a long tube portion 43 that is inserted into the patient's body cavity, and a camera (imaging optical system) 15 provided at the tip of the tube portion 43. The oblique viewing microscope 41 is positioned such that the optical axis of the camera 15 is inclined with respect to the longitudinal axis (central axis) of the tube portion 43. The tube portion 43 has a tip surface 43a that is inclined with respect to the longitudinal axis of the tube portion 43 and perpendicular to the optical axis of the camera 15. Reference numeral 45 denotes a mounting portion supported by the robot arm 5. The oblique viewing microscope 41 also has a distance measuring function.

[0110] As shown in Figure 24, the oblique viewer 41 has a built-in motorized attachment (field of view direction changing unit) 47. The motorized attachment 47 consists of a motor for the lens barrel 43 that rotates the lens barrel 43 around its longitudinal axis, and a motor for the viewing axis that rotates an image rotator (not shown) that rotates the viewing axis of the camera 15 around its axis, etc. (neither of which are shown).

[0111] By driving the motor for the lens barrel, the lens barrel 43 rotates around its longitudinal axis, which allows the direction in which the front surface 43a, which has a certain angle with respect to the longitudinal axis of the lens barrel 43, faces, and the optical axis direction of the camera 15, i.e., the field of view direction of the oblique view mirror 41, to be changed around the longitudinal axis of the lens barrel 43, as shown in Figure 25.

[0112] Rotating the barrel portion 43 of the oblique viewing mirror 41 around its longitudinal axis is equivalent to changing the distribution of UP, DOWN, LEFT, and RIGHT in the curvature direction of the curvature portion 17 in the motorized scope 3, while keeping the curvature amount constant. For example, as shown in Figure 26, the upward field of view when the tip surface 43a of the oblique viewing mirror 41 and the camera 15 are pointed diagonally upward is equivalent to the field of view when the curvature portion 17 of the motorized scope 3 is bent by about 30 degrees in the UP direction, as shown in Figure 27. Also, for example, as shown in Figure 28, the downward field of view when the tip surface 43a of the oblique viewing mirror 41 and the camera 15 are pointed diagonally downward is equivalent to the field of view when the curvature portion 17 of the motorized scope 3 is bent by about 30 degrees in the DOWN direction, as shown in Figure 29.

[0113] Furthermore, by driving the motor for the viewing axis, for example as shown in Figure 30, the viewing axis of the camera 15 can be rotated around its axis, thereby rotating the vertical orientation of the endoscopic image acquired by the camera 15 around the viewing axis.

[0114] Multiple library data stored in the auxiliary storage device 27 include at least one relative parameter relating to the relative position and orientation between the camera 15 of the oblique endoscope 41 and the observation target S captured by the camera 15. Examples of relative parameters include distance information between the camera 15 and the observation target S, rotation angle information about the longitudinal axis of the endoscope tube 43, and roll angle information about the visual axis. The roll angle information about the visual axis is the angle about the axis of the visual axis of the camera 15. Hereinafter, this information will be referred to as distance information, endoscope tube angle information, and roll angle information about the visual axis. The auxiliary storage device 27 stores at least one of the distance information, endoscope tube angle information, and roll angle information about the visual axis, associated with each procedure scene, in order to realize each endoscopic field of view that has been set in advance for each procedure scene.

[0115] The procedure support method includes, instead of the steps of changing at least one of the angles of the camera 15, the position and orientation of the motorized scope 3 (steps S4 and S8 in the first embodiment), a step of changing at least one of the angles of the tube portion 43 around its longitudinal axis, the position and orientation of the oblique viewer 41, based on retrieved library data. The procedure support program causes the controllers 21, 23, and 25 of the control device 7 to execute this step.

[0116] For example, when the control device 7 retrieves distance information from the auxiliary storage device 27, it controls at least one of the electric attachment 47 and the robot arm 5 to match the actual distance between the camera 15 and the object of observation S, calculated from the image information of the object of observation S, with the retrieved distance information. Specifically, as shown in Figure 24, the main controller 21 compares the current distance measurement to the object of observation S with the retrieved distance information to calculate a trajectory to match the distance between the camera 15 and the object of observation S with the distance information. The main controller 21 then distributes the information into a position and attitude command indicating the amount of change in position and attitude around the pivot point, an angle command around the visual axis, and a barrel angle command. The position and attitude command is then input to the position and attitude controller 25, and the angle command around the visual axis and the barrel angle command are input to the field of view controller 23.

[0117] The position and orientation controller 25, in accordance with the position and orientation command, uses the inverse kinematics of the robot arm 5 to determine the amount of drive required for each joint to match the distance between the camera 15 and the observation target S with the distance information. The determined amount of drive for each joint is input to each motor of the robot arm 5 as an angle command (endoscopic motion command) for each joint.

[0118] Meanwhile, the field of view controller 23 determines the rotation angle around the optical axis and the rotation angle of the lens barrel 43 necessary to match the distance between the camera 15 and the observation target S with the distance information, according to the angle command around the optical axis and the angle command of the lens barrel 43. The determined rotation angle around the optical axis and the rotation angle of the lens barrel 43 are input as motor angle commands to the optical axis motor and the lens barrel motor of the electric attachment 47, respectively.

[0119] Each joint of the robot arm 5 operates according to its respective angle command, and each motor of the electric attachment 47 is driven according to its respective motor angle command, thereby obtaining an endoscopic field of view in which the distance between the camera 15 and the observation target S matches the distance information.

[0120] Furthermore, when the control device 7 retrieves the barrel angle information from the auxiliary storage device 27, the main controller 21 first compares the current angle of the barrel 43 around its longitudinal axis with the retrieved barrel angle information to calculate a trajectory that will match the rotation angle of the barrel 43 to the barrel angle information. Then, the position and attitude controller 25 and the field of view controller 23 control at least one of the motorized attachment 47 and the robot arm 5. This makes the angle of the barrel 43 around its longitudinal axis, calculated from the image information of the object of observation S, match the retrieved barrel angle information. The control by the position and attitude controller 25 and the field of view controller 23 is the same as in the case of distance information, so a detailed explanation is omitted.

[0121] Furthermore, when the control device 7 retrieves roll angle information around the visual axis from the auxiliary storage device 27, the main controller 21 compares the current angle around the axis of the camera 15's visual axis with the retrieved roll angle information around the visual axis to calculate a trajectory that matches the angle around the axis of the visual axis to the roll angle information around the visual axis. Then, the position and orientation controller 25 and the field of view controller 23 control at least one of the motorized attachment 47 and the robot arm 5 to match the angle around the axis of the camera 15's visual axis, calculated from the image information of the object being observed S, to the retrieved roll angle information around the visual axis. The control by the position and orientation controller 25 and the field of view controller 23 is the same as in the case of distance information, so the explanation is omitted.

[0122] When the field of view controller 23 drives the motor for the telescope tube of the electric attachment 47, it calculates the current angle of the telescope tube 43 by converting the motor angle of the motor for the telescope tube into an angle around the longitudinal axis of the telescope tube 43. Furthermore, when the field of view controller 23 drives the motor for the visual axis of the electric attachment 47, it calculates the current angle around the axis of the visual axis by converting the motor angle of the visual axis motor into an angle around the axis of the visual axis. The calculated current angle of the telescope tube 43 and the current angle around the axis of the visual axis are stored in the main controller 21, respectively.

[0123] According to the endoscope system 1, procedure support method, and procedure support program of this embodiment, each library data includes relative parameters relating to the relative position and orientation between the camera 15 of the oblique endoscope 41 and the observation target S. Therefore, the operator can be provided with an endoscopic field of view that corresponds to both the procedure scene and the observation target S. In this case, for example, if the angle of the barrel 43 around its longitudinal axis is changed based on the barrel angle information, the field of view direction of the oblique endoscope 41, which has a constant angle with respect to the longitudinal axis of the barrel 43, changes around the longitudinal axis of the barrel 43, thereby switching the field of view direction of the oblique endoscope 41 to diagonally upward or diagonally downward, etc. As a result, the field of view direction of the oblique endoscope 41 can be directed towards the desired observation target simply by rotating the barrel 43 around its longitudinal axis. Therefore, even when an oblique endoscope 41 is used as an endoscope, the observation target specified by the operator can be easily placed in the endoscopic field of view.

[0124] [Third Embodiment] Next, an endoscopy system, a procedure support method, and a procedure support program according to a third embodiment of the present invention will be described. The endoscope system 1 according to this embodiment differs from the first and second embodiments in that, instead of the motorized scope 3 and oblique viewer 41, it includes, for example, a straight viewer (endoscope) 51 without a bending section 17, as shown in Figure 31. Hereinafter, parts that share the same components as the endoscope system 1, procedure support method, and procedure support program according to the first and second embodiments will be denoted by the same reference numerals and their descriptions will be omitted.

[0125] The straight-viewing endoscope 51 comprises a long tube portion 53 that is inserted into the patient's body cavity, and a camera (imaging optical system) 15 provided at the tip of the tube portion 53. The straight-viewing endoscope 51 is positioned so that the longitudinal axis (central axis) of the tube portion 53 and the optical axis of the camera 15 coincide. The tube portion 53 has a tip surface 53a that is perpendicular to the longitudinal axis of the tube portion 53 and the optical axis of the camera 15. Reference numeral 55 denotes a mounting portion supported by the robot arm 5. The straight-viewing endoscope 51 also has a distance measuring function.

[0126] As shown in Figure 32, the straight-viewing endoscope 51 has a built-in motorized attachment 57. The motorized attachment 57 consists of a motor for the viewing axis that rotates the viewing axis of the camera 15 around its axis, that is, around the longitudinal axis of the endoscope barrel 53. As shown in Figure 33, by rotating the viewing axis of the camera 15 around its axis, the vertical orientation of the endoscopic image acquired by the camera 15 can be rotated around the viewing axis.

[0127] The robot arm 5 functions as a field of view direction changing unit that changes the angle around a pivot axis (rotation axis) perpendicular to the longitudinal axis of the microscope tube 53 in response to a change in the position where the object being observed on the monitor 11 screen is captured. For example, if the object being observed, such as biological tissue, is captured in the center of the monitor 11 screen, and the position where the object is captured is changed to the upper edge of the screen, the angle around the pivot point P is changed, with the mounting portion 55 of the straight-viewing microscope 51 acting as the pivot point P, as shown in Figures 34 and 36. As a result, the field of view direction of the straight-viewing microscope 51 is changed to capture the object at the upper edge of the screen, and as a result, the position of the object being observed on the monitor 11 screen is offset in a direction that intersects the optical axis of the camera 15, as shown in Figures 35 and 37.

[0128] As shown in Figures 34 and 35, when the object of observation is captured in the center of the endoscopic field of view, the object is located in the center of the screen, but there are parts that are hidden and difficult to see. In contrast, as shown in Figures 36 and 37, when the object of observation is captured at the edge of the field of view, the object is located at the edge of the screen, and parts that were previously hidden become visible. In other words, by changing the field of view direction of the straight endoscope 51 so that the object of observation is captured at the edge of the field of view, an endoscopic field of view at an angle in which the object of observation is peered into can be obtained. In Figures 34 and 36, reference numeral 59 indicates a treatment instrument, and reference numeral S indicates the object of observation.

[0129] Multiple library data stored in the auxiliary storage device 27 include at least one relative parameter relating to the relative position and orientation between the camera 15 of the direct viewing endoscope 51 and the observation target S captured by the camera 15. Examples of relative parameters include distance information between the camera 15 and the observation target S, position information for capturing the observation target on the screen, and roll angle information around the visual axis. Hereinafter, this information will be referred to as distance information, target position information, and roll angle information around the visual axis. The auxiliary storage device 27 stores at least one of the distance information, target position information, and roll angle information around the visual axis, associated with each procedure scene, in order to realize each endoscopic field of view that has been set in advance for each procedure scene. For example, the desired position for capturing the observation target on the screen of the monitor 11 may be set as target position information by voice operation using the headset 35 or button operation using the hand switch 37.

[0130] The procedure support method includes, instead of the steps of changing at least one of the angles of the camera 15, the position and orientation of the motorized scope 3 (steps S4 and S8 in the first embodiment), a step of changing at least one of the angles of the straight-viewing endoscope 51 around the pivot point P, the position and orientation of the straight-viewing endoscope 51 based on called library data, and the procedure support program causes the controllers 21, 23, and 25 of the control device 7 to perform this step.

[0131] For example, when the control device 7 retrieves distance information from the auxiliary storage device 27, it controls at least one of the electric attachment 57 and the robot arm 5 to match the actual distance between the camera 15 and the object being observed S, calculated from the image information of the object being observed S, with the retrieved distance information. Specifically, as shown in Figure 32, the main controller 21 compares the current distance measurement to the object being observed S with the retrieved distance information to calculate a trajectory that will match the distance between the camera 15 and the object being observed S with the distance information. The main controller 21 then divides the input into position and attitude commands indicating the amount of change in position and attitude around the pivot point and angle commands around the visual axis, and then inputs the position and attitude commands to the position and attitude controller 25 and the angle commands around the visual axis to the field of view controller 23.

[0132] The position and orientation controller 25, in accordance with the position and orientation command, uses the inverse kinematics of the robot arm 5 to determine the amount of drive required for each joint to match the distance between the camera 15 and the observation target S with the distance information. The determined amount of drive for each joint is input to each motor of the robot arm 5 as an angle command (endoscopic motion command) for each joint.

[0133] Meanwhile, the field of view controller 23 determines the rotation angle around the visual axis necessary to match the distance between the camera 15 and the observation target S with the distance information, according to the angle command around the visual axis. The determined rotation angle around the visual axis is input as a motor angle command to the visual axis motor of the electric attachment 57.

[0134] Each joint of the robot arm 5 operates according to its respective angle command, and the motor for the viewing axis of the electric attachment 57 rotates according to the motor angle command, thereby obtaining an endoscopic field of view in which the distance between the camera 15 and the object of observation S matches the distance information.

[0135] Furthermore, when the control device 7 retrieves target position information from the auxiliary storage device 27, the main controller 21 compares the current position of the object being observed on the monitor 11 screen with the retrieved target position information to calculate a trajectory to match the position of the object being observed on the monitor 11 screen to the target position information. Then, the position and orientation controller 25 and the field of view controller 23 control at least one of the motorized attachment 57 and the robot arm 5. This makes the position of the object being observed on the screen, calculated from the image information of the object being observed S, match the retrieved target position information. The control by the position and orientation controller 25 and the field of view controller 23 is the same as in the case of distance information, so a detailed explanation is omitted.

[0136] Furthermore, when the control device 7 retrieves roll angle information around the visual axis from the auxiliary storage device 27, the main controller 21 compares the current angle around the axis of the camera 15's visual axis with the retrieved roll angle information around the visual axis to calculate a trajectory that matches the angle around the axis of the visual axis to the roll angle information around the visual axis. Then, the position and orientation controller 25 and the field of view controller 23 control at least one of the motorized attachment 57 and the robot arm 5 to match the angle around the axis of the camera 15's visual axis, calculated from the image information of the object being observed S, to the retrieved roll angle information around the visual axis. The control by the position and orientation controller 25 and the field of view controller 23 is the same as in the case of distance information, so the explanation is omitted.

[0137] When the field of view controller 23 drives the motor for the visual axis of the electric attachment 57, it calculates the current angle around the axis of the visual axis by converting the motor angle of the visual axis motor into an angle around the axis of the visual axis. The calculated current angle is stored in the main controller 21.

[0138] According to the endoscope system 1, procedure support method, and procedure support program of this embodiment, each library data includes relative parameters relating to the relative position and posture between the camera 15 of the straight-viewing endoscope 51 and the object of observation S. Therefore, the operator can be provided with an endoscopic field of view that corresponds to both the procedure scene and the object of observation S. In this case, for example, if the position on the monitor 11 screen where the object of observation is captured is changed to the edge of the screen based on the object position information, the angle around the pivot axis P of the straight-viewing endoscope 51 is changed, thereby changing the field of view direction of the straight-viewing endoscope 51 to one in which the object of observation is captured at the edge of the field of view. This makes it possible to create an endoscopic field of view at an angle in which the object of observation is peered into. Therefore, even when a straight-viewing endoscope 51 is used as the endoscope, the object of observation specified by the operator can be easily placed in the endoscopic field of view.

[0139] Although each embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and design changes and the like are also included within the scope of the gist of the present invention. For example, the present invention is not limited to being applied to the above embodiments and their respective modifications, but may also be applied to embodiments that appropriately combine these embodiments and their modifications, and is not particularly limited.

[0140] Alternatively, for example, the auxiliary storage device 27 may store each library data along with metadata that can identify it. In this case, the main controller 21 may acquire metadata input by the operator according to the procedure scene, or it may acquire metadata corresponding to the procedure scene identified by processing image information. The main controller 21 may then retrieve the library data corresponding to the acquired metadata from the auxiliary storage device 27.

[0141] Furthermore, although this embodiment describes a robot arm 5 of a 6-axis articulated robot as an example of an electric arm, the electric arm does not need to have 6 degrees of freedom; it may be a robot arm with fewer degrees of freedom. For example, a robot arm consisting of a roll axis and two pitch axes, with three joints at the tip that allow for 3 degrees of freedom of movement, may be used. In this case, for example, the robot arm may have an electric attachment that rotates the electric scope 3. Alternatively, a robot arm with a 4-degree-of-freedom configuration having a roll joint at the tip may be used. [Explanation of Symbols]

[0142] 1 Endoscopy System 3. Motorized endoscope 5. Robot arm (electric arm, field of view direction changing unit) 13 Insertion section (long section) 17 Curved section 21. Main Controller (Processor) 23. Curved Controller (Processor) 25 Position and Attitude Controller (Processor) 27 Auxiliary storage device (storage device) 35 Headset (Input Unit) 37. Handheld switch (input section) 39 Footswitch (Input Section) 41. Oblique viewer (endoscope) 43. Lens tube section 47. Electric attachment (viewing direction changing section) 51. Direct viewing endoscope (endoscopy) 53 Lens Tube Section S Observation Subject

Claims

1. An endoscope equipped with an imaging optical system for photographing the object to be observed, An electric arm that changes the position and orientation of the endoscope while supporting it, The endoscope includes a field of view direction changing unit that can change the field of view direction, A storage device that stores multiple library data to realize endoscopic views associated with each procedure scene, Equipped with at least one processor, The library data includes at least one relative parameter relating to the relative position and orientation between the imaging optical system and the object being observed. The endoscope comprises a long section at its tip where the imaging optical system is located, The field of view direction changing section is a bending section that can change the angle of the imaging optical system in the endoscope, The library data includes parameters relating to the orientation of the imaging optical system on the base coordinates, which are determined by the sum of the inclination of the longitudinal axis of the long portion on the base coordinates and the curvature of the curved portion. An endoscope system in which the processor retrieves library data associated with any procedure scene from the storage device, and then controls at least one of the field of view direction changing unit and the electric arm based on the retrieved library data.

2. The endoscopic system according to claim 1, wherein the processor identifies the procedure scene by processing image information acquired by the imaging optical system, and then retrieves the library data associated with the identified procedure scene from the storage device.

3. The relative parameter is distance information between the imaging optical system and the object being observed. The endoscope system according to claim 1, wherein the processor controls at least one of the field of view direction changing unit and the electric arm, thereby matching the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, with the distance information.

4. The relative parameter is distance information between the imaging optical system and the object being observed, The endoscope system according to claim 2, wherein the processor controls at least one of the field of view direction changing unit and the electric arm to match the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, with the distance information.

5. The relative parameter is orientation information of the imaging optical system with respect to the object being observed. The endoscope system according to claim 1, wherein the processor controls at least one of the field of view direction changing unit and the electric arm so that the orientation of the imaging optical system relative to the object of observation, calculated from the image information acquired by the imaging optical system, matches the orientation information.

6. The relative parameter is orientation information of the imaging optical system with respect to the object being observed, The endoscope system according to claim 2, wherein the processor controls at least one of the field of view direction changing unit and the electric arm so that the orientation of the imaging optical system relative to the object of observation, calculated from the image information acquired by the imaging optical system, matches the orientation information.

7. The relative parameter is the inclination information of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, about the optical axis. The endoscope system according to claim 1, wherein the processor controls at least one of the field of view direction changing unit and the electric arm, thereby matching the inclination of the elongated object being observed around the optical axis of the imaging optical system, calculated from the image information acquired by the imaging optical system, with the inclination information.

8. The relative parameter is inclination information about the optical axis of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, The endoscope system according to claim 2, wherein the processor controls at least one of the field of view direction changing unit and the electric arm, thereby matching the inclination of the elongated object being observed around the optical axis of the imaging optical system, calculated from the image information acquired by the imaging optical system, with the inclination information.

9. The endoscopic system according to any one of claims 1 to 8, wherein the processor processes image information acquired by the imaging optical system to recognize the object to be observed.

10. The endoscope system according to any one of claims 1 to 8, further comprising an input unit that causes the user to specify the object to be observed based on image information acquired by the imaging optical system.

11. The processor, The process of retrieving library data associated with an arbitrary procedure scene from a storage device, among multiple library data sets for realizing endoscopic views associated with each procedure scene, The process includes the step of changing at least one of the field of view direction, position, and orientation of the field of view direction changing section of the endoscope that photographs the object of observation, based on the retrieved library data. Each of the library data includes at least one relative parameter relating to the relative position and orientation between the imaging optical system of the endoscope and the object being observed. The endoscope comprises a long section at which the imaging optical system is located at the tip, and a curved section in which the angle of the imaging optical system can be changed. By changing the angle of the imaging optical system with the bending portion, the field of view direction of the endoscope is changed. A method for operating an endoscope system, wherein the library data includes parameters relating to the orientation of the imaging optical system on the base coordinates, which are determined by the sum of the inclination of the longitudinal axis of the long portion on the base coordinates and the amount of curvature of the curved portion.

12. The method for operating an endoscope system according to claim 11, wherein the processor identifies the procedure scene by processing image information acquired by the imaging optical system, and then retrieves the library data associated with the identified procedure scene from the storage device.

13. The relative parameter is distance information between the imaging optical system and the object being observed. The method for operating an endoscope system according to claim 11, wherein the processor changes at least one of the field of view direction, position, and orientation of the endoscope, thereby matching the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, with the distance information.

14. The relative parameter is distance information between the imaging optical system and the object being observed, The method for operating an endoscope system according to claim 12, wherein the processor changes at least one of the field of view direction, position, and orientation of the endoscope, thereby matching the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, with the distance information.

15. The relative parameter is orientation information of the imaging optical system with respect to the object being observed. The method for operating an endoscope system according to claim 11, wherein the processor changes at least one of the field of view direction, position, and orientation of the endoscope, thereby matching the orientation of the imaging optical system with respect to the object of observation, calculated from the image information acquired by the imaging optical system, to the orientation information.

16. The relative parameter is orientation information of the imaging optical system with respect to the object being observed, The method for operating an endoscope system according to claim 12, wherein the processor changes at least one of the field of view direction, position, and orientation of the endoscope, thereby matching the orientation of the imaging optical system with respect to the object of observation, calculated from the image information acquired by the imaging optical system, with the orientation information.

17. The relative parameter is the inclination information of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, about the optical axis. The method for operating an endoscope system according to claim 11, wherein the processor changes at least one of the field of view direction, position, and orientation of the endoscope, thereby matching the tilt of the elongated object being observed around the optical axis of the imaging optical system, which is calculated from the image information acquired by the imaging optical system, with the tilt information.

18. The relative parameter is inclination information about the optical axis of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, The method for operating an endoscope system according to claim 12, wherein the processor changes at least one of the field of view direction, position, and orientation of the endoscope, thereby matching the tilt of the elongated object being observed around the optical axis of the imaging optical system, which is calculated from the image information acquired by the imaging optical system, with the tilt information.

19. A method for operating an endoscope system according to any one of claims 11 to 18, wherein the object to be observed is recognized by processing the image information acquired by the imaging optical system with the processor.

20. A method for operating an endoscope system according to any one of claims 11 to 18, wherein the observation target is specified by the user based on image information acquired by the imaging optical system.

21. The process of retrieving library data associated with an arbitrary procedure scene from a storage device, among multiple library data sets for realizing endoscopic views associated with each procedure scene, Based on the retrieved library data, the computer is instructed to perform the following steps: change at least one of the field of view direction, position, and orientation of the field of view direction changing section of the endoscope that photographs the object of observation; Each of the library data includes at least one relative parameter relating to the relative position and orientation between the imaging optical system of the endoscope and the object being observed. The endoscope comprises a long section at which the imaging optical system is located at the tip, and a curved section in which the angle of the imaging optical system can be changed. By changing the angle of the imaging optical system with the bending portion, the field of view direction of the endoscope is changed. A procedure support program that includes a parameter relating to the orientation of the imaging optical system on the base coordinates, which is determined by the sum of the inclination of the longitudinal axis of the long portion on the base coordinates and the amount of curvature of the curved portion.

22. A step of identifying the procedure scene by processing the image information acquired by the imaging optical system, A procedure support program according to claim 21, which causes a computer to perform the steps of: calling up the library data associated with the identified procedure scene from the storage device.

23. The relative parameter is distance information between the imaging optical system and the object being observed. The procedure support program according to claim 21, wherein the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, is matched to the distance information by changing at least one of the field of view direction, position, and orientation of the endoscope.

24. The relative parameter is distance information between the imaging optical system and the object being observed, The procedure support program according to claim 22, wherein the distance between the imaging optical system and the object being observed, calculated from the image information acquired by the imaging optical system, is matched to the distance information by changing at least one of the field of view direction, position, and orientation of the endoscope.

25. The relative parameter is orientation information of the imaging optical system with respect to the object being observed. The procedure support program according to claim 21, wherein the orientation of the imaging optical system relative to the object being observed, calculated from the image information acquired by the imaging optical system, is matched to the orientation information by changing at least one of the field of view direction, position, and orientation of the endoscope.

26. The relative parameter is orientation information of the imaging optical system with respect to the object being observed, The procedure support program according to claim 22, wherein the orientation of the imaging optical system relative to the object of observation, calculated from the image information acquired by the imaging optical system, is matched to the orientation information by changing at least one of the field of view direction, position, and orientation of the endoscope.

27. The relative parameter is the inclination information of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, about the optical axis. The procedure support program according to claim 21, which changes at least one of the field of view direction, position, and orientation of the endoscope to match the tilt information of the elongated observation target around the optical axis of the imaging optical system, calculated from the image information acquired by the imaging optical system.

28. The relative parameter is inclination information about the optical axis of the elongated object being observed, which extends in a direction perpendicular to the optical axis of the imaging optical system, The procedure support program according to claim 22, which changes at least one of the field of view direction, position, and orientation of the endoscope to match the tilt information of the elongated observation target around the optical axis of the imaging optical system, calculated from the image information acquired by the imaging optical system.

29. A procedure support program according to any one of claims 21 to 28, which causes a computer to perform the step of recognizing the object to be observed by processing image information acquired by the imaging optical system.

30. A procedural support program according to any one of claims 21 to 28, wherein the user specifies the object to be observed based on the image information acquired by the imaging optical system.

Images