Visual field control device, visual field control system, visual field control method, and visual field control program

Abstract

To provide visual field control device, a system, a method, and a program capable of calling out a desired endoscope visual field corresponding to a procedural scene.SOLUTION: A visual field control device includes a processor for executing candidate presentation processing for presenting information on library data for achieving an endoscope visual field associated with a procedural scene to a user as an execution candidate for endoscope visual field adjustment, visual field adjustment processing for outputting instructions to change at least one of the position, the posture, and the visual field direction of an electric scope when the user hopes that the endoscope visual field adjustment is executed, and candidate changing processing for presenting information on another library data classified to be of the same procedural scene as that of the library data as the execution candidate to the user as another execution candidate when the user does not hope that the endoscope visual field adjustment is executed. The library data includes at least one of relative parameters concerning the relative position and posture between the electric scope and an observation object.SELECTED DRAWING: None

Description

[Technical field]

[0001] The present invention relates to a visual field control device, a visual field control system, a visual field control method, and a visual field control program. [Background technology]

[0002] Conventionally, there is known a technology for providing an endoscopic field of view according to a user's specification during endoscopic surgery (see, for example, Patent Document 1). In the technology described in Patent Document 1, when a user selects an object such as a blood vessel, nerve, or tumor that he or she wishes to track, an endoscopic field of view is generated based on tracking parameters such as preset distance and speed according to the selected tracking object. In the technology described in Patent Document 1, tracking parameters are set in one-to-one correspondence for each tracking object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent Application Publication No. 2020 / 0015906 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are cases where the endoscopic field of view needs to be finely adjusted depending on the user's skill level, differences in procedures, differences between users such as preference for the field of view and the type of endoscope, or differences between cases such as the progression of a tumor, obesity cases, adhesions and blood vessel running patterns, etc. In such cases, the endoscopic field of view needs to be manually adjusted, which creates a problem of impeding smooth progress of the procedure.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a field of view control device, a field of view control system, a field of view control method, and a field of view control program that allow a user to easily call up a desired endoscopic field of view according to a procedure scene. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention provides the following means. A first aspect of the present invention is a visual field control device that includes a processor that executes a scene determination process that determines a current procedure scene based on at least one of an endoscopic image of an observation object acquired by an endoscope, a position and posture of the endoscope, a viewing direction of the endoscope, and input information of a procedure scene by a user; a candidate presentation process that presents to a user information on library data for realizing an endoscopic field of view associated with the determined procedure scene as an execution candidate for endoscopic visual field adjustment; a visual field adjustment process that, if the user wishes to perform endoscopic visual field adjustment based on the presented library data, outputs an instruction to change at least one of the position, posture, and viewing direction of the endoscope based on the library data; and a candidate change process that, if the user does not wish to perform endoscopic visual field adjustment based on the presented library data, presents to the user information on other library data that is classified into the same procedure scene as the library data of the execution candidate as the other execution candidate, wherein the library data includes at least one relative parameter related to the relative position and posture of the endoscope and the observation object.

[0007] According to this aspect, during endoscopic surgery, after the current procedure scene is determined by operation of the processor, information regarding library data for realizing the endoscopic field of view associated with that procedure scene is presented to the user as an implementation candidate for adjusting the endoscopic field of view.

[0008] Then, if the user wishes to perform endoscopic visual field adjustment based on the presented library data, the processor outputs an instruction to change at least one of the position, posture, and visual field direction of the endoscope based on the library data. This provides the user with a desired endoscopic visual field. On the other hand, if the user does not wish to perform endoscopic visual field adjustment based on the presented library data, information on other library data that is classified into the same procedure scene as the candidate library data is presented to the user as other candidates for execution. This allows the user to select more preferred library data, making it possible to provide the endoscopic visual field that the user desires.

[0009] Therefore, the user can call up the desired endoscopic field of view corresponding to the procedure scene, without the need for manual adjustment, by simply selecting and choosing from the library data presented by the processor according to the user's preferences or case.

[0010] The field of view control device of the above-mentioned aspect may be configured so that the library data includes numerical information representing at least one of the position, posture, and field of view direction of the endoscope when endoscopic field of view adjustment is performed based on the library data, and the processor presents the numerical information to a user as information regarding the library data during the candidate presentation process and the candidate change process. With this configuration, the user can visualize the desired endoscopic image based on the information on the library data presented by the processor. Also, by using numerical information as the information on the library data, the amount of information to be processed is smaller than that of an image, and therefore the image can be displayed more quickly.

[0011] In addition, the field of view control device of the above-mentioned aspect may be configured so that the library data includes a schematic diagram of the endoscopic image representing at least one of the position, posture, and field of view direction of the endoscope when endoscopic field of view adjustment is performed based on the library data, and the processor presents the schematic diagram of the endoscopic image to the user as information related to the library data during the candidate presentation process and the candidate change process. By using a schematic diagram of an endoscopic image as information on the library data, the characteristics of the field of view can be more easily understood than with an actual endoscopic image, making it easier to imagine the field of view after field adjustment.

[0012] In addition, the field of view control device of the above-mentioned aspect may be configured so that the library data includes a schematic diagram showing the position and posture of the endoscope, which represents at least one of the position, posture and field of view direction of the endoscope when endoscopic field of view adjustment is performed based on the library data, and the processor presents to the user the schematic diagram showing the position and posture of the endoscope as information related to the library data during the candidate presentation process and the candidate change process. By using a schematic diagram showing the position and posture of the endoscope as information about the library data, it becomes easier to visualize the positional relationship between the endoscope and the user when adjusting the field of view.

[0013] In addition, the field of view control device of the above-mentioned aspect may be configured so that the library data includes endoscopic images acquired in past cases that represent at least one of the position, posture, and field of view direction of the endoscope when endoscopic field of view adjustment is performed based on the library data, and the processor presents to the user the endoscopic images acquired in the past cases as information related to the library data during the candidate presentation process and the candidate change process. By using endoscopic images acquired in past cases as information related to library data, it becomes easier to grasp subtle differences in the field of view between library data.

[0014] The field of view control device of the above-mentioned aspect may be configured so that the library data is associated with tag information representing attributes or characteristics of the library data in addition to the procedure scene, and the processor performs a filtering process to narrow down the library data presented as the execution candidates in accordance with a filtering condition based on the tag information specified by a user.

[0015] With this configuration, the processor narrows down the library data that the user desires from among multiple library data that are classified into the same procedure scene based on the attributes or features of the library data, thereby enabling the user to more quickly and accurately call up the library data that the user desires.

[0016] The field of view control device of the above-mentioned aspect may be configured so that the library data is associated with tag information that represents attributes or characteristics of the library data in addition to the procedure scene, and the processor performs a ranking process in which the library data with the highest priority are presented as candidates for execution in accordance with a priority order based on the tag information specified by a user.

[0017] With this configuration, the processor ranks multiple library data classified into the same procedure scene based on the attributes or features of the library data in accordance with the user's request, thereby enabling the user to call up the library data desired more quickly and efficiently.

[0018] In the visual field control device according to the above aspect, the processor may calculate the priority order based on the tag information and a weighting constant set by a user. By using weighting constants set by the user, prioritization can be performed that reflects more detailed user needs.

[0019] The field of view control device according to the above aspect may be configured so that the processor executes a scene presentation process in which the determined current procedure scene is presented to a user, and if the user determines that the presented procedure scene matches the actual procedure scene, the candidate presentation process is executed based on that procedure scene, and if the user determines that the presented procedure scene does not match the actual procedure scene, a re-determination process is executed in which the current procedure scene is re-determined. By adding the user's judgment to the correctness of the procedure scene judgment derived by the processor, it is possible to improve the accuracy of extraction of library data desired by the user.

[0020] A second aspect of the present invention is a field of view control system comprising the endoscope, a position and posture change unit that changes the position and posture of the endoscope, a field of view direction change unit that can change the field of view direction of the endoscope, and any one of the above-mentioned field of view control devices, wherein the processor controls at least one of the position and posture change unit and the field of view direction change unit based on the presented library data.

[0021] In the field of view control system according to the above aspect, the field of view direction changing unit may be a bending mechanism capable of changing an angle of an imaging optical system in the endoscope. With this configuration, the angle of the imaging optical system changes according to the amount of bending of the bending mechanism, making it possible to change the viewing direction of the endoscope with a simple configuration that only requires bending the bending mechanism.

[0022] In the field of view control system according to the above aspect, the endoscope is an oblique mirror having a barrel portion that houses an imaging optical system, and the optical axis of the imaging optical system is arranged at an angle relative to the central axis of the barrel portion, and the field of view direction change unit may change the angle around the central axis of the barrel portion.

[0023] When the angle around the central axis of the barrel is changed, the viewing direction of the oblique scope, which has a certain angle with respect to the central axis of the barrel, changes around the central axis of the barrel, so that the viewing direction of the oblique scope can be switched to diagonally upward or diagonally downward, etc. This makes it possible to direct the viewing direction of the oblique scope to a desired observation target simply by rotating the barrel around the central axis. Therefore, even when an oblique scope is used as an endoscope, it is easy to place the observation target specified by the surgeon in the endoscope field of view.

[0024] In the field of view control system according to the above aspect, the endoscope is a direct-view endoscope having a barrel section that houses an imaging optical system and is arranged with the central axis of the barrel section coinciding with the optical axis of the imaging optical system, and the field of view direction change section may change the angle around a rotation axis perpendicular to the central axis of the barrel section in response to a change in the position at which the object of observation is captured on a display screen that displays the endoscopic image acquired by the imaging optical system.

[0025] The viewing direction changing unit changes the angle around the rotation axis perpendicular to the central axis of the tube unit in response to a change in the position on the display screen where the observation target is captured, thereby changing the viewing direction of the direct endoscope to a direction that captures the observation target at the changed position on the display screen. As a result, for example, when the position where the observation target is captured is changed to the edge of the display screen, the viewing direction of the direct endoscope is changed to a direction that captures the observation target at the edge of the angle of view, thereby making it possible to create an endoscopic field of view at an angle at which the observation target is peered into. Therefore, even when a direct endoscope is used as the endoscope, the observation target specified by the surgeon can be easily placed in the endoscopic field of view.

[0026] The visual field control system according to the above aspect may include a memory that stores the library data in association with each of the procedure scenes, and after the processor calls up from the memory the library data associated with the determined current procedure scene, it may present information about the called-up library data as the execution candidate.

[0027] A third aspect of the present invention is a view control method which determines a current procedure scene based on at least one of an endoscopic image of an observation object acquired by an endoscope, a position and posture of the endoscope, a viewing direction of the endoscope, and procedure scene input information by a user, and presents information regarding library data for realizing an endoscopic view associated with the determined procedure scene to the user as an execution candidate for endoscopic view adjustment, and if the user wishes to perform endoscopic view adjustment based on the presented library data, changes at least one of the position, posture, and viewing direction of the endoscope based on the library data, and if the user does not wish to perform endoscopic view adjustment based on the presented library data, presents information regarding other library data classified into the same procedure scene as the library data of the execution candidate to the user as the other execution candidate, and the library data includes at least one relative parameter related to the relative position and posture of the endoscope and the observation object.

[0028] The field of view control method of the above aspect may be such that the library data includes numerical information representing at least one of the position, posture, and field of view direction of the endoscope when endoscopic field of view adjustment is performed based on the library data, a schematic diagram of the endoscopic image, a schematic diagram showing the position and posture of the endoscope, and at least one of the endoscopic images obtained in the past cases, and at least one of the numerical information, the schematic diagram of the endoscopic image, the schematic diagram showing the position and posture of the endoscope, and the endoscopic images obtained in the past cases are presented to a user as the execution candidate and other execution candidates.

[0029] In the field of view control method according to the above aspect, the library data is associated with tag information which represents attributes or characteristics of the library data in addition to the procedure scene, and the library data presented as the execution candidates may be narrowed down in accordance with narrowing-down conditions based on the tag information specified by a user.

[0030] A fourth aspect of the present invention is a field of view control program that causes a computer to execute a scene determination process for determining a current procedure scene based on at least one of an endoscopic image of an observation object acquired by an endoscope, a position and posture of the endoscope, a viewing direction of the endoscope, and input information of a procedure scene by a user; a candidate presentation process for presenting to a user information on library data for realizing an endoscopic field of view associated with the determined procedure scene as an execution candidate for endoscopic field of view adjustment; a field of view adjustment process for changing at least one of the position, posture, and viewing direction of the endoscope based on the library data if the user wishes to perform endoscopic field of view adjustment based on the presented library data; and a candidate change process for presenting to a user information on other library data classified into the same procedure scene as the library data of the execution candidate as the other execution candidate if the user does not wish to perform endoscopic field of view adjustment based on the presented library data, wherein the library data includes at least one relative parameter related to the relative position and posture of the endoscope and the observation object.

[0031] The field of view control program of the above-mentioned aspect may include library data including numerical information representing at least one of the position, posture, and field of view direction of the endoscope when endoscopic field of view adjustment is performed based on the library data, a schematic diagram of the endoscopic image, a schematic diagram showing the position and posture of the endoscope, and at least one of endoscopic images obtained in past cases, and during the candidate presentation process and the candidate change process, at least one of the numerical information, the schematic diagram of the endoscopic image, the schematic diagram showing the position and posture of the endoscope, and the endoscopic image obtained in the past case may be presented to the user as information related to the library data.

[0032] The field of view control program of the above-mentioned aspect may be configured so that the library data is associated with tag information representing attributes or characteristics of the library data in addition to the procedure scene, and the computer is caused to execute a filtering process to narrow down the library data presented as the execution candidates in accordance with a narrowing-down condition based on the tag information specified by a user. Effect of the Invention

[0033] According to the present invention, an effect is achieved in that a user can easily call up a desired endoscopic field of view corresponding to a procedure scene. [Brief description of the drawings]

[0034] [Figure 1] 1 is a schematic configuration diagram of a visual field control system according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram illustrating a schematic configuration of an electric scope. [Diagram 3] FIG. 1 is a diagram showing an example of a series of treatment steps. [Figure 4] FIG. 2 is a schematic diagram of a visual field control device. [Diagram 5] 3 is a flowchart illustrating a field of view control method according to the first embodiment of the present invention. [Figure 6] FIG. 2 is a diagram for explaining calling of a library candidate by a processor according to the first embodiment of the present invention. [Figure 7] FIG. 13 is a diagram showing an example of how library candidates are presented to a user. [Figure 8] FIG. 11 is a diagram for explaining calling of a library candidate by a processor according to a second embodiment of the present invention. [Figure 9] 10 is a flowchart illustrating a field of view control method according to a second embodiment of the present invention. [Figure 10] FIG. 11 is a diagram illustrating a filtering process performed by a processor. [Figure 11] FIG. 11 is a diagram for explaining calling of a library candidate by a processor according to a third embodiment of the present invention. [Figure 12] 10 is a flowchart illustrating a field of view control method according to a third embodiment of the present invention. [Figure 13] FIG. 13 is a diagram illustrating a ranking process performed by a processor. [Figure 14] 13A and 13B are diagrams illustrating a process of calling a library candidate by a processor in a case where filtering processing and prioritization processing are combined. [Figure 15] FIG. 13 is a diagram illustrating a filtering process and a ranking process performed by a processor. [Figure 16] FIG. 11 is a diagram illustrating how to set the presentation order of library data. [Figure 17A] FIG. 13 is a diagram for explaining an example of calculations in filtering processing and ranking processing. [Figure 17B] FIG. 13 is a diagram for explaining an example of calculations in filtering processing and ranking processing. [Figure 18] 13A and 13B are diagrams illustrating a process of calling a library candidate by a processor in a case where the prioritization process and the filtering process are combined. [Figure 19] FIG. 13 is a diagram illustrating the ranking process and filtering process performed by a processor. [Figure 20] FIG. 13 is a diagram for explaining calling of a library candidate by a processor according to a fourth embodiment of the present invention. [Figure 21] 13 is a diagram showing an example of how information on the determined current procedure scene is presented to a user. FIG. [Figure 22] 1A and 1B are diagrams illustrating the relationship between the inclination of the motorized scope and the direction of the camera. [Figure 23] 3 is a diagram illustrating the direction of the imaging optical system as viewed from the base coordinate system. FIG. [Figure 24] 1A and 1B are diagrams illustrating the relationship between the posture of an object to be observed and the direction of a camera. [Diagram 25] 3A and 3B are diagrams illustrating the orientation of an imaging optical system with respect to an observation target. [Figure 26] 1A and 1B are diagrams illustrating the relationship between the direction in which a long-sized observation target travels and the inclination of the camera about the optical axis. [Figure 27]FIG. 13 is a diagram showing an example of an endoscope field of view in a state in which the direction in which a long-sized object to be observed travels is oriented horizontally relative to the screen of the display. [Figure 28] 11A and 11B are diagrams illustrating a method for adjusting the roll angle of the motorized scope in accordance with the direction in which a long observation target travels. [Figure 29] 1A to 1C are diagrams illustrating a method for adjusting the roll angle of the motorized scope in accordance with the posture of a patient or an operating table. [Figure 30A] FIG. 13 is a diagram showing the relationship between the coordinates of an operating table or a patient and the base coordinates under normal circumstances. [Figure 30B] FIG. 13 illustrates the relationship between the coordinates of a tilted operating table or patient and the uncalibrated base coordinates. [Figure 30C] FIG. 13 shows the relationship between the coordinates of a tilted operating table or patient and the calibrated base coordinates. [Figure 31A] FIG. 13 is a diagram showing the relationship between the coordinates of an operating table or a patient and the base coordinates under normal circumstances. [Figure 31B] FIG. 13 is a diagram showing the relationship between the coordinates of a tilted operating table or patient and the orientation of a motorized scope when the library data is not calibrated. [Figure 31C] FIG. 13 is a diagram showing the relationship between the coordinates of a tilted operating table or patient and the orientation of the motorized scope when calibrating library data. [Diagram 32] FIG. 13 is a schematic configuration diagram of an oblique mirror of a visual field control system according to a fifth embodiment of the present invention. [Diagram 33] FIG. 2 shows an oblique mirror supported by a robot arm. [Diagram 34] 1 is a diagram illustrating how the direction of view of an oblique mirror changes around the longitudinal axis of the lens barrel. [Diagram 35] FIG. 2 is a diagram illustrating the field of view when looking up through an oblique mirror. [Diagram 36] 13 is a diagram illustrating the field of view when the bending mechanism of the electric scope is bent about 30 degrees in the UP direction. FIG. [Figure 37] FIG. 2 is a diagram illustrating the downward field of view of an oblique mirror. [Figure 38]13 is a diagram illustrating the field of view when the bending mechanism of the electric scope is bent about 30 degrees in the DOWN direction. FIG. [Figure 39] FIG. 2 is a diagram illustrating the rotation of the camera's visual axis around an axis line. [Diagram 40] FIG. 13 is a schematic configuration diagram of a direct-view mirror of a visual field control system according to a sixth embodiment of the present invention. [Diagram 41] FIG. 2 is a diagram illustrating the rotation of the camera's visual axis around an axis line. [Diagram 42] FIG. 13 is a diagram showing a state in which an observation target is captured at the center of the endoscope's visual field. [Diagram 43] FIG. 13 is a diagram showing a state in which the object to be observed is positioned at the center of the screen. [Diagram 44] FIG. 13 is a diagram showing a state in which an observation target is captured at the edge of the angle of view. [Diagram 45] FIG. 13 is a diagram showing how an endoscopic field of view at an angle for peering into an observation target allows hidden areas to be seen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] [First embodiment] A visual field control device, a visual field control system, a visual field control method, and a visual field control program according to a first embodiment of the present invention will be described below with reference to the drawings. 1, the visual field control system 1 according to this embodiment includes an electric scope (endoscope) 3 that acquires images of tissues in a patient's body cavity, a robot arm (position and attitude changing unit) 5 that supports the electric scope 3, a visual field control device 7 that controls the electric scope 3 and the robot arm 5, a memory 9 that stores a database of library data for endoscope adjustment, a display 11 that displays images, etc., and an input device 13 such as a mouse, keyboard, voice, and touch panel. A user can input necessary instructions to the visual field control device 7 by using the input device 13.

[0036] 2, the motorized scope 3 includes a long insertion section 15 with a camera 19 at its tip, and a bending mechanism (field of view direction changer) 17 that changes the inclination angle of the field of view of the camera 19. The motorized scope 3 also includes a bending motor that drives the bending mechanism 17 and a roll motor that rolls the insertion section 15 around its longitudinal axis (both not shown). The motorized scope 3 also has a distance measurement function that measures the distance to an observation target S, such as a blood vessel, captured within the field of view of the camera 19. A known mechanism can be used as the mechanism that realizes the distance measurement function.

[0037] Camera 19 is composed of, for example, at least one lens and an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor). Camera 19 may be a monocular camera or a stereo camera. Camera 19 is provided with a rotation mechanism (position and attitude change unit) for adjusting the angle of the imaging element, and a motor (not shown) for driving the rotation mechanism.

[0038] The robot arm 5 is, for example, an electric holder of a general-purpose six-axis articulated robot that movably holds the electric scope 3 to an arbitrary position. The robot arm 5 is provided with a motor (not shown) for each joint that operates the joint.

[0039] A plurality of library data for realizing an endoscopic field of view of the motorized scope 3 associated with each procedure scene is stored in the memory 9. The library data includes at least one relative parameter (library parameter) related to the relative position and posture between the motorized scope 3 and the observation target S.

[0040] Examples of relative parameters include scope axis roll angle information, distance information between the camera 19 of the motorized scope 3 and the observation target S, and bending amount information of the bending mechanism 17 of the motorized scope 3. The scope axis roll angle information is the roll angle from the initial state around the longitudinal axis of the insertion section 15 of the motorized scope 3. The distance information between the camera 19 of the motorized scope 3 and the observation target S is the distance between the camera 19 and the observation target S in the direction along the visual axis of the camera 19. The bending amount information of the bending mechanism 17 of the motorized scope 3 is the orientation of the camera 19 with respect to the scope axis of the motorized scope 3. Hereinafter, these pieces of information are referred to as roll angle information, distance information, and bending amount information.

[0041] In other words, in the memory 9, an endoscopic field of view suitable for each procedure scene is preset for each procedure scene, and library data including at least one of roll angle information, distance information, and curvature amount information for realizing each of these endoscopic fields of view is stored in association with each procedure scene.

[0042] The field of view control device 7 includes a recognition unit 21 that determines the current procedure scene, a judgment unit 23 that selects library data as an execution candidate for endoscopic field of view adjustment, a presentation unit 25 that presents information about the selected library data to the user, and a control unit 27 that controls the position and posture of the electric scope 3 based on the library data.

[0043] The recognition unit 21 determines the current procedure scene based on at least one of the endoscopic image acquired by the motorized scope 3, the position and posture of the motorized scope 3, and the procedure scene input information by the user. The procedure scene is classified by, for example, a specific operator's operation, an anatomical feature on the screen, and a treatment tool on the screen.

[0044] The specific operator operation is, for example, dissection, incision, unfolding, hemostasis, etc. The anatomical features on the screen are, for example, landmarks and anatomical structures, etc. The landmarks are, for example, major blood vessels and their branches, the ureter, the bile duct, and a marked tumor. The anatomical structures are, for example, lymph nodes, mesenteric depressions, the course of the intestine, nerve plexuses, and connective tissue layers. The treatment tools on the screen are, for example, the type of treatment tool, movements such as opening and closing, the gripping amount of the treatment tool, and the condition of the subject being operated on. The type of treatment tool is, for example, monopolar or bipolar, grasping forceps or dissection forceps, needle holder, etc.

[0045] For example, the recognition unit 21 determines the current procedure scene by recognizing features of treatment tools, blood vessels, organs, etc. from an endoscopic image. The recognition unit 21 also determines the current procedure scene by estimating the position, posture, and viewing direction of the motorized scope 3 from the values ​​of each motor of the motorized scope 3 and the robot arm 5. The recognition unit 21 also determines the procedure scene input by the user as the current procedure scene.

[0046] In this embodiment, the procedure flow may be programmed in advance, and library data associated with each procedure scene may be switched depending on the procedure steps, i.e., procedure scenes. In this case, for example, the recognition unit 21 may switch the current procedure scene by determining a series of procedure steps as shown in Fig. 3 using AI (Artificial Intelligence).

[0047] The procedure scene is, for example, an anatomical location being treated or an operation by the surgeon such as an incision operation. The AI ​​may estimate the procedure steps based on the anatomical location and the operation by the surgeon such as an incision. The operation by the surgeon includes, for example, an incision operation, as well as a spreading operation in which the surgeon spreads the tissue including the lesion in an appropriate direction using forceps, a dissection operation in which the tissue including the lesion is separated from the tissue to be left by cutting the tissue using an energy treatment tool such as an electric scalpel, a dissection of a specific blood vessel, a hemostasis operation against bleeding, a clip of a blood vessel, an operation of squeezing an organ, and a surgical operation such as a fluorescence observation.

[0048] The procedure steps may also be recognized by the surgeon. For example, the surgeon may specify the procedure steps by using a UI (User Interface) such as voice or button operation. In this case, for example, the recognition unit 21 may switch library data when the surgeon specifies the procedure step using the UI. Also, the recognition unit 21 may determine a specified procedure scene based on the surgeon's instruction such as "move to the next procedure step," "return to the previous procedure step," or "move to the IMA (Inferior Mesenteric Artery) resection step."

[0049] The determination unit 23 refers to the database stored in the memory 9 to extract library data associated with the current procedure scene determined by the recognition unit 21. When there are multiple library data classified into the same procedure scene, the determination unit 23 randomly extracts one of the library data excluding the library data that has already been presented.

[0050] In addition, when the user selects that he or she does not wish to perform endoscopic field of view adjustment based on the extracted library data, the judgment unit 23 extracts other library data that is classified into the same procedure scene as the extracted library data from the database in the memory 9.

[0051] The presentation unit 25 displays information about the library data extracted by the determination unit 23 on the display 11 as an execution candidate for endoscopic visual field adjustment. The information about the library data includes, for example, numerical information indicating at least one of the position, posture, and visual field direction of the motorized scope 3 when endoscopic visual field adjustment is performed based on the library data, a schematic diagram of an endoscopic image, a schematic diagram showing the positions and postures of the motorized scope 3 and the robot arm 5, and an endoscopic image acquired in a past case. The numerical information indicating the position and posture of the motorized scope 3 is, for example, a group of parameters such as the roll angle, pitch angle, and yaw angle of the motorized scope 3, the insertion amount of the insertion unit 15, and the bending angle of the bending mechanism 17 of the motorized scope 3. The presentation unit 25 also accepts an input from the user as to whether or not the user wishes to perform endoscopic visual field adjustment based on the library data displayed on the display 11.

[0052] The schematic diagram of the endoscopic image may be, for example, an image obtained by converting an endoscopic image obtained in a past case into a schematic image by image processing, or an image generated by the AI ​​(Artificial Intelligence) of the image generation system using the relative parameters of the library of execution candidates and the current position, posture and / or field of view direction of the electric scope 3 as an endoscopic image predicted after executing the library of execution candidates.

[0053] When displaying numerical information as information relating to library data, the amount of information to be processed is smaller than when displaying an image, and therefore the display speed can be increased. In addition, when a schematic diagram of an endoscopic image is displayed as information about library data, the characteristics of the field of view can be displayed more clearly than in an actual endoscopic image, allowing the user to intuitively grasp the image of the field of view after performing field of view adjustment.

[0054] In addition, if a schematic diagram showing the position and posture of the motorized scope 3 and robot arm 5 is displayed as information about the library data, the user can imagine the positional relationship between the motorized scope 3 and robot arm 5 and the user when adjusting the field of view, which leads to smooth progress of the procedure. In addition, when endoscopic images acquired in past cases are displayed as information related to library data, there is an advantage that the user can easily grasp subtle differences in field of view between the library data.

[0055] When a user selects that he or she wishes to perform endoscopic field of view adjustment based on the library data presented by the presentation unit 25, the control unit 27 outputs an instruction to change the position and attitude of the motorized scope 3 based on the library data to the motorized scope 3 and the robot arm 5. As a result, the bending mechanism 17 of the motorized scope 3, the rotation mechanism of the camera 19, and at least one of the joints of the robot arm 5 are controlled, thereby adjusting the position and attitude of the motorized scope 3.

[0056] For example, when the presented library data includes distance information as a relative parameter, the control unit 27 controls at least one of the bending mechanism 17 of the electric scope 3 and the robot arm 5 to match the actual distance between the camera 19 of the electric scope 3 and the object to be observed S, which is calculated from the image information of the object to be observed S, with the distance information.

[0057] Specifically, the control unit 27 compares the current distance measurement value to the observation target S with the distance information included in the library data, thereby calculating a trajectory for making the distance between the camera 19 of the motorized scope 3 and the observation target S match the distance information. The current distance measurement value to the observation target S is measured by the distance measurement function of the motorized scope 3.

[0058] Next, the control unit 27 determines the drive amount of each joint of the robot arm 5 required to make the distance between the camera 19 and the observation target S match the distance information by using the inverse kinematics of the robot arm 5 based on the calculated trajectory. The determined drive amount of each joint is input to each motor of the robot arm 5 as an angle command for each joint. Furthermore, the control unit 27 determines the bending angle of the bending mechanism 17 required to make the distance between the camera 19 and the observation target S match the distance information based on the calculated trajectory. The determined bending angle of the bending mechanism 17 is input to the motor of the motorized scope 3 as a motor angle command.

[0059] Each joint of the robot arm 5 operates according to its respective angle command, and the bending mechanism 17 of the motorized scope 3 bends according to the motor angle command, thereby obtaining an endoscopic field of view in which the distance between the camera 19 of the motorized scope 3 and the object to be observed S matches the distance information.

[0060] Furthermore, for example, when the presented library data includes curvature amount information as a relative parameter, the control unit 27 controls at least one of the curvature mechanism 17 of the electric scope 3 and the robot arm 5 to match the orientation of the camera 19 of the electric scope 3 with respect to the observation object S calculated from the image information of the observation object S with the curvature amount information.

[0061] Specifically, the control unit 27 compares the current bending amount of the bending mechanism 17 with bending amount information included in the library data, and calculates a trajectory for making the bending amount of the bending mechanism 17 coincide with the bending amount information.

[0062] Next, the control unit 27 determines the drive amount of each joint of the robot arm 5 required to make the bending amount of the bending mechanism 17 coincide with the bending amount information by using the inverse kinematics of the robot arm 5 based on the calculated trajectory. The determined drive amount of each joint is input to each motor of the robot arm 5 as an angle command for each joint. Furthermore, the control unit 27 determines the bending angle of the bending mechanism 17 required to make the bending amount of the bending mechanism 17 coincide with the bending amount information based on the calculated trajectory. The determined bending angle of the bending mechanism 17 is input to the motor of the motorized scope 3 as a motor angle command.

[0063] Each joint of the robot arm 5 operates according to its respective angle command, and the bending mechanism 17 of the motorized scope 3 bends according to the motor angle command, thereby obtaining an endoscopic field of view in which the orientation of the camera 19 of the motorized scope 3 relative to the object of observation S matches the bending amount information.

[0064] Furthermore, for example, when the presented library data includes roll angle information, the control unit 27 controls the robot arm 5 to match the inclination of the object S around the optical axis of the camera 19 of the motorized scope 3, which is calculated from the image information of the object S, with the roll angle information.

[0065] Specifically, the control unit 27 compares the current roll angle of the scope axis with the roll angle information to determine the amount of change in position and attitude around the pivot point required to make the roll angle of the scope axis coincide with the roll angle information. The determined amount of change in position and attitude around the pivot point includes, for example, the roll angle, pitch angle, yaw angle, and amount of advancement and retreat of the motorized scope 3.

[0066] The control unit 27 uses the inverse kinematics of the robot arm 5 based on the amount of change in position and posture about the determined pivot point to determine the drive amount of each joint required to make the roll angle of the scope axis coincide with the roll angle information. The determined drive amount of each joint is input to each motor of the robot arm 5 as an angle command for each joint. Each joint of the robot arm 5 operates according to the respective angle command, thereby obtaining an endoscopic field of view in which the tilt of the observation target S around the optical axis of the camera 19 of the motorized scope 3 coincides with the roll angle information.

[0067] When the control unit 27 controls the robot arm 5, it calculates the Euler angles (roll, pitch, yaw) based on the angles of each joint by using the forward kinematics of the robot arm 5. This calculates the current roll angle of the scope axis. The calculated current roll angle of the scope axis is stored by the control unit 27.

[0068] When the control unit 27 controls the bending mechanism 17 of the electric scope 3, the control unit 27 converts the motor angle of the bending mechanism 17 into a bending amount to calculate the current bending amount of the bending mechanism 17. The calculated current bending amount is stored by the control unit 27.

[0069] The above-mentioned visual field control device 7 is realized by, for example, a dedicated or general-purpose computer. That is, as shown in Fig. 4, the visual field control device 7 includes a processor 20 such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) constituting a recognition unit 21, a judgment unit 23, a presentation unit 25, and a control unit 27, a main storage device 31, and an auxiliary storage device 33.

[0070] The main storage device 31 is a RAM (Random Access Memory) or the like that is used as a working area for the processor 20. The auxiliary storage device 33 is a computer-readable non-transitory recording medium such as a solid state drive (SSD) or a hard disk drive (HDD). A visual field control program that causes the processor 20 to execute processing is stored in the auxiliary storage device 33. The main storage device 31 and the auxiliary storage device 33 may be connected to the visual field control device 7 via a network.

[0071] The field of view control program causes the processor 20 to execute a scene determination process for determining the current procedure scene, a candidate presentation process for presenting to the user information relating to library data associated with the determined procedure scene as an execution candidate for endoscopic field of view adjustment, a field of view adjustment process for changing at least one of the position, posture and field of view direction of the electric scope 3 based on the presented library data if the user wishes to perform endoscopic field of view adjustment based on the presented library data, and a candidate change process for presenting to the user information relating to other library data classified into the same procedure scene as the library data of the execution candidate as another execution candidate if the user does not wish to perform endoscopic field of view adjustment based on the presented library data.

[0072] The processor 20 executes processing according to the visual field control program, thereby realizing the above-mentioned functions of the recognition unit 21, the judgment unit 23, the presentation unit 25, and the control unit 27. Hereinafter, each of the above-mentioned processes by the processor 20 will be described as processes by the recognition unit 21, the judgment unit 23, the presentation unit 25, or the control unit 27.

[0073] Next, the operation of the visual field control device 7, the visual field control system 1, the visual field control method and the visual field control program configured as above will be described with reference to the flowchart of FIG. 5 and FIG. When endoscopic surgery is performed using the visual field control device 7, visual field control system 1, visual field control method, and visual field control program according to the present embodiment, first, the insertion section 15 of the motorized scope 3 is inserted into the patient's body. Then, the camera 19 of the insertion section 15 captures an image of the observation target S, such as a blood vessel, to obtain an endoscopic image of the observation target S (step S1). The obtained endoscopic image is displayed on the display 11 and input to the recognition section 21 of the visual field control device 7.

[0074] Next, the recognition unit 21 determines the current procedure scene based on at least one of the endoscopic image input from the motorized scope 3, the position and posture of the motorized scope 3, and the procedure scene input information by the user (step S2). In the example shown in Fig. 6, the recognition unit 21 recognizes features of blood vessels, organs, etc. from the endoscopic image from the motorized scope 3 to determine the current procedure scene.

[0075] Next, the determination unit 23 refers to the database stored in the memory 9 to randomly extract one piece of library data associated with the current procedure scene determined by the recognition unit 21 (step S3). The library data extracted by the determination unit 23 is also referred to as a library candidate.

[0076] Next, the presenting unit 25 displays information about the library candidates extracted by the determining unit 23 on the display 11 as execution candidates for endoscopic field of view adjustment, as shown in Fig. 7 for example (step S4). In the example shown in Fig. 7, a group of parameters such as the roll angle (deg), pitch angle (deg), and yaw angle (deg) of the motorized scope 3, the insertion amount (mm) of the insertion unit 15, and UD (UP DOWN, deg) and LR (LEFT RIGHT, deg) of the bending mechanism 17 of the motorized scope 3 are displayed as library candidates.

[0077] Next, the user selects whether or not he / she wishes to perform endoscopic visual field adjustment based on the library candidates displayed on the display 11 (step S5). In the example shown in Fig. 7, if the user wishes to perform the adjustment, he / she inputs "YES" to call the presented library candidates, and if the user does not wish to perform the adjustment, he / she inputs "NO" to not call the presented library candidates. If the user inputs "desiring to perform the adjustment, i.e., to call the library candidates," the controller 27 calls the presented library candidates.

[0078] Then, the control unit 27 outputs an instruction to change the position and posture of the motorized scope 3 based on the called library candidate to the robot arm 5 and the motorized scope 3 (step S6). As a result, the position and posture of the motorized scope 3 are adjusted, and the endoscopic field of view desired by the user is displayed on the display 11.

[0079] On the other hand, if an input is made indicating that execution is not desired, i.e., that the program is not to be called, the process returns to step 3. Then, the determination unit 23 randomly extracts one other library candidate that is classified into the same procedure scene as the library candidate displayed on the display 11 (step S3).

[0080] Next, the presenting unit 25 displays information about the library candidate newly extracted by the determining unit 23 on the display 11 as another execution candidate for endoscopic visual field adjustment (step S4). Then, the presenting unit 25 accepts an input from the user regarding the new library candidate displayed on the display 11.

[0081] Steps S3 to S5 are repeated until the user selects that he or she wishes to perform endoscopic field adjustment based on the library candidates displayed on the display 11. This allows the user to select a more preferable library candidate, making it possible to provide the endoscopic field of view desired by the user.

[0082] 6 and 7, if there is no input from the user even after a predetermined time has elapsed in step S5, the process returns to step S2. Then, the recognition unit 21 determines the procedure scene (step S2), the determination unit 23 extracts library candidates (step S3), and the presentation unit 25 presents the library candidates (step S4). This causes the library candidates displayed on the display 11 to be updated in real time. The user selects whether or not he or she wishes to perform endoscopic field adjustment for the updated library candidates (step S5).

[0083] As described above, according to the visual field control device 7, visual field control system 1, visual field control method, and visual field control program of the present embodiment, the processor 20 of the visual field control device 7 presents library data suitable for the current procedure scene to the user as candidates for endoscopic visual field adjustment. In addition, the presented library data is changed to the next candidate according to the user's selection. Then, the endoscopic visual field adjustment is performed based on the library data selected by the user.

[0084] Therefore, the user can call up a desired endoscopic field of view according to a procedure scene, without the need for manual adjustment, by simply selecting library data presented by the processor according to the user's preference or case. In addition, the user only needs to indicate through the input device 15 whether or not he or she wishes to perform endoscopic field adjustment based on the library candidates presented by the processor 20, which reduces the amount of verbal instructions from the user.

[0085] Furthermore, in conventional endoscopic surgery, when the freedom of endoscopic operation is increased in order to provide the surgeon with a better endoscopic field of view, the diversification of adjustment parameters can lead to an increase in the scopist's judgment and the surgeon's instructions to the scopist. However, by adopting the field of view control device 7, field of view control system 1, field of view control method, and field of view control program of this embodiment, it is possible to support the judgment of the surgeon and scopist, leading to a reduction in surgical time and effort.

[0086] Second Embodiment Next, a visual field control device 7, a visual field control system 1, a visual field control method, and a visual field control program according to a second embodiment of the present invention will be described. The field of view control device 7, field of view control system 1, field of view control method and field of view control program of this embodiment differ from the first embodiment in that, for example, as shown in Figure 8, the processor 20 performs a filtering process to narrow down the library data presented as execution candidates for endoscopic field of view adjustment. Hereinafter, parts having configurations in common with the visual field control device 7, visual field control system 1, visual field control method, and visual field control program according to the first embodiment will be given the same reference numerals and descriptions thereof will be omitted.

[0087] In the memory 9, a plurality of library data are stored in association with tag information representing attributes or features of each library data in addition to the procedure scene. The tag information is assumed to be, for example, information on the user, information on the facility where the visual field control system 1 is used, information on the patient, information on the type of treatment tool, information on the type of the electric scope 3, information on cases such as whether or not the case is a difficult case, information on the library data, and surgery results such as the amount of bleeding and the operation time. The information on the user includes the name, ID, sex, age, the facility to which the user belongs, and the like. The information on the facility includes the name of the facility, the area, and which procedure process is adopted, and the like. The information on the patient includes the name, ID, sex, age, and progress, and the like. The information on the library data includes the registration date and time of the library data, the execution date and time of the library data, and the execution frequency of the library data, and the like.

[0088] The user sets the narrowing down conditions based on tag information in advance or during the operation using the input device 13. The narrowing down conditions include, for example, narrowing down to only library data associated with specific tag information, narrowing down to only library data associated with tag information in a specific quantitative range of values, narrowing down to only library data associated with tag information in a range of values ​​calculated from a plurality of tag information by a specific formula, and not narrowing down based on tag information.

[0089] Examples of library data associated with specific tag information include library data linked to an individual user, such as library data previously registered by the user, library data linked to the facility to which the user belongs, such as library data registered at the facility to which the user belongs, or library data linked to all system users, such as all library data registered by users of the visual field control system 1.

[0090] Examples of library data associated with tag information in a specific quantitative numerical range include library data of past cases in which the amount of bleeding or the surgical time is within a specified range, or library data whose registration date and time are within a specified period.

[0091] The determination unit 23 extracts, from the database in the memory 9, a plurality of library candidates associated with the current procedure scene determined by the recognition unit 21, and then narrows down the library candidates to be extracted as execution candidates according to narrowing-down conditions based on tag information designated by the user. When there are a plurality of narrowed-down library candidates, any one of them may be randomly extracted.

[0092] Furthermore, when the user selects that he or she does not wish to perform endoscopic visual field adjustment based on the extracted library candidates, the determination unit 23 may randomly select any one of the narrowed-down library candidates. When all narrowed-down library candidates have already been presented, the user may be notified of this via the presentation unit 25.

[0093] Next, the operation of the visual field control device 7, the visual field control system 1, the visual field control method and the visual field control program configured as above will be described with reference to the flowchart of FIG. 9 and FIG. When performing endoscopic surgery, the user sets the filtering conditions based on tag information in advance or during surgery.

[0094] When the recognition unit 21 determines the current procedure scene (step S2), the judgment unit 23 extracts a plurality of library candidates associated with the current procedure scene from the database stored in the memory 9.

[0095] Next, the extracted multiple library candidates are narrowed down by the judgment unit 23 according to the set narrowing down conditions (step S3-1). Then, the judgment unit 23 extracts one library candidate to be presented as an execution candidate from the narrowed down multiple library candidates (step S3-2). The extracted library candidate is displayed on the display 11 by the presentation unit 25 (step S4).

[0096] In Fig. 10, an example of narrowing down library candidates to those linked to an individual user, an example of narrowing down library candidates to those linked to the user's affiliated facility, and an example of narrowing down library candidates to those linked to all system users by filtering processing are shown. One narrowing down condition may be set according to the user's preference or case, etc.

[0097] In step S5, if the user inputs that he or she does not wish to perform endoscopic visual field adjustment based on the library candidates displayed on the display 11, the process returns to step S3-2. Then, the determination unit 23 randomly extracts another one of the narrowed-down library candidates (step S3-2), and then the presentation unit 25 displays the extracted library candidate on the display 11 (step S4).

[0098] As described above, according to the visual field control device 7, visual field control system 1, visual field control method, and visual field control program of the present embodiment, the processor 20 narrows down the library data desired by the user from among multiple library data classified into the same procedure scene based on the attributes or features of the library data. Therefore, the library data desired by the user can be called up more quickly and accurately. This is particularly effective when there are multiple library data classified into the same procedure scene.

[0099] Third Embodiment Next, a visual field control device 7, a visual field control system 1, a visual field control method, and a visual field control program according to a third embodiment of the present invention will be described. The field of view control device 7, field of view control system 1, field of view control method and field of view control program of this embodiment differ from the second embodiment in that, instead of a filtering process, the processor 20 executes a ranking process that prioritizes library data presented as execution candidates for endoscopic field of view adjustment, as shown in FIG. 11, for example. Hereinafter, parts having the same configuration as the visual field control device 7, the visual field control system 1, the visual field control method, and the visual field control program according to the second embodiment will be given the same reference numerals and description thereof will be omitted.

[0100] The library data is stored in the memory 9 in a state associated with tag information that indicates attributes or characteristics of the library data in addition to the procedure scenes. The tag information is the same as that in the second embodiment.

[0101] The user sets the priority order based on tag information in advance or during the operation using the input device 13. For example, the priority order may be set based on the magnitude of a numerical value calculated from a plurality of tag information using a specific calculation formula, or based on the numerical value of specific quantitative tag information. For example, the priority order based on the numerical value of specific quantitative tag information may be set by giving a higher priority to library data that is registered or used more recently in chronological order, or by giving a higher priority to library data that is called up and used more frequently.

[0102] The judgment unit 23 extracts from the database in the memory 9 a plurality of library candidates associated with the current procedure scene determined by the recognition unit 21, and then prioritizes the library candidates to be extracted as execution candidates in accordance with the priority order based on the tag information specified by the user.

[0103] Furthermore, when the user selects that he or she does not wish to perform endoscopic visual field adjustment based on the extracted library candidates, the determination unit 23 extracts library candidates with the next highest priority after the already extracted library candidates. When all prioritized library candidates have already been presented, the user may be notified of this via the presentation unit 25.

[0104] Next, the operation of the visual field control device 7, the visual field control system 1, the visual field control method and the visual field control program configured as above will be described with reference to the flowchart of FIG. 12 and FIG. When performing endoscopic surgery, the user sets priorities based on tag information in advance or during surgery.

[0105] When the recognition unit 21 determines the current procedure scene (step S2), the judgment unit 23 extracts a plurality of library candidates associated with the current procedure scene from the database stored in the memory 9.

[0106] Next, the judgment unit 23 prioritizes the extracted library candidates according to the set priority order (step S3-1'). Then, the judgment unit 23 extracts the library candidate with the highest priority as an execution candidate (step S3-2). The extracted library candidates are displayed on the display 11 by the presentation unit 25 (step S4).

[0107] 13 shows an example in which the most recently used library candidate is preferentially extracted and an example in which the library candidate with a high frequency of call-up is preferentially extracted by the ranking process. A single priority order may be set according to the user's preference, case, etc.

[0108] In step S5, if the user inputs that he or she does not wish to perform endoscopic visual field adjustment based on the library candidates displayed on the display 11, the process returns to step S3-2. Then, after the determination unit 23 extracts a library candidate having the next highest priority after the library candidates already extracted (step S3-2), the presentation unit 25 displays the extracted library candidate on the display 11 (step S4). .

[0109] As described above, according to the visual field control device 7, visual field control system 1, visual field control method, and visual field control program of the present embodiment, the processor 20 ranks multiple library data classified into the same procedure scene according to the user's request based on the attributes or features of the library data. Therefore, the user can call up the library data he or she desires more quickly and efficiently. This is also particularly effective when there are multiple library data classified into the same procedure scene.

[0110] The filtering process of the second embodiment and the ranking process of the third embodiment can be combined as appropriate. For example, as shown in Figures 14 and 15, the judgment unit 23 may narrow down the library candidates to be presented as execution candidates from among the extracted multiple library candidates in accordance with narrowing down conditions specified by the user, and then prioritize the narrowed down library candidates for selection as execution candidates in accordance with the priority order specified by the user.

[0111] In this case, for example, as shown in FIG. 16, the presentation order of the library candidates may be set by multiplying the tag information associated with each library candidate specified by the user by a weighting constant Z set by the user according to his / her preferences.

[0112] First, the user specifies tag information to be used for filtering. In the example shown in Fig. 17A, the tag information specified by the user includes tag information indicating library candidates whose registrant is the person A, tag information indicating library candidates whose registrants are users B and C other than the person, tag information indicating library candidates whose endoscopic images do not include bleeding, and tag information indicating library candidates whose registration dates are recent.

[0113] The user also designates weighting constants to be used in setting the weighting matrix. In the example shown in Fig. 17A, the weighting constants designated by the user are 10 for the library data of the registered user A, 5 for the library data of the registered user B, and 1 for the library data of the registered user C.

[0114] 17A, the determination unit 23 extracts a plurality of library candidates associated with the current procedure scene, and then performs a filter process on the extracted plurality of library candidates based on tag information designated by the user to arrange the library candidates into a matrix. At this time, library candidates in which bleeding is present in the endoscopic image may be excluded.

[0115] 17A illustrates an example of an n-row, 3-column matrix for library candidates 1 to n, in which a component 1 or a component 0 indicates whether each registrant is the person A, user B, or user C. There may be multiple registrants, such as library candidate 4, which was registered for a surgery in which both person A and user B were present.

[0116] In addition, the determination unit 23 sets a weighting matrix based on the weighting constants set by the user. In Fig. 17A, a 3-row, 1-column matrix is ​​illustrated, in which the weighting constant of the library candidate of the registrant A is 10, the weighting constant of the library candidate of the registrant B is 5, and the weighting constant of the library candidate of the registrant C is 1.

[0117] The determination unit 23 calculates the score of each library candidate by multiplying these two matrices as shown in Fig. 17B. The determination unit 23 then sets the presentation order of the library candidates by prioritizing each library candidate in descending order of the score. For candidates with the same score, the one with the newest registration date may be given priority.

[0118] In this modified example, the ranking process is performed after the filtering process, but the order of the processes may be changed. For example, as shown in Fig. 18 and Fig. 19, the determination unit 23 may prioritize the library candidates to be presented as execution candidates according to the priority order specified by the user for the extracted library candidates, and then narrow down the library candidates to be selected as execution candidates according to the narrowing down conditions specified by the user for the prioritized library candidates. By using weighting constants set by the user to assign priorities as in this modified example, it is possible to assign priorities that reflect the user's wishes in more detail.

[0119] [Fourth embodiment] Next, a visual field control device 7, a visual field control system 1, a visual field control method, and a visual field control program according to a fourth embodiment of the present invention will be described. The field of view control device 7, field of view control system 1, field of view control method and field of view control program of this embodiment differ from the first embodiment in that, as shown in Figure 20, the user is allowed to judge whether the current procedure scene determined by the processor 20 is correct or incorrect. Hereinafter, parts having configurations in common with the visual field control device 7, visual field control system 1, visual field control method, and visual field control program according to the first embodiment will be given the same reference numerals and descriptions thereof will be omitted.

[0120] The recognition unit 21 executes a scene presentation process to present the determined current procedure scene to the user. Specifically, the recognition unit 21 displays information on the determined current procedure scene as a scene recognition result on the display 11, as shown in Fig. 21. Then, the recognition unit 21 accepts an input from the user as to whether or not the determination of the current procedure scene is correct. In the example shown in Fig. 21, the user inputs YES if the determination of the current procedure scene by the recognition unit 21 is correct, and NO if it is not correct.

[0121] When the user inputs that the information on the current procedure scene displayed on the display 11 matches the actual procedure scene, i.e., is correct, the determination unit 23 executes a candidate presentation process based on the procedure scene. On the other hand, when the user inputs that the information on the current procedure scene displayed on the display 11 does not match the actual procedure scene, i.e., is incorrect, the recognition unit 21 executes a re-determination process to re-determine the current procedure scene.

[0122] The recognition unit 21 may perform the re-determination process using unused information among the endoscopic image input from the electric scope 3, the position and posture of the electric scope 3, and the input information of the procedure scene by the user, or may perform the re-determination process by appropriately adding unused information.

[0123] According to the visual field control device 7, visual field control system 1, visual field control method, and visual field control program of the present embodiment, the accuracy of extraction of library data desired by the user can be improved by adding a user's judgment to the correctness of the procedure scene judgment derived by the processor 20. Furthermore, since the user can determine at a glance whether the procedure scene judgment is correct or not, the subsequent extraction of library candidates can be more efficiently carried out based on the accurate procedure scene judgment.

[0124] The above embodiments can be modified as follows. In the above embodiments, the relative parameter has been described by way of example using bending amount information, which is the bending amount of the bending mechanism 17 of the motorized scope 3. For example, as shown in Fig. 22, even if the bending amount of the bending mechanism 17 is the same, the degree to which the camera 19 of the motorized scope 3 looks up or down at the observation target S changes depending on the inclination of the motorized scope 3.

[0125] Therefore, instead of the curvature amount information, for example, as shown in Fig. 23, information indicating the orientation of the camera 19 of the motorized scope 3 as viewed from the base coordinate system having the center of the base of the robot arm 5 as the origin, i.e., the orientation of the visual axis of the camera 19 based on the base coordinate system, may be adopted as a relative parameter. Hereinafter, this information will be referred to as orientation information as viewed from the base coordinate system.

[0126] The orientation of the camera 19 of the motorized scope 3 seen from the base coordinates is determined by the sum of the inclination of the motorized scope 3 with respect to the base coordinates and the amount of bending of the bending mechanism 17. In a certain procedure scene, by matching the orientation of the camera 19 seen from the base coordinates to the orientation information seen from the base coordinates associated with that procedure scene, it is possible to prevent the orientation of the camera 19 from changing even if the inclination of the motorized scope 3 changes in response to changes in the position of biological tissue due to individual differences, etc. Therefore, the endoscopic field of view at the time of setting and registration can be reproduced more easily and accurately.

[0127] In addition, when the bending amount information of the bending mechanism 17 of the electric scope 3 and the direction information of the visual axis of the camera 19 are used as relative parameters, for example, as shown in FIG. 24, when the posture of the observation subject S changes due to the assistant's operation of the forceps, the viewing angle of the camera 19 with respect to the observation subject S may change relatively.

[0128] Therefore, as the relative parameter, for example, as shown in Fig. 25, information indicating the orientation of the camera 19 of the motorized scope 3 with respect to the observation target S, i.e., the orientation of the visual axis of the camera 19 as viewed from the coordinates of the observation target S, may be adopted. Hereinafter, this information is referred to as orientation information as viewed from the target. In a certain procedure scene, the orientation of the camera 19 of the motorized scope 3 as viewed from the observation target S is matched with the orientation information as viewed from the target associated with the procedure scene, so that the orientation of the camera 19 of the motorized scope 3 is determined according to the posture of the organ or biological tissue. Therefore, even if the posture or orientation of the biological tissue side changes due to the assistant's forceps operation, etc., the endoscopic field of view, i.e., the viewing angle, can be made the same as that at the time of setting and registration.

[0129] When the scope axis roll angle information is used as the relative parameter, the tilt of the camera 19 of the motorized scope 3 about the optical axis in the direction in which a long observation target S such as an intestine or a blood vessel runs may be recognized, as shown in Fig. 26. Then, as the roll angle information, a parameter may be set to orient the direction in which the intestine or blood vessel runs in the horizontal or vertical direction on the screen of the display 11.

[0130] In an actual procedure, as shown in Fig. 27, for example, the field of view is adjusted so that the direction in which the intestines or blood vessels run, i.e., the direction in which the dissection line extends, is oriented horizontally or vertically on the screen of the display 11. By orienting the direction in which the intestines or blood vessels run horizontally or vertically on the screen of the display 11 through control based on library data, an endoscopic field of view that better suits individual differences in patients can be obtained. In Fig. 27, symbol A indicates the intestines, symbol B indicates the nerve bundle, symbol C indicates the IMA (Inferior Mesenteric Artery), and symbol D indicates the dissection line.

[0131] In this case, for example, as shown in Fig. 28, the direction in which a long observation target S such as an intestine or a blood vessel travels is approximated by a curve, and then the inclinations of multiple points are calculated to calculate an average angle of those inclinations. Then, the roll angle of the motorized scope 3 and the bending angles of the bending mechanism 17 in the UD (UP DOWN) direction and LR (LEFT RIGHT) direction may be adjusted to match the average angle.

[0132] In addition, by inputting or detecting the coordinates of the patient or operating table in advance, the roll angle of the motorized scope 3 can be aligned in the Z-axis direction (usually vertically upward) with respect to the coordinates of the patient or operating table, as shown in FIG. 29, for example.

[0133] In each of the above embodiments, the library data or the base coordinates may be calibrated according to the inclination of the patient himself or the inclination of the operating table on which the patient lies. For example, as shown in Figs. 30A-30C, the inclination of the patient or the operating table may be detected, and then the detected inclination may be added to the base coordinates to correct the base coordinates. For example, as shown in Figs. 31A-31C, the inclination of the patient or the operating table may be detected, and then the detected inclination may be added to the library data to correct the library data. In Figs. 30A-30C and 31A-31C, the symbol E indicates the operating table or the patient.

[0134] Furthermore, in the present embodiment, the scope axis roll angle has been exemplified as the relative parameter, but instead of this, for example, the roll angle around the optical axis of the camera 19 of the motorized scope 3, i.e., around the visual axis, may be adopted. In this case, the control unit 27 may calculate the Euler angles (roll, pitch, yaw) based on the angles of each joint of the robot arm 5 by using, for example, the forward kinematics of the hand of the motorized scope 3 in addition to the forward kinematics of the robot arm 5.

[0135] This embodiment can also be applied to the case of performing the deployment operation in a state where an endoscopic field of view seen from above is created by retracting the motorized scope 3 to the vicinity of the insertion point of the patient, for example, to the vicinity of the trocar. In this case, as the relative parameters, for example, bending amount information in which the bending amount of the bending mechanism 17 of the motorized scope 3 is zero, scope axis advancement / retraction amount information in which the advancement / retraction amount of the motorized scope 3 is zero, and scope axis roll angle information in which the roll angle is a desired value may be adopted. The state in which the bending amount of the bending mechanism 17 is zero is a state in which the bending mechanism 17 faces the longitudinal axis direction of the insertion section 15. The state in which the advancement / retraction amount of the motorized scope 3 is zero is a state in which the motorized scope 3 is located near the insertion point of the patient.

[0136] By such control based on the library data, after the bending amount of the bending mechanism 17 is returned to zero, the advancement / retraction amount of the motorized scope 3 is set to zero, i.e., retracted to the removal limit, and the roll angle of the motorized scope 3 is adjusted. A completely bird's-eye view of the endoscope can be created by retracting the motorized scope 3 to the vicinity of the trocar in a state in which the bending mechanism 17 is extended straight along the longitudinal axis of the insertion section 15. In this case, the control unit 27 may calculate the current advancement / retraction amount of the motorized scope 3 based on the angles of each joint of the robot arm 5 by using the forward kinematics of the robot arm 5. The calculated current advancement / retraction amount of the motorized scope 3 is stored by the control unit 27.

[0137] Fifth embodiment Next, a visual field control device, a visual field control system, a visual field control method, and a visual field control program according to a fifth embodiment of the present invention will be described. The visual field control system 1 according to this embodiment differs from the first to fourth embodiments in that, instead of the electric scope 3, an oblique scope (endoscope) 41 not equipped with a bending mechanism 17 is provided as shown in FIG. Hereinafter, parts having the same configuration as the visual field control device 7, the visual field control system 1, the visual field control method and the visual field control program according to the first to fourth embodiments will be denoted by the same reference numerals and description thereof will be omitted.

[0138] The oblique scope 41 comprises a long barrel section 43 to be inserted into the patient's body cavity, and a camera (imaging optical system) 19 provided at the tip of the barrel section 43. The oblique scope 41 is arranged with the optical axis of the camera 19 inclined relative to the longitudinal axis (central axis) of the barrel section 43. The barrel section 43 has a tip surface 43a inclined relative to the longitudinal axis of the barrel section 43 and perpendicular to the optical axis of the camera 19. The oblique scope 41 also has a distance measuring function. Reference numeral 45 denotes a mounting section supported by the robot arm 5.

[0139] 33, the oblique scope 41 is supported by an electric attachment (field of view direction changer) 47 attached to the tip of the robot arm 5. The electric attachment 47 is composed of a barrel motor that rotates the barrel 43 about its longitudinal axis, a visual axis motor that rotates an image rotator (not shown) that rotates the visual axis of the camera 19 about its axis, and the like (both not shown).

[0140] By driving the motor for the telescope tube portion, the telescope tube portion 43 rotates around the longitudinal axis, and as shown in Figure 34, the direction in which the tip surface 43a, which has a certain angle with respect to the longitudinal axis of the telescope tube portion 43, faces and the optical axis direction of the camera 19, i.e., the field of view direction of the oblique mirror 41, can be changed around the longitudinal axis of the telescope tube portion 43.

[0141] Rotating the lens barrel 43 of the oblique mirror 41 around the longitudinal axis is equivalent to changing the distribution of UP, DOWN, LEFT, and RIGHT in the bending direction of the bending mechanism 17 in the electric scope 3 while keeping the bending amount constant. For example, as shown in Fig. 35, the field of view when looking up when the tip surface 43a of the oblique mirror 41 and the camera 19 are directed diagonally upward is equivalent to the field of view when the bending mechanism 17 of the electric scope 3 is bent about 30 degrees in the UP direction as shown in Fig. 36. Also, as shown in Fig. 37, the field of view when looking down when the tip surface 43a of the oblique mirror 41 and the camera 19 are directed diagonally downward is equivalent to the field of view when the bending mechanism 17 of the electric scope 3 is bent about 30 degrees in the DOWN direction as shown in Fig. 38.

[0142] In addition, by driving the visual axis motor, the visual axis of the camera 19 can be rotated around an axis, as shown in Figure 39, for example, so that the vertical direction of the endoscopic image acquired by the camera 19 can be rotated around the visual axis.

[0143] The multiple library data stored in the memory 9 include at least one relative parameter related to the relative position and posture between the camera 19 of the oblique scope 41 and the observation target S photographed by the camera 19. Examples of the relative parameter include distance information between the camera 19 and the observation target S, rotation angle information about the longitudinal axis of the tube part 43, and roll angle information about the visual axis. The roll angle information about the visual axis is an angle about the axis of the visual axis of the camera 19. Hereinafter, these pieces of information are referred to as distance information, tube part angle information, and roll angle information about the visual axis. The memory 9 stores at least one of distance information, tube part angle information, and roll angle information about the visual axis for realizing each endoscopic field of view preset for each procedure scene in association with each procedure scene.

[0144] The field of view control method changes at least one of the angle of the camera 19, and the position and attitude of the motorized scope 3, based on relative parameters included in the presented library data, and changes at least one of the angle around the longitudinal axis of the tube unit 43, and the position and attitude of the oblique mirror 41. The field of view control program causes the control unit 27 to execute its processing.

[0145] For example, when the presented library data includes distance information, the control unit 27 controls at least one of the electric attachment 47 and the robot arm 5 to match the actual distance between the camera 19 and the object to be observed S calculated from the image information of the object to be observed S with the distance information.

[0146] Specifically, the control unit 27 calculates a trajectory for making the distance between the camera 19 and the observation target S coincide with the distance information by comparing the current distance measurement value between the camera 19 and the observation target S with the distance information. Next, the control unit 27 determines the drive amount of each joint required for making the distance between the camera 19 and the observation target S coincide with the distance information by using the inverse kinematics of the robot arm 5 based on the calculated trajectory. The determined drive amount of each joint is input to each motor of the robot arm 5 as an angle command for each joint. In addition, the control unit 27 determines the rotation angle around the visual axis and the rotation angle of the lens barrel unit 43 required for making the distance between the camera 19 and the observation target S coincide with the distance information based on the calculated trajectory. The determined rotation angle around the visual axis and the rotation angle of the lens barrel unit 43 are input to the visual axis motor and the lens barrel motor of the electric attachment 47 as motor angle commands, respectively.

[0147] Each joint of the robot arm 5 operates according to its respective angle command, and each motor of the electric attachment 47 is driven according to its respective motor angle command, thereby obtaining an endoscopic field of view in which the distance between the camera 19 and the object to be observed S matches the distance information.

[0148] Furthermore, when the presented library data includes the lens barrel angle information, the control unit 27 calculates a trajectory for making the rotation angle of the lens barrel 43 coincide with the lens barrel angle information by comparing the current angle of the lens barrel 43 around the longitudinal axis with the lens barrel angle information. The control unit 27 then controls at least one of the electric attachment 47 and the robot arm 5 based on the calculated trajectory. This causes the angle of the lens barrel 43 around the longitudinal axis calculated from the image information of the observation target S to coincide with the lens barrel angle information. The control of the electric attachment 47 and the robot arm 5 by the control unit 27 is the same as in the case of the distance information, and therefore will not be described.

[0149] Furthermore, when the presented library data includes roll angle information, the control unit 27 compares the current angle around the axis of the visual axis of the camera 19 with the roll angle information around the visual axis to calculate a trajectory for making the angle around the axis of the visual axis coincide with the roll angle information around the visual axis. Then, based on the calculated trajectory, the control unit 27 controls at least one of the electric attachment 47 and the robot arm 5 to make the angle around the axis of the visual axis of the camera 19 calculated from the image information of the observation target S coincide with the roll angle information around the visual axis. The control of the electric attachment 47 and the robot arm 5 by the control unit 27 is similar to that in the case of distance information, and therefore a description thereof will be omitted.

[0150] When the control unit 27 drives the barrel motor of the electric attachment 47, it converts the motor angle of the barrel motor into an angle amount around the longitudinal axis of the barrel 43, thereby calculating the current angle amount of the barrel 43. When the control unit 27 drives the visual axis motor of the electric attachment 47, it converts the motor angle of the visual axis motor into an angle amount around the axis line of the visual axis, thereby calculating the current angle amount around the axis line of the visual axis. The calculated current angle amount of the barrel 43 and the current angle amount around the axis line of the visual axis are each stored by the control unit 27.

[0151] According to the visual field control device 7, visual field control system 1, visual field control method, and visual field control program of the present embodiment, for example, when the angle around the longitudinal axis of the barrel section 43 is changed based on the barrel section angle information, the visual field direction of the oblique mirror 41, which has a certain angle with respect to the longitudinal axis of the barrel section 43, changes around the longitudinal axis of the barrel section 43, thereby switching the visual field direction of the oblique mirror 41 to obliquely upward or downward, etc. As a result, the visual field direction of the oblique mirror 41 can be directed toward the desired observation target S simply by rotating the barrel section 43 around the longitudinal axis. Therefore, even when the oblique mirror 41 is used as the endoscope, the user can easily place the desired observation target S in the endoscopic visual field.

[0152] Sixth Embodiment Next, a visual field control device, a visual field control system, a visual field control method, and a visual field control program according to a sixth embodiment of the present invention will be described. The visual field control system 1 according to this embodiment differs from the first to fifth embodiments in that, instead of the electric scope 3 and the oblique scope 41, a direct-view scope (endoscope) 51 not having a bending mechanism 17 is provided, as shown in FIG. 40, for example. Hereinafter, parts having the same configuration as the visual field control device 7, the visual field control system 1, the visual field control method and the visual field control program according to the first to fifth embodiments will be denoted by the same reference numerals and description thereof will be omitted.

[0153] The direct endoscope 51 comprises a long barrel section 53 to be inserted into the patient's body cavity, and a camera (imaging optical system) 19 provided at the tip of the barrel section 53. The direct endoscope 51 is arranged with the longitudinal axis (central axis) of the barrel section 53 coinciding with the optical axis of the camera 19. The barrel section 53 has a tip surface 53a perpendicular to the longitudinal axis of the barrel section 53 and the optical axis of the camera 19. The direct endoscope 51 also has a distance measuring function. Reference numeral 55 denotes an attachment section supported by the robot arm 5.

[0154] The direct viewing mirror 51 is supported by an electric attachment (not shown) attached to the tip of the robot arm 5. The electric attachment is composed of a visual axis motor and the like that rotates the visual axis of the camera 19 about an axis, i.e., about the longitudinal axis of the lens barrel portion 53. As shown in Fig. 41, the direct viewing mirror 51 rotates the visual axis of the camera 19 about an axis, thereby rotating the up-down direction of the endoscopic image captured by the camera 19 about the visual axis.

[0155] The robot arm 5 functions as a viewing direction changing unit that changes the angle around a pivot axis (rotation axis) perpendicular to the longitudinal axis of the tube unit 53 in response to a change in the position at which the observation target S on the screen of the display 11 is captured. For example, when the position at which the observation target S in a biological tissue or the like is captured is changed to the upper end of the screen from a state where the observation target S is captured at the center of the screen of the display 11, the angle around the pivot point P is changed with the mounting unit 55 of the direct microscope 51 as the pivot point P, as shown in Figs. 42 and 44. As a result, the viewing direction of the direct microscope 51 is changed to a direction at which the observation target S is captured at the upper end of the screen, and as a result, the position of the observation target S on the screen of the display 11 is offset in a direction intersecting the optical axis of the camera 19, as shown in Figs. 43 and 45, for example.

[0156] As shown in Fig. 42 and Fig. 43, when the observation target S is captured in the center of the endoscopic visual field, the observation target S is located in the center of the screen, but some parts are hidden and difficult to see. In contrast, as shown in Fig. 44 and Fig. 45, when the observation target S is captured at the edge of the angle of view, the observation target S is located at the edge of the screen, and the hidden parts that were not visible become visible. In other words, by changing the viewing direction of the direct endoscope 51 to a direction that captures the observation target at the edge of the angle of view, an endoscopic visual field at an angle at which the observation target S can be viewed can be obtained. In Figs. 43 and 45, the reference numeral 59 denotes a treatment tool, and the reference numeral S denotes the observation target S.

[0157] The multiple library data stored in memory 9 include at least one relative parameter related to the relative position and posture between the camera 19 of the direct-view mirror 51 and the observation target S photographed by the camera 19. Examples of the relative parameter include distance information between the camera 19 and the observation target S, position information on the screen where the observation target S is captured, and roll angle information around the visual axis. Hereinafter, these pieces of information are referred to as distance information, target position information, and roll angle information around the visual axis.

[0158] At least one of distance information, target position information, and roll angle information about the visual axis for realizing each endoscopic field of view preset for each procedure scene is stored in the memory 9 in association with each procedure scene. For example, a desired position at which the observation target S on the screen of the display 11 is to be captured may be set as target position information by voice operation using the headset 35 or button operation using the hand switch 37.

[0159] The field of view control method changes at least one of the angle of the camera 19, and the position and attitude of the motorized scope 3, based on relative parameters included in the presented library data, and changes at least one of the angle of the direct-view mirror 51 about the pivot point P, and the position and attitude of the direct-view mirror 51. The field of view control program causes the control unit 27 to execute its processing.

[0160] For example, when the presented library data includes distance information, the control unit 27 controls at least one of the electric attachment and the robot arm 5 to match the actual distance between the camera 19 and the object to be observed S calculated from the image information of the object to be observed S with the distance information.

[0161] Specifically, the control unit 27 calculates a trajectory for making the distance between the camera 19 and the observation target S coincide with the distance information by comparing the current distance measurement value between the camera 19 and the observation target S with the distance information. Next, the control unit 27 determines the drive amount of each joint required for making the distance between the camera 19 and the observation target S coincide with the distance information by using the inverse kinematics of the robot arm 5 based on the calculated trajectory. The determined drive amount of each joint is input to each motor of the robot arm 5 as an angle command for each joint. In addition, the control unit 27 determines the rotation angle around the visual axis required for making the distance between the camera 19 and the observation target S coincide with the distance information based on the calculated trajectory. The determined rotation angle around the visual axis is input to the visual axis motor of the electric attachment as a motor angle command.

[0162] Each joint of the robot arm 5 operates according to its respective angle command, and the visual axis motor of the electric attachment rotates according to the motor angle command, thereby obtaining an endoscopic field of view in which the distance between the camera 19 and the object to be observed S matches the distance information.

[0163] Furthermore, when the presented library data includes target position information, the control unit 27 compares the current position of the observation target S on the screen of the display 11 with the target position information to calculate a trajectory for matching the position of the observation target S on the screen of the display 11 with the target position information. Then, the control unit 27 controls at least one of the electric attachment and the robot arm 5. This causes the position of the observation target S on the screen calculated from the image information of the observation target S to match the target position information. The control by the control unit 27 is similar to that in the case of distance information, and therefore a description thereof will be omitted.

[0164] Furthermore, when the presented library data includes roll angle information, the control device 7 compares the current angle around the axis of the visual axis of the camera 19 with the roll angle information around the visual axis to calculate a trajectory for making the angle around the axis of the visual axis coincide with the roll angle information around the visual axis. Then, the control unit 27 controls at least one of the electric attachment and the robot arm 5 based on the calculated trajectory to make the angle around the axis of the visual axis of the camera 19, calculated from the image information of the observation target S, coincide with the roll angle information around the visual axis. The control of the electric attachment and the robot arm 5 by the control unit 27 is similar to that in the case of distance information, and therefore will not be described.

[0165] When the control unit 27 drives the visual axis motor of the electric attachment, it converts the motor angle of the visual axis motor into an angle amount around the axis of the visual axis to calculate the current angle amount around the axis of the visual axis. The calculated current angle amount is stored by the control unit 27.

[0166] Although each embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like within the scope of the present invention are also included. For example, the present invention is not limited to the above-mentioned embodiments and each modified example, but may be applied to an embodiment in which these embodiments and modified examples are appropriately combined, and is not particularly limited.

[0167] In addition, in each of the above embodiments, the field of view control system 1 is provided with a memory 9, but instead, the memory 9 may be connected to the field of view control device 7 via a communication network such as a LAN, a WAN, or the Internet.

[0168] In each of the above-described embodiments, one library candidate is presented, extracted by the determination unit 23. Alternatively, a plurality of library candidates extracted by the determination unit 23 may be presented to the user, so that the user can select a desired one from the presented plurality of library candidates.

[0169] In addition, after recalling a library candidate for which the user has selected that he or she does not wish to perform endoscopic visual field adjustment, the library candidate may be selected. If the user's desired library candidate is not presented, the user may manually create an endoscopic view. The created endoscopic view can be registered as new library data, and can be used in similar procedure scenes in subsequent endoscopic surgeries.

[0170] Also, for example, in this embodiment, the robot arm 5 of a six-axis articulated robot has been described as an example of an electric arm, but the electric arm does not need to have six degrees of freedom, and may be a robot arm with fewer degrees of freedom. [Explanation of symbols]

[0171] 1. Vision Control System 3. Electric scope (endoscope) 5 Robot arm (position and orientation change unit, viewing direction change unit) 7. Field of view control device 17 Bending mechanism (view direction change part) 20 processors 41 Oblique endoscope (endoscope) 43 Tube 47 Electric attachment (view direction change part) 51 Direct endoscope (endoscope) 53 Tube S Observation target

Claims

1. a scene determination process for determining a current procedure scene based on at least one of an endoscopic image of an observation target acquired by an endoscope, a position and posture of the endoscope, a viewing direction of the endoscope, and input information of a procedure scene by a user; a candidate presentation process for presenting information on library data for realizing an endoscopic field of view associated with the determined procedure scene to a user as an execution candidate for adjusting the endoscopic field of view; a visual field adjustment process that outputs an instruction to change at least one of the position, the attitude, and the visual field direction of the endoscope based on the presented library data when the user desires to perform an endoscopic visual field adjustment based on the presented library data; and a candidate change process for presenting, to the user, information on other library data classified into the same procedure scene as the library data of the execution candidate as another execution candidate when the user does not wish to perform endoscopic visual field adjustment based on the presented library data, A field of view control device, wherein the library data includes at least one relative parameter relating to the relative position and attitude between the endoscope and the object to be observed.

2. the library data includes numerical information representing at least one of a position, a posture, and a viewing direction of the endoscope when an endoscope viewing field adjustment is performed based on the library data; The visual field control device according to claim 1 , wherein the processor presents the numeric information to the user as information relating to the library data during the candidate presenting process and the candidate changing process.

3. the library data includes a schematic diagram of the endoscopic image representing at least one of a position, a posture, and a viewing direction of the endoscope when an endoscopic viewing field adjustment is performed based on the library data; The visual field control device according to claim 1 , wherein the processor presents a schematic diagram of the endoscopic image as information related to the library data to the user during the candidate presentation process and the candidate change process.

4. the library data includes a schematic diagram showing a position and a posture of the endoscope, which represents at least one of a position, a posture, and a viewing direction of the endoscope when an endoscope viewing field adjustment is performed based on the library data; The visual field control device according to claim 1 , wherein the processor presents to the user a schematic diagram showing the position and orientation of the endoscope as information related to the library data during the candidate presentation process and the candidate change process.

5. the library data includes endoscopic images acquired in past cases that represent at least one of a position, a posture, and a viewing direction of the endoscope when endoscopic viewing adjustment is performed based on the library data; The visual field control device according to claim 1 , wherein the processor presents to the user, during the candidate presentation process and the candidate change process, endoscopic images acquired in the past cases as information related to the library data.

6. The library data is associated with tag information that represents attributes or features of the library data in addition to the procedure scene; The visual field control device according to claim 1 , wherein the processor executes a filtering process for narrowing down the library data presented as the execution candidates in accordance with a narrowing-down condition based on the tag information designated by a user.

7. The library data is associated with tag information that represents attributes or features of the library data in addition to the procedure scene; 2. The visual field control device according to claim 1, wherein the processor executes a ranking process for presenting the library data as the execution candidates in descending order of priority according to the tag information designated by a user.

8. 8. The visual field control device according to claim 7, wherein the processor calculates the priority order based on the tag information and a weighting constant set by a user.

9. The processor, execute a scene presentation process for presenting the determined current procedure scene to a user; If the user determines that the presented procedure scene matches an actual procedure scene, the candidate presentation process is executed based on the procedure scene; The visual field control device according to claim 1 , further comprising: a re-evaluation process for re-evaluating the current procedure scene when the user determines that the presented procedure scene does not match the actual procedure scene.

10. The endoscope; a position and attitude changing unit for changing the position and attitude of the endoscope; a visual field direction changing unit capable of changing the visual field direction of the endoscope; The visual field control device according to claim 1, A visual field control system in which the processor controls at least one of the position and orientation change unit and the visual field direction change unit based on the presented library data.

11. The visual field control system according to claim 10 , wherein the visual field direction changing unit is a bending mechanism capable of changing an angle of an imaging optical system in the endoscope.

12. the endoscope is an oblique viewer having a lens barrel portion that houses an imaging optical system, and an optical axis of the imaging optical system is inclined with respect to a central axis of the lens barrel portion; The visual field control system according to claim 10 , wherein the visual field direction changing unit changes an angle of the lens barrel unit about the central axis.

13. the endoscope is a direct-viewing endoscope having a lens barrel portion that houses an imaging optical system, the lens barrel portion being arranged such that a central axis of the lens barrel portion coincides with an optical axis of the imaging optical system; The field of view control system according to claim 10, wherein the field of view direction changing unit changes the angle around a rotation axis perpendicular to the central axis of the telescope tube unit in response to a change in the position at which the object of observation is captured on a display screen that displays the endoscopic image acquired by the imaging optical system.

14. a memory for storing the library data in association with each of the procedure scenes; The visual field control system according to claim 10 , wherein the processor calls up the library data associated with the determined current procedure scene from the memory, and then presents information about the called library data as the execution candidate.

15. determining a current procedure scene based on at least one of an endoscopic image of an observation target acquired by an endoscope, a position and posture of the endoscope, a viewing direction of the endoscope, and input information of the procedure scene by a user; presenting information about library data for realizing an endoscopic visual field associated with the determined procedure scene to a user as an execution candidate for adjusting the endoscopic visual field; If the user desires to perform endoscope visual field adjustment based on the presented library data, at least one of the position, posture, and visual field direction of the endoscope is changed based on the library data; If the user does not wish to perform endoscopic visual field adjustment based on the presented library data, information on other library data classified into the same procedure scene as the library data of the execution candidate is presented to the user as another execution candidate, A field of view control method, wherein the library data includes at least one relative parameter relating to the relative position and attitude between the endoscope and the object to be observed.

16. the library data includes at least one of numerical information representing at least one of the position, posture, and viewing direction of the endoscope when endoscopic visual field adjustment is performed based on the library data, a schematic diagram of the endoscopic image, a schematic diagram showing the position and posture of the endoscope, and an endoscopic image acquired in a past case; The field of view control method according to claim 15, wherein at least one of the numerical information, a schematic diagram of the endoscopic image, a schematic diagram showing the position and posture of the endoscope, and an endoscopic image obtained in the past case is presented to the user as the execution candidate and other execution candidates.

17. The library data is associated with tag information that represents attributes or features of the library data in addition to the procedure scene; The view control method according to claim 15, wherein the library data presented as the execution candidates is narrowed down according to a narrowing down condition based on the tag information designated by a user.

18. a scene determination process for determining a current procedure scene based on at least one of an endoscopic image of an observation target acquired by an endoscope, a position and posture of the endoscope, a viewing direction of the endoscope, and input information of a procedure scene by a user; a candidate presentation process for presenting information on library data for realizing an endoscopic field of view associated with the determined procedure scene to a user as an execution candidate for adjusting the endoscopic field of view; a visual field adjustment process for changing at least one of the position, the attitude, and the visual field direction of the endoscope based on the presented library data when the user desires to perform an endoscope visual field adjustment based on the presented library data; and if the user does not wish to perform endoscopic visual field adjustment based on the presented library data, a candidate change process is executed in the computer to present to the user, as another candidate for execution, information on other library data that is classified into the same procedure scene as the library data of the candidate for execution; A field of view control program, wherein the library data includes at least one relative parameter relating to the relative position and posture between the endoscope and the object of observation.

19. the library data includes at least one of numerical information representing at least one of the position, posture, and viewing direction of the endoscope when endoscopic visual field adjustment is performed based on the library data, a schematic diagram of the endoscopic image, a schematic diagram showing the position and posture of the endoscope, and an endoscopic image acquired in a past case; The field of view control program of claim 18, wherein during the candidate presentation process and the candidate change process, at least one of the numerical information, a schematic diagram of the endoscopic image, a schematic diagram showing the position and posture of the endoscope, and an endoscopic image obtained in the past case is presented to the user as information regarding the library data.

20. The library data is associated with tag information that represents attributes or features of the library data in addition to the procedure scene; 20. The viewing field control program according to claim 18, which causes a computer to execute a filtering process for narrowing down the library data presented as the execution candidates in accordance with a narrowing-down condition based on the tag information designated by a user.

Images