Endoscopic system, control method, and control program

Abstract

This endoscope system (1) is intended to control the field of view of an endoscope (2) on the basis of a control parameter (12), and is provided with a storage unit (9) for storing history data (14) of the control parameter (12) during a surgery in association with accompanying information (13) for the surgery, and a processor (7). The accompanying information (13) includes information relating to at least one of an operator of the surgery, an operative procedure of the surgery, and a patient. The processor (7) calculates a recommendation value for the control parameter (12) on the basis of the history data (14).

Description

Technical Field

[0004] ,

[0006] , , , , ,

[0005] , , , , ,

[0001] The present invention relates to an endoscope system, a control method, and a control program.

Background Art

[0002] Conventionally, an endoscope system that automatically causes an endoscope to follow a treatment instrument has been known (see, for example, Patent Document 1). During laparoscopic surgery, it is important to maintain an appropriate view of the endoscope. According to Patent Document 1, the view of the endoscope is controlled so that the treatment instrument is disposed at a predetermined target point in the endoscope image, thereby maintaining an appropriate view.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The view is controlled based on a plurality of control parameters including the distance from the endoscope to the treatment site or treatment instrument. Appropriate values of the control parameters vary depending on conditions such as the operator, the surgical procedure, the target organ, the facility where the surgery is performed, and the patient, and the operator's labor for adjusting the control parameters according to the conditions is required.

[0005] The present invention has been made in view of the above circumstances, and an object thereof is to provide an endoscope system, a control method, and a control program capable of obtaining appropriate control parameters for the view without requiring the labor of the operator.

Means for Solving the Problems

[0006] One aspect of the present invention is an endoscope system for controlling the field of view of an endoscope based on control parameters, comprising: a storage unit and a processor, which store historical data of the control parameters during surgery in association with ancillary information of the surgery, wherein the ancillary information includes information relating to at least one of the surgeon, surgical procedure and patient of the surgery, and the processor calculates recommended values ​​for the control parameters based on the historical data.

[0007] Another aspect of the present invention is a control method for controlling the field of view of an endoscope based on control parameters, the method comprising storing historical data of the control parameters during surgery in a memory unit in association with ancillary information of the surgery, wherein the ancillary information includes information relating to at least one of the surgeon, surgical procedure, and patient of the surgery, and calculating recommended values ​​for the control parameters based on the historical data.

[0008] Another aspect of the present invention is a control program for causing a computer to execute a control method for controlling the field of view of an endoscope based on control parameters, wherein the control method includes storing historical data of the control parameters during surgery in a storage unit in association with ancillary information of the surgery, the ancillary information including information relating to at least one of the surgeon, surgical procedure and patient of the surgery, and calculating recommended values ​​for the control parameters based on the historical data. [Effects of the Invention]

[0009] According to the present invention, it is possible to obtain appropriate control parameters for the field of view without requiring any extra effort from the operator. [Brief explanation of the drawing]

[0010] [Figure 1] This is an overall configuration diagram of the endoscope system according to the first embodiment. [Figure 2] Figure 1 is a block diagram of the endoscope system. [Figure 3A] This is a diagram illustrating the tracking parameters. [Figure 3B] A diagram for explaining the tracking parameter. [Figure 3C] A diagram for explaining the tracking parameter. [Figure 4A] A flowchart of the control method according to the first embodiment. [Figure 4B] A flowchart of the preoperative process SA in FIG. 4A. [Figure 4C] A flowchart of the intraoperative process SB in FIG. 4A. [Figure 4D] A flowchart of the postoperative process SC in FIG. 4A. [Figure 5] A diagram showing an example of the history data in the first embodiment. [Figure 6A] A flowchart of the intraoperative process SB of the control method according to the second embodiment. [Figure 6B] A flowchart of the postoperative process SC of the control method according to the second embodiment. [Figure 7] A diagram showing an example of the history data in the second embodiment. [Figure 8A] A flowchart of the intraoperative process SB of the control method according to the third embodiment. [Figure 8B] A flowchart of the postoperative process SC of the control method according to the third embodiment. [Figure 9A] A diagram for explaining an example of the method for detecting the characteristics of a scene. [Figure 9B] A diagram for explaining another example of the method for detecting the characteristics of a scene. [Figure 9C] A diagram for explaining another example of the method for detecting the characteristics of a scene. [Figure 10A] A diagram for explaining an example of the method for applying the control parameter to the treatment instrument. [Figure 10B] A diagram for explaining another example of the method for applying the control parameter to the treatment instrument. [Figure 10C] A diagram for explaining another example of the method for applying the control parameter to the treatment instrument. [Figure 11A] A flowchart of a modification example of the intraoperative process SB of the first embodiment. [Figure 11B] It is a flowchart of a modification of the intraoperative process SB of the second embodiment.

Modes for Carrying Out the Invention

[0011] (First Embodiment) An endoscope system, a control method, and a control program according to a first embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the endoscope system 1 according to the present embodiment inserts an endoscope 2 and a treatment instrument 20 into the body of a patient as the subject A, and uses the treatment instrument 20 to treat a treatment target site while observing the treatment instrument 20 with the endoscope 2. For example, it is used in laparoscopic surgery. As shown in FIGS. 1 and 2, the endoscope system 1 includes an endoscope 2, a moving device 3 that changes the position and posture of the endoscope 2, a user interface 4, a display device 5, and a control device 6.

[0012] The endoscope 2 has a camera 2a including an imaging element such as a CCD image sensor or a CMOS image sensor, and acquires an image B in the subject A by the camera 2a. The camera 2a may be a three-dimensional camera that acquires a stereo image. The image B is transmitted from the endoscope 2 to the display device 5 via the control device 6 and displayed on the display device 5. The display device 5 is an arbitrary display such as a liquid crystal display or an organic EL display.

[0013] The moving device 3 includes an electric holder 3a composed of an articulated robot arm and is controlled by the control device 6. The endoscope 2 is held at the tip of the electric holder 3a, and the position and posture of the tip of the endoscope 2 are three-dimensionally changed by the operation of the electric holder 3a. The moving device 3 does not necessarily have to be separate from the endoscope 2 and may be integrally formed as a part of the endoscope 2. Further, the moving device 3 may be another mechanism capable of changing the position and posture of the tip of the endoscope 2, such as a bending part provided at the tip of the endoscope 2.

[0014] User interface 4 is equipped with an input device and accepts input operations to the control device 6 through the input device. User interface 4 includes, for example, a voice user interface (VUI) having a headset with a microphone, and a graphical user interface (GUI) having a keyboard, mouse, and touchpad. Users such as surgeons can input to the control device 6 by operating the GUI before and after surgery, and by operating the VUI during surgery.

[0015] The control device 6 is an endoscope processor that controls the endoscope 2 and the mobile device 3. As shown in Figure 2, the control device 6 comprises at least one processor 7, a memory 8, a storage unit 9, and an input / output interface 10. The control device 6 is connected to peripheral devices 2, 3, 4, and 5 via the input / output interface 10, and transmits and receives images B and signals via the input / output interface 10.

[0016] The memory unit 9 is a computer-readable non-temporary recording medium, such as a hard disk drive, optical disc, or flash memory. The memory unit 9 stores a control program 11 that causes the processor 7 to execute a control method described later. The memory unit 9 also stores control parameters 12 used to control the mobile device 3 and supplementary information 13 related to past surgeries. In addition, the memory unit 9 stores historical data 14 of the control parameters 12 at least during surgery.

[0017] Some of the processing performed by processor 7, as described below, may be implemented by dedicated logic circuits or hardware such as FPGA (Field Programmable Gate Array), SoC (System-On-A-Chip), ASIC (Application Specific Integrated Circuit), or PLD (Programmable Logic Device).

[0018] The processor 7 controls the mobile device 3 according to a control program 11 loaded from the storage unit 9 into memory 8 such as RAM (Random Access Memory), thereby controlling the field of view F of the endoscope 2. The control of the field of view F of the endoscope 2 is a tracking control that keeps the tracking target within the field of view F by making the endoscope 2 follow the tracking target. The tracking target is any subject present in the image B during surgery, for example, a predetermined treatment instrument 20, a predetermined organ, or a predetermined tissue.

[0019] Figures 3A to 3C illustrate an example of tracking control when the tracking target is a treatment instrument 20. In this tracking control, the processor 7 detects the three-dimensional position of the tip 20a of the treatment instrument 20 using known means such as stereo measurement using stereo image B. Next, the processor 7 changes the position and orientation of the endoscope 2 by controlling the moving device 3 based on the three-dimensional position of the tip 20a and control parameters 12, only when the tip 20a is located outside a predetermined specific region C within the field of view F, thereby moving the tip 20a toward the specific region C. Through this tracking control, the position of the field of view F is controlled so that the tip 20a remains positioned within the specific region C.

[0020] The control parameter 12 includes the tracking parameter P. The tracking parameter P is a parameter relating to at least one of the positional relationship between the endoscope 2 and a specific region C, and the tracking speed of the endoscope 2 relative to the target being tracked. During surgery, the user can input a desired value for the tracking parameter P to the control device 6 by operating the user interface 4, thereby changing the tracking parameter P to the desired value.

[0021] In the examples in Figures 3A to 3C, the tracking parameter P includes the base distance d, the size of the specific region C, the position of the specific region C on image B, and the tracking speed. The base distance d is the distance between the endoscope 2 and the specific region C, for example, the distance along the optical axis from the tip 2b of the endoscope 2 to the center of the three-dimensional specific region C. The size of the specific region C includes the lateral and vertical sizes Sx,Sy on image B and the actual size Sz in the depth direction. The position of the specific region C on image B is defined, for example, by two offset amounts Δx,Δy in the lateral and vertical directions from the center of image B to the center of the specific region C. The tracking speed is the movement speed of the endoscope 2.

[0022] Next, we will describe the control methods that processor 7 performs. As shown in Figure 4A, the control method according to this embodiment includes a pre-operative process SA performed before surgery, an intra-operative process SB performed during surgery, and a post-operative process SC performed after surgery. As shown in Figure 4B, the pre-operative process SA includes step SA1 for receiving surgical information and steps SA2 to SA5 for setting control parameters 12 for the surgery to be performed.

[0023] After the control device 6 is activated, the user, such as the surgeon, inputs surgical information into the control device 6 by operating the user interface 4. The processor 7 receives the input surgical information (step SA1). Surgical information is information about a surgery to be performed and includes at least one of the following: surgeon information, surgical information, facility information, and patient information. Surgeon information is information about the surgeon performing the surgery and includes the surgeon's name, career history, specialty, etc. Surgical information includes information about the surgical area and information about the surgical procedure. Information about the surgical area includes, for example, the name of the surgical area such as the large intestine or gallbladder, and information about the surgical procedure includes, for example, the name of the surgical procedure such as sigmoid colectomy or low anterior resection. Facility information is information about the facility where the surgery will be performed and includes the facility name, size (e.g., number of beds), group relationship, etc. Patient information is information about the patient and includes the patient's name, sex, age, height and BMI, as well as information related to the difficulty of the surgery, etc. Thus, surgical information includes multiple items such as the surgeon, surgical procedure, and patient.

[0024] Next, the processor 7 sets the control parameters 12 based on the surgical information (steps SA2 to SA5). Specifically, the processor 7 searches the storage unit 9 for supplementary information 13 that matches the surgical information in at least predetermined items. In this embodiment, the predetermined items include at least the surgeon's name. As will be described later, after surgery using the control device 6, the control parameters 12 adjusted by the surgeon are stored in the storage unit 9 in association with the supplementary information 13. The supplementary information 13, like the surgical information, is information related to the surgery. When the surgeon uses the control device 6 for the first time, there is no supplementary information 13 in the storage unit 9 that matches the surgical information.

[0025] If no accompanying information 13 matching the surgical information exists in the storage unit 9 (NO in step SA2), the processor 7 sets the control parameter 12 to its initial value (step SA3). The processor 7 may determine the initial value based on the user's operation of the user interface 4. For example, the processor 7 may allow the user to select one of several initial values. If supplementary information 13 matching the surgical information exists in the memory unit 9 (YES in step SA2), the processor 7 proceeds to step SA4. Steps SA4 and SA5 will be described later.

[0026] After step SA2, processor 7 executes the intraoperative process SB. As shown in Figure 4C, the in-surgery process SB includes step SB1, which causes the endoscope 2 to follow the target based on control parameters 12; step SB2, which creates historical data 14 of the control parameters 12 during surgery in the storage unit 9; steps SB3 and SB4, which change the control parameters 12 based on the operation of the user interface 4; and step SB5, which records the changed control parameters 12.

[0027] The processor 7 starts the intraoperative process SB (step SB0) in response to a start trigger input by, for example, the operation of the user interface 4, and then starts follow-up control (step SB1). The processor 7 performs follow-up control based on the control parameters 12 set in the preoperative process SA, for example, based on the initial control parameters 12.

[0028] After the start of tracking control, the processor 7 creates history data 14 (step SB2) and accepts changes to the tracking parameters P based on the operation of the user interface 4 (step SB3). Specifically, the processor 7 creates history data 14 in the storage unit 9 to record the values ​​of the tracking parameter P during surgery (step SB2).

[0029] When the user wants to change the tracking parameter P, they input the desired value of the tracking parameter P to the control device 6 by operating the user interface 4. The processor 7 receives the input value (YES in step SB3) and changes the value of the tracking parameter P to the input value (step SB4). As a result, the tracking parameter P is adjusted to the value desired by the user, and the processor 7 performs tracking control based on the adjusted tracking parameter P. The processor 7 also records the input value of the tracking parameter P in the history data 14 (step SB5).

[0030] The history data 14 created as described above is an operation log of the user interface 4, showing the history of the surgeon's operations on the user interface 4 to change the tracking parameter P. Figure 5 is an example of an operation log 14 showing the history of changes to the base distance d, where the horizontal axis is the time since the start of surgery and the vertical axis is the base distance d. In this example, the base distance d has been changed from the initial value d0 to distances d1, d2, and d3. The base distance d(d0, d1, d2, d3) may also be recorded in association with the time since the start of surgery.

[0031] The processor 7 responds, for example, to a termination trigger entered by the user in the user interface 4, to terminate the intraoperative process SB (YES in step SB6), and then to start the postoperative process SC. As shown in Figure 4D, the post-operative process SC includes: step SC1 storing history data 14 in the storage unit 9 in association with supplementary information; step SC2 calculating a recommended value for the tracking parameter P from the history data 14; step SC3 asking the user whether or not to change the tracking parameter P to the recommended value; step SC4 changing the tracking parameter P based on the user's response; and step SC5 storing the changed tracking parameter P in the storage unit 9 in association with supplementary information 13.

[0032] The processor 7 stores the history data 14 created in the intraoperative process SB in the storage unit 9 in association with the accompanying information 13 (step SC1). The accompanying information 13 is at least a part of the surgical information entered in step SA1 and includes at least surgeon information. The accompanying information 13 may be set by the user.

[0033] Next, the processor 7 calculates the average value of the tracking parameter P modified by the user as the recommended value (step SC2). In the case of the base distance d in Figure 5, the recommended value is the average value of distances d1, d2, and d3. The recommended value may be a value other than the average, for example, the median, mode, or longest usage time of the tracking parameter P in the operation log 14, or a time-weighted average value Σyi·ti / Σti with usage time as the weight. yi is the value of the tracking parameter P, and ti is the length of time the tracking parameter yi is used.

[0034] Next, the processor 7 presents a query to the user, for example, by displaying an optical indication on the display device 5 asking "whether or not to change the tracking parameters" or by outputting an audio message asking "whether or not to change the tracking parameters" (step SC3). The processor 7 may also query the user using any other means. The user inputs an answer to the query to the control device 6 by operating the user interface 4.

[0035] If the response to make the change is received (YES in step SC3), the processor 7 changes the tracking parameter P to the recommended value (step SC4) and stores the changed tracking parameter P in the memory unit 9 in association with the accompanying information 13 (step SC5). As a result, the tracking parameter P adjusted by the operator is stored in the memory unit 9 in association with the operator's information. On the other hand, if the response is that no changes will be made (NO in step SC3), the processor 7 will not perform steps SC4 and SC5 and will terminate the post-operative process SC.

[0036] As a result of performing the above steps SC1 to SC5, after surgery using the control device 6, the associated history data 14 and the adjusted follow-up parameters P are stored in the storage unit 9, along with the accompanying information 13. When the control device 6 is used after the adjusted tracking parameters P have been stored in the memory unit 9, the processor 7 sets the tracking parameters P based on the surgical information in the pre-surgical process SA (steps SA2, SA4, SA5).

[0037] Specifically, if there is supplementary information 13 that matches the surgical information, at least in the name of the surgeon (YES in step SA2), the processor 7 asks the user whether or not to apply the tracking parameter P associated with the supplementary information 13 that matches the surgical information to the surgery to be performed (step SA4), and thereby suggests to the surgeon that they use the tracking parameter P that they themselves have adjusted in the past.

[0038] The user inputs the answer to the inquiry into the control device 6 by operating the user interface 4. If the response to apply is received (YES in step SA4), the processor 7 sets the associated follow-up parameter P as the follow-up parameter P for the upcoming surgery (step SA5). This allows the surgeon to use the follow-up parameter P that they have adjusted in previous surgeries for the upcoming surgery. On the other hand, if the response is that it does not apply (NO in step SA4), the processor 7 sets the follow parameter P to its initial value (step SA3).

[0039] In endoscopic surgery, maintaining an appropriate field of view with the endoscope is crucial, and the appropriate control parameters for this vary depending on factors such as the surgeon, surgical procedure, target organ, surgical facility, and patient. Therefore, adjusting the control parameters requires effort from the surgeon. For example, if the scopist operates the endoscope manually, it requires effort to train the scopist by having them experience multiple surgeries beforehand, and effort for the surgeon to instruct the scopist on how to operate the endoscope during the surgery. If the field of view of the endoscope is controlled automatically, it is necessary to pre-set the surgeon's preferred control parameters or for the surgeon to adjust the control parameters during the surgery by operating the user interface.

[0040] According to this embodiment, during surgery, history data 14 showing the history of changes made by the surgeon to the tracking parameter P is stored in the storage unit 9 in association with the accompanying information 13, and a recommended value for the tracking parameter P is calculated from the history data 14. This makes it possible to obtain an appropriate tracking parameter P without requiring any extra effort from the surgeon.

[0041] Furthermore, according to this embodiment, after surgery, the follow-up parameter P, which has been changed to a recommended value, is stored in the memory unit 9 in association with the accompanying information 13, and the use of the changed follow-up parameter P is suggested to the surgeon in subsequent surgeries. Therefore, in subsequent surgeries, the surgeon can use the follow-up parameter P that they have adjusted from the start of the surgery, saving the trouble of adjusting the follow-up parameter P.

[0042] Furthermore, according to this embodiment, the processor 7 asks the user whether or not to change the tracking parameter P to a recommended value, and the user decides whether or not to change it. This allows the tracking parameter P to be changed only when the user desires it. The recommended value is not necessarily the most desirable value, and it may be preferable not to change it to the recommended value. Similarly, the processor 7 asks the user whether or not to apply the modified tracking parameter P to the upcoming surgery, and the user decides whether or not to apply it. This ensures that the adjusted tracking parameter P is applied to the upcoming surgery only when the user desires it.

[0043] (Second Embodiment) Next, an endoscope system, control method, and control program according to a second embodiment of the present invention will be described. This embodiment differs from the first embodiment in the control parameters used to calculate the recommended value. In this embodiment, the configurations that differ from the first embodiment will be described, and components common to both embodiments will be denoted by the same reference numerals and their descriptions will be omitted. The endoscope system 1 according to this embodiment comprises an endoscope 2, a mobile device 3, a user interface 4, a display device 5, and a control device 6, similar to the first embodiment.

[0044] The control parameter 12 includes the tracking parameter P, as well as the unit change amount Δ of the tracking parameter P. The unit change amount Δ is the amount of change in the tracking parameter P per operation of the user interface 4. For each operation of the user interface 4, the processor 7 changes the tracking parameter P by the unit change amount Δ. Therefore, for example, in the case of a base distance d, the user can change the base distance d by distance Δ each time they operate the user interface 4.

[0045] Next, we will describe the control methods that processor 7 performs. The control method according to this embodiment includes a pre-operative process SA, an intra-operative process SB, and a post-operative process SC, similar to the first embodiment. Figures 6A and 6B show the intraoperative process SB and the postoperative process SC in this embodiment, respectively.

[0046] As shown in Figure 6A, the intraoperative process SB includes step SB1, step SB12 for creating historical data 14 of the intraoperative tracking parameter P in the storage unit 9, steps SB3 and SB4 for changing the tracking parameter P based on the operation of the user interface 4, and step SB15 for recording the amount of adjustment of the tracking parameter P.

[0047] The processor 7 creates history data 14 (step SB12), accepts changes to the tracking parameter P (step SB3), and changes the tracking parameter P based on the operation of the user interface 4 (step SB4). Specifically, the processor 7 creates history data 14 in the storage unit 9 to record the amount of adjustment of the tracking parameter P (step SB12).

[0048] If the initial unit change amount Δ is too small, the user operates the user interface 4 multiple times in succession during a single adjustment of the tracking parameter P. Successive operations mean, for example, that the time interval between the i-th operation and the next i+1-th operation is less than or equal to a predetermined threshold. The adjustment amount is the change in the tracking parameter P during a single adjustment, for example, the change in the tracking parameter P within a predetermined time. The processor 7 may calculate the adjustment amount as the product of the unit change amount Δ and the number of consecutive operations of the user interface 4. After step SB4, the processor 7 calculates the adjustment amount of the tracking parameter P and records the adjustment amount in the history data 14 (step SB15).

[0049] Figure 7 shows an example of historical data 14. In Figure 7, in the first (t1) and third (t3) adjustments, the base distance d is changed by twice the unit change amount Δ through two consecutive operations. In the second adjustment (t2), the base distance d is changed by four times the unit change amount Δ through four consecutive operations.

[0050] As shown in Figure 6B, the post-operative process SC includes step SC1, step SC12 which calculates a recommended value for the unit change amount Δ from the history data 14, step SC13 which asks the user whether or not to change the unit change amount Δ to the recommended value, step SC14 which changes the unit change amount Δ based on the user's response, and step SC15 which stores the unit change amount Δ in the storage unit 9 in association with the accompanying information 13.

[0051] The processor 7 calculates the mode of the adjustment amount in the historical data 14 as the recommended value (step SC12). For example, in the example in Figure 7, since two consecutive operations are performed most frequently, the mode is twice the unit change amount Δ. The recommended value may be a value other than the mode, for example, the average of the maximum and minimum values ​​of the adjustment amount, the median, the greatest common denominator, the approximate value that minimizes the error from each adjustment amount, or the time-weighted average value Σyi·ti / Σti, where yi is the adjustment amount and ti is the length of time the adjustment amount yi is used.

[0052] Next, the processor 7, similar to step SC3, presents a query to the user, for example, by displaying an optical display on the display device 5 or by outputting sound (step SC13). The user inputs their answer to the query to the control device 6 using the user interface 4.

[0053] If the response to make the change is received (YES in step SC13), the processor 7 changes the unit change amount Δ to the recommended value (step SC14) and stores the changed unit change amount Δ in the storage unit 9 in association with the accompanying information 13 (step SC15). On the other hand, if the response is that no changes will be made (NO in step SC13), the processor 7 will not perform steps SC14 and SC15 and will terminate the post-operative process SC.

[0054] As a result of performing the above steps SC1, SC12 to SC15, after surgery using the control device 6, the adjusted unit change amount Δ associated with the accompanying information 13 is newly stored in the memory unit 9. When the control device 6 is used after the adjusted unit change amount Δ has been stored in the memory unit 9, the processor 7 sets the unit change amount Δ based on the surgical information in the pre-surgical process SA (steps SA2, SA4, SA5).

[0055] Specifically, if there is supplementary information 13 that matches the surgical information, at least in the name of the surgeon (YES in step SA2), the processor 7 asks the user whether or not to apply the unit change amount Δ associated with the supplementary information 13 that matches the surgical information to the surgery to be performed (step SA4), and thereby suggests to the surgeon that they use the unit change amount Δ that they themselves have adjusted in the past.

[0056] If the response is to apply (YES in step SA4), the processor 7 sets the associated unit change amount Δ as the unit change amount Δ for the upcoming surgery (step SA5). This allows the surgeon to use the unit change amount Δ that they adjusted in a previous surgery for the upcoming surgery. On the other hand, if the response is that it does not apply (NO in step SA4), the processor 7 sets the unit change amount Δ to its initial value (step SA3).

[0057] Thus, according to this embodiment, during surgery, history data 14 showing the history of adjustments made by the surgeon to the tracking parameter P is stored in the storage unit 9 in association with the accompanying information 13, and a recommended value for the unit change amount Δ is calculated from the history data 14. This makes it possible to obtain an appropriate unit change amount Δ without requiring any extra effort from the surgeon.

[0058] Furthermore, according to this embodiment, after surgery, the unit change amount Δ, which has been changed to a recommended value, is stored in the memory unit 9 in association with the accompanying information 13, and the use of the changed unit change amount Δ is suggested to the surgeon in subsequent surgeries. Therefore, in subsequent surgeries, the surgeon can use the unit change amount Δ that they have adjusted from the start of the surgery, saving the trouble of adjusting the unit change amount Δ.

[0059] Furthermore, according to this embodiment, the processor 7 asks the user whether or not to change the unit change amount Δ to a recommended value, and the user decides whether or not to change it. This allows the unit change amount Δ to be changed only when the user desires it. Similarly, the processor 7 asks the user whether or not to apply the modified unit change amount Δ to the upcoming surgery, and the user decides whether or not to apply it. This ensures that the adjusted unit change amount Δ is applied to the upcoming surgery only when the user desires it.

[0060] (Third embodiment) Next, an endoscope system, control method, and control program according to a third embodiment of the present invention will be described. This embodiment differs from the first and second embodiments in the historical data used to calculate the recommended values. In this embodiment, the configurations that differ from the first embodiment will be described, and the same reference numerals will be used for components common to the first embodiment, and their descriptions will be omitted. The endoscope system according to this embodiment comprises an endoscope 2, a mobile device 3, a user interface 4, a display device 5, and a control device 6, similar to the first embodiment.

[0061] In this embodiment, the history data 14 is either a group of images B in a time series during surgery, or a history of control parameters 12 detected from each image B in the group, or information correlated therewith. Next, we will describe the control methods that processor 7 performs. The control method according to this embodiment includes a pre-operative process SA, an intra-operative process SB, and a post-operative process SC, similar to the first embodiment. Figures 8A and 8B show the intraoperative process SB and the postoperative process SC in this embodiment, respectively.

[0062] As shown in Figure 8A, the intraoperative process SB includes step SB1, step SB22 for creating history data 14 in the storage unit 9, and steps SB3 and SB4. The processor 7 stores the image B input from the endoscope 2 to the control device 6 in the storage unit 9, thereby creating a group of images consisting of time-series images B during the surgical process SB as history data 14 in the storage unit 9 (step SB22).

[0063] As shown in Figure 8B, the post-operative process SC includes step SC1, step SC22 for calculating recommended values ​​for control parameters 12 from historical data 14, step SC23 for asking the user whether to change the control parameters 12 to the recommended values, step SC24 for changing the control parameters 12 based on the user's response, and step SC25 for storing the changed control parameters 12 in the storage unit 9 in association with the accompanying information 13.

[0064] The processor 7 detects the control parameter 12 or information correlated with the control parameter 12, and the scene features from each image B as history data 14 stored in the memory unit 9, and calculates a recommended value for the control parameter 12 for each scene (step SC22). The control parameter 12 is, for example, at least one of the tracking parameter P and the unit change amount Δ.

[0065] For example, the processor 7 detects the distance d' from the endoscope 2 to a predetermined target (see Figure 3B) as information correlated with the control parameter 12. The predetermined target is, for example, a tracking object in image B, and the distance d' changes as the base distance d changes. The processor 7 calculates the distance d' by analyzing each image B using known means. For example, the processor 7 may calculate the distance d' by stereo measurement using stereo images B. Alternatively, the processor 7 may calculate the distance d' from each image B using a learning model. The learning model is created by machine learning (e.g., deep learning) of various images B and distances d' and is pre-stored in the memory unit 9.

[0066] As shown in Figures 9A to 9C, surgery generally involves multiple scenes. Figure 9A shows a scene in which a membrane is cut using an electrosurgical unit 20A, and Figure 9B shows a scene in which blood vessel D is dissected using forceps 20B. An example of scene features is the body tissue (blood vessel or organ, etc.) and the instruments 20A, 20B in image B. The processor 7 detects at least one type of body tissue and instrument 20A, 20B in each image B, for example, using known image recognition techniques.

[0067] Figure 9C shows a scene in which a blood vessel is treated using forceps 20B. Another example of a scene feature is that the distance α between a given body tissue and the tip 20a of the instrument 20B is below a predetermined threshold. For example, the processor 7 recognizes the region E of the blood vessel D and the region G of the instrument 20B in image B, calculates the distance α between the centroid of region E and the tip 20a, and detects that the distance α is below a threshold as a feature.

[0068] Next, the processor 7 calculates a recommended value for each detected scene feature, which is the average value of the control parameter 12 or the information correlated with it, such as the average value of distance d'. This calculates a recommended value for the control parameter 12 that is appropriate for each scene. The recommended value may be a value other than the mean, for example, the median, mode, or longest usage time of the control parameter 12 or the information correlated with it, or a time-weighted mean value Σyi·ti / Σti, where yi is the value of the control parameter 12 and ti is the length of usage time for the control parameter yi.

[0069] Next, the processor 7, similar to step SC3, presents a query to the user, for example, by displaying an optical display on the display device 5 or by outputting sound (step SC23). The user inputs their response to the query to the control device 6 using the user interface 4.

[0070] If the response to make the change is received (YES in step SC23), the processor 7 changes the control parameter 12 to the recommended value (step SC24) and stores the changed control parameter 12 in the memory unit 9 in association with the accompanying information 13 and the scene characteristics (step SC25). On the other hand, if the response is that no changes will be made (NO in step SC23), the processor 7 will not perform steps SC24 and SC25 and will terminate the post-operative process SC.

[0071] When the control device 6 is used after the control parameters 12 have been stored in the memory unit 9 in association with the scene characteristics, the processor 7 sets the control parameters 12 in the pre-operative process SA based on surgical information and user responses (steps SA2 to SA5). When the adjusted control parameters 12 are set as the control parameters 12 for the surgery to be performed (step SA5), in the surgical process SB, the processor 7 detects scene features from image B and controls the mobile device 3 based on the control parameters 12 corresponding to the features. As a result, during the surgery, the control parameters 12, such as tracking parameters P including base distance d, are automatically changed to appropriate values ​​according to the scene.

[0072] Thus, according to this embodiment, the image B taken during surgery is stored in the storage unit 9 as history data 14, associated with the accompanying information 13, and recommended values ​​for the control parameters 12 in each scene are calculated from the history data 14. This makes it possible to obtain appropriate control parameters 12 without requiring any extra effort from the surgeon.

[0073] Furthermore, according to this embodiment, after surgery, the control parameters 12, which have been changed to recommended values, are stored in the memory unit 9 in association with the accompanying information 13, and the surgeon is then asked whether or not to use the changed control parameters 12 in subsequent surgeries. Therefore, in subsequent surgeries, the surgeon can use the control parameters 12 that they have adjusted from the start of the surgery, thus saving the trouble of adjusting the control parameters 12.

[0074] Furthermore, according to this embodiment, the processor 7 asks the user whether or not to change the control parameter 12 to a recommended value, and the user decides whether or not to change it. This allows the control parameter 12 to be changed only when the user desires it. Similarly, the processor 7 asks the user whether or not to apply the modified control parameters 12 to the upcoming surgery, and the user decides whether or not to apply them. This ensures that the adjusted control parameters 12 are applied to the upcoming surgery only when the user desires it.

[0075] In this embodiment, the processor 7 stores the image B during the surgical process SB as history data 14. However, instead, the control parameter 12 or information correlated with the control parameter 12, and the scene features may be stored as history data 14. In other words, during the intraoperative process SB, the processor 7 detects control parameters 12 or information correlated therewith, and scene characteristics from each image B input from the endoscope 2 to the control device 6, and stores the detected control parameters 12 or information correlated therewith, and scene characteristics as history data 14 in the storage unit 9. During the postoperative process SC, the processor 7 calculates recommended values ​​for the control parameters 12 of each scene from the history data 14.

[0076] In each of the above embodiments, the processor 7 changes the control parameter 12 to a recommended value based on the user's response. Alternatively, the control parameter 12 may be automatically changed to a recommended value without asking the user whether or not to change the control parameter 12. In this case, steps SC3, SC13, and SC23 are omitted in each embodiment. Similarly, in each embodiment, the processor 7 may apply control parameters 12 associated with the surgical information and corresponding supplementary information to the surgery to be performed without querying the user. In this case, step SA4 is omitted in each embodiment.

[0077] In each of the above embodiments, the processor 7 calculates the recommended value in the post-surgical process SC, but instead, the recommended value may be calculated in the pre-surgical process SA. In this case, the processor 7 executes steps SC2-SC4, SC12-SC14, or SC22-SC24 in the pre-operative process SA of the surgery to be performed later. That is, in the pre-operative process SA, the processor 7 reads from the storage unit 9 the associated history data 14 that matches the surgical information 13, calculates recommended values ​​for the control parameters 12 from the history data 14, and asks the user whether or not to change the control parameters 12 to the recommended values. Even in this way, the surgeon can use the appropriate control parameters 12 that they have adjusted in previous surgeries from the start of the operation, thus saving the trouble of adjusting the control parameters 12.

[0078] There are multiple types of surgical instruments 20 that can be used in surgery, and therefore, there are multiple types of surgical instruments 20 that can be followed. The preferred control parameters 12 may differ for each type of surgical instrument 20. Therefore, in each embodiment, the processor 7 may select the surgical instrument 20 to which the adjusted control parameters 12 are applied based on the type of surgical instrument 20.

[0079] Figures 10A to 10C illustrate the instruments 20 to which the adjusted tracking parameters P are applied. In Figure 10A, the adjusted tracking parameters P (e.g., base distance) are applied to each type of instrument 20. In Figure 10B, the adjusted tracking parameters P are applied to each group of instruments 20. Each group includes, for example, instruments 20 of the same type or with similar uses. In Figure 10C, the adjusted tracking parameters P are applied to all types of instruments 20. Furthermore, the adjusted tracking parameter P may be applied to targets other than the treatment device 20. For example, the target may be an organ such as a blood vessel.

[0080] In each of the above embodiments, the storage unit 9 may store one or more historical data 14 associated with each item. For example, the storage unit 9 may store one or more historical data 14 of surgeries performed by surgeon A, associated with surgeon A, and one or more historical data 14 of surgeries performed by surgeon B, associated with surgeon B. In this case, the processor 7 may calculate recommended values ​​for the control parameters 12 from at least one historical data 14 for each item. For example, the processor 7 may calculate recommended values ​​for operator A from one or more historical data 14 associated with operator A, and calculate recommended values ​​for operator B from one or more historical data 14 associated with operator B. This configuration allows for the automatic creation of appropriate tracking parameters P for each condition, such as the surgeon, surgical procedure, target organ, facility, and patient.

[0081] In each of the embodiments described above, the processor 7 searches for ancillary information that matches the surgical information in at least the name of the surgeon. Alternatively, the processor 7 may search for ancillary information that matches the surgical information in at least one arbitrary item. For example, in step SA2, the processor 7 may search for ancillary information that matches the surgical information in the surgical procedure or the patient. This configuration allows the surgeon to be presented with control parameters 12 that correspond to various surgical conditions. In this case, as described above, the processor 7 may propose control parameters 12 calculated from one or more historical data 14 associated with each item.

[0082] In each of the above embodiments, the processor 7 calculates a recommended value from one historical data 14, but instead, it may calculate a recommended value from multiple historical data 14. An example of multiple historical data 14 is historical data 14 of multiple surgeries performed by the same surgeon. The multiple surgeries may be multiple recent surgeries performed by the same surgeon, or they may be surgeries of the same surgical procedure performed by the same surgeon at the same facility. Another example of multiple historical data 14 is the historical data 14 of the first predetermined number of surgeries performed using the control device 6. The predetermined number may be one or multiple surgeries. In this case, the initial control parameters 12 are used for the predetermined number of surgeries, and recommended values ​​for the control parameters 12 are calculated after the predetermined number of surgeries have been performed.

[0083] In each of the above embodiments, the processor 7 proposes to the user control parameters 12 associated with the same surgeon as the surgeon in the surgical information. Alternatively, the processor 7 may propose to the user control parameters 12 associated with a different surgeon than the surgeon in the surgical information.

[0084] For example, the processor 7 may propose control parameters 12 calculated based on the history data 14 of past surgeries performed by the second surgeon as control parameters 12 for the surgery to be performed by the first surgeon. The first surgeon is the surgeon of the surgical information (i.e., the surgeon who will perform the surgery), and the second surgeon is a different surgeon from the first surgeon. With this configuration, the first surgeon can use the appropriate control parameters 12 adjusted by the second surgeon in the surgery without any extra effort.

[0085] The second surgeon may be a surgeon with more surgical experience than the first surgeon. For example, processor 7 may determine the second surgeon based on the surgeon's experience (number of surgeries performed). The second surgeon is a more experienced surgeon with a higher number of surgeries performed than the first surgeon. With this configuration, the less experienced first surgeon can use appropriate control parameters 12 that the experienced second surgeon has adjusted in past surgeries, and the first surgeon can reproduce the appropriate field of view F during the second surgeon's surgery during their own surgery.

[0086] Processor 7 may determine the second operator based on information other than the operator's background. In one example, processor 7 determines the second surgeon based on their affiliated institution. For instance, the first surgeon might be from a branch hospital, and the second surgeon from the main hospital. In other examples, processor 7 determines the second surgeon based on the surgical area and specialty. For example, in a gallbladder surgery, the first surgeon is a colorectal surgeon who is assigned to perform a laparoscopic cholecystectomy, while the second surgeon is a general surgeon. In another example, processor 7 determines a second surgeon based on patient information. For example, the second surgeon is one who has performed more surgeries on petite female patients than the first surgeon. The processor 7 may calculate recommended values ​​for the control parameters 12 from historical data 14 of past surgeries performed by multiple second surgeons. For example, the processor 7 may calculate recommended values ​​from historical data 14 associated with multiple second surgeons who have performed a large number of surgeries on small female patients.

[0087] In each of the above embodiments, the processor 7 may, in addition to or instead of after surgery, ask the user whether to change the control parameters 12 during the current surgery when a change in the control parameters 12 based on the operation of the user interface 4 satisfies predetermined conditions. This configuration allows the control parameter 12 to be changed to an appropriate value during surgery, reducing the effort required for the surgeon to adjust the control parameter 12 during the operation.

[0088] Figure 11A shows a modified control method of the first embodiment. The predetermined condition is that the number of changes to the control parameter 12 based on the operation of the user interface 4 is greater than or equal to a predetermined number N. Specifically, when the number of times the tracking parameter P has been changed reaches a predetermined number N (YES in step SB7), the processor 7 calculates a recommended value for the tracking parameter P from the history data 14 of the current surgery recorded up to that point (step SB8), and asks the user whether or not to change the tracking parameter P to the recommended value (step SB9). If the user responds that they want to change it (YES in step S9), the processor 7 changes the tracking parameter P to the recommended value (step SB10).

[0089] The specified conditions may be any other conditions. Another example of a specified condition is that the tracking parameter P is changed N or more times within a specified time from the start of surgery. In this case, the increased number of changes in the latter half of the surgery can prevent unnecessary suggestions for changing the recommended value. Another example of a given condition is when, after changing the tracking parameter P, that tracking parameter P continues to be used continuously for a predetermined period of time or longer. Operators tend to continue using an appropriate tracking parameter P for extended periods. Therefore, an appropriate tracking parameter P can be determined for an operator based on the fact that the same tracking parameter P is used for extended periods.

[0090] Figure 11B shows a modified control method of the second embodiment. The predetermined condition is that the number of adjustments of a predetermined adjustment amount of the tracking parameter P is greater than or equal to a predetermined number N. In other words, when the number of adjustments to the tracking parameter P of a predetermined adjustment amount reaches a predetermined number N (YES in step SB17), the processor 7 calculates a recommended value for the unit change amount Δ from the history data 14 of the current surgery recorded up to that point (step SB18), and asks the user whether or not to change the unit change amount Δ to the recommended value (step SB19). If the user answers yes to the change (YES in step SB19), the processor 7 changes the unit change amount Δ to the recommended value (step SB20).

[0091] The specified conditions may be any other conditions. Another example of a specified condition is that, regardless of the adjustment amount, the user interface 4 is operated continuously N or more times within a specified time from the start of the surgery. Another example of a predetermined condition is that a predetermined adjustment amount and an adjustment that is an integer multiple of the predetermined adjustment amount are performed a predetermined number of times N or more.

[0092] Although embodiments and modifications of the present invention have been described above, the present invention is not limited thereto and can be modified as appropriate without departing from the spirit of the invention. [Explanation of symbols]

[0093] 1 Endoscopy System 2 Endoscope 3. Mobile device 4. User Interface 6. Control device 7 Processors 9 Memory section 11 Control Program 12 Control Parameters 13. Additional Information 14. Historical Data 20, 20A, 20B Treatment tools Image B C Specific area

Claims

1. An endoscope system that controls the field of view of an endoscope based on control parameters, A storage unit stores historical data of the control parameters during surgery in association with ancillary information of the surgery, wherein the ancillary information includes information about at least one of the surgeon, surgical procedure, and patient. Equipped with a processor, The processor calculates recommended values ​​for the control parameters based on the historical data, An endoscope system in which the historical data is an operation log of the user interface for changing the control parameters.

2. The control of the field of view of the endoscope is a tracking control that keeps the tracking target within a specific area of ​​the field of view by making the endoscope follow the tracking target, The endoscope system according to claim 1, wherein the control parameter is a tracking parameter relating to at least one of the positional relationship between the endoscope and the specific region and the tracking speed of the endoscope with respect to the tracking target.

3. The control of the field of view of the endoscope is a tracking control that keeps the tracking target within a specific area of ​​the field of view by making the endoscope follow the tracking target, The endoscope system according to claim 1, wherein the control parameter is the amount by which a user interface operation changes a tracking parameter relating to at least one of the positional relationship between the endoscope and the specific region and the tracking speed of the endoscope with respect to the tracking target.

4. The endoscope system according to claim 3, wherein the amount of change is the unit change amount of the tracking parameter per operation of the user interface.

5. The endoscope system according to claim 3, wherein the history data is a history of the amount of adjustment of the tracking parameter by operation of the user interface, and the amount of adjustment is the amount of change of the tracking parameter within a predetermined time.

6. The storage unit stores one or more of the history data associated with each item included in the accompanying information. The endoscope system according to claim 1, wherein the processor calculates the recommended value from at least one historical data for each item.

7. The aforementioned processor, We receive surgical information regarding at least one of the surgeon, surgical procedure, and patient for the upcoming surgery. The endoscopic system according to claim 1, wherein recommended values ​​calculated from the historical data associated with the accompanying information that matches the surgical information in at least one of the surgeon, surgical procedure, and patient are set as the control parameters for the surgery to be performed.

8. The aforementioned processor, The user is asked whether or not to change the control parameter to the calculated recommended value. The endoscopic system according to claim 1, wherein if a response indicating that a change will be made is received, the control parameter is changed to the calculated recommended value.

9. The endoscopic system according to claim 1, wherein the processor automatically changes the control parameters to the calculated recommended values.

10. The endoscopic system according to claim 1, wherein if the control parameter is changed by the user more than a predetermined number of times during the current surgery, the processor changes the control parameter to the recommended value calculated based on the history data of the current surgery.

11. The endoscopic system according to claim 10, wherein the processor changes the control parameter to the recommended value if the control parameter is changed a predetermined number of times or more within a predetermined time from the start of the surgery.

12. The endoscope system according to claim 1, wherein the recommended value is the mean, median, mode, longest usage time, or time-weighted mean of the control parameter in the historical data.

13. The endoscope further comprises a moving device for changing its position and orientation. The endoscope system according to claim 1, wherein the processor controls the moving device based on the control parameters.

14. An endoscope that acquires images, The endoscope system according to claim 13, further comprising a user interface that accepts user input for changing the control parameters.

15. A control method for controlling the field of view of an endoscope based on control parameters, The historical data of the control parameters during surgery is stored in a memory unit in association with the associated information of the surgery, and the associated information includes information about at least one of the surgeon, the surgical procedure, and the patient, and This includes calculating recommended values ​​for the control parameters based on the historical data, A control method wherein the historical data is an operation log of the user interface for changing the control parameters.

16. A control program for causing a computer to execute a control method that controls the field of view of an endoscope based on control parameters, The control method described above is The historical data of the control parameters during surgery is stored in a memory unit in association with the associated information of the surgery, and the associated information includes information about at least one of the surgeon, the surgical procedure, and the patient, and This includes calculating recommended values ​​for the control parameters based on the historical data, A control program in which the historical data is an operation log of the user interface for changing the control parameters.

Images