Endoscope system, control method, and control program

Abstract

This endoscope system (1) is intended to control the field of view of an endoscope (2) on the basis of a control parameter (12), and is provided with a storage unit (9) for storing history data (14) of the control parameter (12) during a surgery in association with accompanying information (13) for the surgery, and a processor (7). The accompanying information (13) includes information relating to at least one of an operator of the surgery, an operative procedure of the surgery, and a patient. The processor (7) calculates a recommendation value for the control parameter (12) on the basis of the history data (14).

Description

Endoscope system, control method, and control program

[0001] The present invention relates to an endoscope system, a control method, and a control program.

[0002] Conventionally, an endoscopic system that automatically tracks an endoscope with a treatment tool is known (see, for example, Patent Document 1). During laparoscopic surgery, it is important to maintain an appropriate field of view of the endoscope. According to Patent Document 1, the field of view of the endoscope is controlled so that the treatment tool is positioned at a predetermined target point within the endoscopic image, thereby maintaining an appropriate field of view.

[0003] International Publication No. 2022 / 054882

[0004] The field of view is controlled based on multiple control parameters, including the distance from the endoscope to the treatment site or treatment tool, etc. The appropriate values ​​of the control parameters vary depending on the conditions of the surgeon, surgical procedure, target organ, surgical facility, patient, etc., and the surgeon must take the time to adjust the control parameters according to the conditions.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an endoscopic system, a control method, and a control program that can obtain appropriate control parameters for the field of view without requiring the surgeon to do anything.

[0006] One aspect of the present invention is an endoscopic system that controls the field of view of an endoscope based on control parameters, and includes a memory unit that stores historical data of the control parameters during surgery in association with accompanying information about the surgery, the accompanying information including information about at least one of the surgeon, surgical procedure, and patient of the surgery, and a processor, and the processor calculates recommended values ​​of the control parameters based on the historical data.

[0007] Another aspect of the present invention is a control method for controlling the field of view of an endoscope based on control parameters, the control method including: storing historical data of the control parameters during surgery in a memory unit in association with accompanying information about the surgery, the accompanying information including information about at least one of the surgeon, surgical procedure, and patient of the surgery; and calculating a recommended value of the control parameter based on the historical data.

[0008] Another aspect of the present invention is a control program for causing a computer to execute a control method for controlling the field of view of an endoscope based on control parameters, the control method including storing historical data of the control parameters during surgery in a memory unit in association with accompanying information about the surgery, the accompanying information including information about at least one of the surgeon, surgical procedure, and patient of the surgery, and calculating a recommended value of the control parameter based on the historical data.

[0009] According to the present invention, it is possible to obtain appropriate control parameters for the field of view without requiring the surgeon to take any time and effort.

[0010] 4A is a diagram illustrating an overall configuration of an endoscopic system according to a first embodiment. FIG. 4B is a block diagram of the endoscopic system of FIG. 1. FIG. 4C is a diagram illustrating tracking parameters. FIG. 4D is a diagram illustrating tracking parameters. FIG. 4E is a flowchart of a control method according to the first embodiment. FIG. 4F is a flowchart of a pre-operative step SA of FIG. 4A. FIG. 4G is a flowchart of an intra-operative step SB of FIG. 4A. FIG. 4H is a flowchart of a post-operative step SC of FIG. 4A. FIG. 4I is a diagram illustrating an example of history data in the first embodiment. FIG. 4I is a flowchart of an intra-operative step SB of a control method according to a second embodiment. FIG. 4I is a flowchart of a post-operative step SC of a control method according to the second embodiment. FIG. 4I is a flowchart of an intra-operative step SB of a control method according to a third embodiment. FIG. 4I is a flowchart of a post-operative step SC of a control method according to the third embodiment. FIG. 4I is a diagram illustrating an example of a method of detecting scene features. FIG. 4I is a diagram illustrating another example of a method of detecting scene features. FIG. 4I is a diagram illustrating another example of a method of detecting scene features. FIG. 4I is a diagram illustrating an example of a method of applying control parameters to a treatment tool. FIG. 4I is a diagram illustrating another example of a method of applying control parameters to a treatment tool. FIG. 4I is a flowchart of a modified example of the intra-operative step SB of the first embodiment. 10 is a flowchart of a modified example of the intraoperative process SB of the second embodiment.

[0011] First Embodiment An endoscopic system, a control method, and a control program according to a first embodiment of the present invention will be described with reference to the drawings. As shown in Fig. 1, an endoscopic system 1 according to this embodiment is used in surgery in which an endoscope 2 and a treatment tool 20 are inserted into the body of a patient, who is an object A, and a treatment target site is treated with the treatment tool 20 while the treatment tool 20 is being observed through the endoscope 2, for example, in laparoscopic surgery. As shown in Figs. 1 and 2, the endoscopic system 1 includes an endoscope 2, a moving device 3 that changes the position and attitude of the endoscope 2, a user interface 4, a display device 5, and a control device 6.

[0012] The endoscope 2 has a camera 2a including an imaging element such as a CCD image sensor or a CMOS image sensor, and acquires an image B of the inside of the subject A by the camera 2a. The camera 2a may be a three-dimensional camera that acquires stereo images. The image B is transmitted from the endoscope 2 via the control device 6 to the display device 5, where it is displayed. The display device 5 is any display such as a liquid crystal display or an organic EL display.

[0013] The moving device 3 includes an electric holder 3a consisting of a multi-joint robot arm, and is controlled by a control device 6. The endoscope 2 is held at the tip of the electric holder 3a, and the position and orientation of the tip of the endoscope 2 are changed three-dimensionally by the operation of the electric holder 3a. The moving device 3 does not necessarily have to be separate from the endoscope 2, but may be formed integrally as part of the endoscope 2. The moving device 3 may also be another mechanism capable of changing the position and orientation of the tip of the endoscope 2, such as a bending portion provided at the tip of the endoscope 2.

[0014] The user interface 4 includes an input device and receives input operations for the control device 6 through the input device. The user interface 4 includes, for example, a voice user interface (VUI) having a headset with a microphone, and a graphical user interface (GUI) having a keyboard, mouse, touchpad, etc. A user such as a surgeon can input to the control device 6 by operating the GUI before and after surgery, and by operating the VUI during surgery, for example.

[0015] The control device 6 is an endoscope processor that controls the endoscope 2 and the moving device 3. As shown in Fig. 2, the control device 6 includes at least one processor 7, a memory 8, a storage unit 9, and an input / output interface 10. The control device 6 is connected to the peripheral devices 2, 3, 4, and 5 via the input / output interface 10, and transmits and receives an image B, signals, and the like via the input / output interface 10.

[0016] The storage unit 9 is a computer-readable non-transitory recording medium, such as a hard disk drive, an optical disk, or a flash memory. The storage unit 9 stores a control program 11 that causes the processor 7 to execute a control method, which will be described later. The storage unit 9 also stores control parameters 12 used to control the moving device 3 and associated information 13 related to surgeries performed in the past. The storage unit 9 also stores history data 14 of the control parameters 12 at least during the surgeries.

[0017] Some of the processing described below performed by processor 7 may be realized by dedicated logic circuits or hardware such as an FPGA (Field Programmable Gate Array), a SoC (System-On-A-Chip), an ASIC (Application Specific Integrated Circuit), or a PLD (Programmable Logic Device).

[0018] The processor 7 controls the moving device 3 in accordance with a control program 11 loaded from the storage unit 9 into a memory 8 such as a RAM (Random Access Memory), thereby controlling the field of view F of the endoscope 2. The control of the field of view F of the endoscope 2 is tracking control that causes the endoscope 2 to track the target, thereby keeping the target captured within the field of view F. The target is any subject present in the image B during surgery, such as a specified treatment tool 20, a specified organ, or a specified tissue.

[0019] 3A to 3C illustrate an example of tracking control when the target to be tracked is a treatment tool 20. In this tracking control, the processor 7 detects the three-dimensional position of the tip 20a of the treatment tool 20 using a known means such as stereo measurement using stereo images B. Next, only when the tip 20a is located outside a predetermined specific area C within the field of view F, the processor 7 controls the movement device 3 based on the three-dimensional position of the tip 20a and the control parameters 12 to change the position and attitude of the endoscope 2, thereby moving the tip 20a toward the specific area C. By such tracking control, the position of the field of view F is controlled so that the tip 20a remains positioned within the specific area C.

[0020] The control parameters 12 include a following parameter P. The following parameter P is a parameter related to at least one of the positional relationship between the endoscope 2 and the specific region C and the following speed of the endoscope 2 relative to the target to be tracked. During surgery, the user operates the user interface 4 to input a desired value of the following parameter P to the control device 6, thereby changing the following parameter P to the desired value.

[0021] 3A to 3C, the tracking parameter P includes a base distance d, the size of the specific region C, the position of the specific region C on the image B, and a tracking speed. The base distance d is the distance between the endoscope 2 and the specific region C, and is, for example, the distance along the optical axis from the tip 2b of the endoscope 2 to the center of the three-dimensional specific region C. The size of the specific region C includes horizontal and vertical dimensions Sx and Sy on the image B, and an actual size Sz in the depth direction. The position of the specific region C on the image B is defined, for example, by two offset amounts Δx and Δy in the horizontal and vertical directions from the center of the image B to the center of the specific region C. The tracking speed is the movement speed of the endoscope 2.

[0022] Next, a control method executed by the processor 7 will be described. As shown in Fig. 4A, the control method according to this embodiment includes a pre-operative step SA executed before surgery, an intra-operative step SB executed during surgery, and a post-operative step SC executed after surgery. As shown in Fig. 4B, the pre-operative step SA includes step SA1 for receiving surgery information and steps SA2 to SA5 for setting control parameters 12 for the upcoming surgery.

[0023] After starting the control device 6, a user, such as a surgeon, inputs surgical information into the control device 6 by operating the user interface 4. The processor 7 receives the input surgical information (step SA1). The surgical information is information about the upcoming surgery and includes at least one of surgeon information, surgery information, facility information, and patient information. The surgeon information is information about the surgeon performing the surgery, including the surgeon's name, background, specialty, etc. The surgical information includes information about the surgical field and information about the surgical procedure. The information about the surgical field includes, for example, the name of the surgical field, such as the colon or gallbladder, and the information about the surgical procedure includes, for example, the name of the surgical procedure, such as sigmoid resection or low anterior resection. The facility information is information about the facility where the surgery will be performed, including the facility name, size (e.g., number of beds), group affiliation, etc. The patient information is information about the patient, including the patient's name, gender, age, height, BMI, and information related to the difficulty of the surgery. Thus, the surgical information includes multiple items, such as the surgeon, surgical procedure, and patient.

[0024] Next, the processor 7 sets the control parameters 12 based on the surgery information (steps SA2 to SA5). Specifically, the processor 7 searches the storage unit 9 for accompanying information 13 that matches the surgery information in at least certain items. In this embodiment, the certain items include at least the name of the surgeon. As will be described later, after a surgery is performed using the control device 6, the control parameters 12 adjusted by the surgeon are stored in the storage unit 9 in association with the accompanying information 13. The accompanying information 13 is information related to the surgery, similar to the surgery information. When the surgeon uses the control device 6 for the first time, the accompanying information 13 that matches the surgery information is not present in the storage unit 9.

[0025] If the storage unit 9 does not contain any associated information 13 matching the surgical information (NO in step SA2), the processor 7 sets the control parameters 12 to their initial values ​​(step SA3). The processor 7 may determine the initial values ​​based on the user's operation of the user interface 4. For example, the processor 7 may allow the user to select one of multiple initial values. If the storage unit 9 contains any associated information 13 matching the surgical information (YES in step SA2), the processor 7 proceeds to step SA4. Steps SA4 and SA5 will be described later.

[0026] After step SA2, the processor 7 executes the in-operative process SB. As shown in Fig. 4C, the in-operative process SB includes step SB1 of making the endoscope 2 follow the tracking target based on the control parameters 12, step SB2 of creating history data 14 of the control parameters 12 during the surgery in the storage unit 9, steps SB3 and SB4 of changing the control parameters 12 based on the operation of the user interface 4, and step SB5 of recording the changed control parameters 12.

[0027] The processor 7 starts the in-operative step SB (step SB0) in response to a start trigger input by operating the user interface 4, and then starts the tracking control (step SB1). The processor 7 performs the tracking control based on the control parameters 12 set in the pre-operative step SA, for example, based on the initial setting control parameters 12.

[0028] After starting the tracking control, the processor 7 creates history data 14 (step SB2) and accepts (step SB3) a change in the tracking parameter P based on an operation of the user interface 4. Specifically, the processor 7 creates history data 14 in the storage unit 9 for recording the value of the tracking parameter P during surgery (step SB2).

[0029] When the user wishes to change the tracking parameter P, the user operates the user interface 4 to input a desired value of the tracking parameter P to the control device 6. The processor 7 receives the input value (YES in step SB3) and changes the value of the tracking parameter P to the input value (step SB4). As a result, the tracking parameter P is adjusted to the value desired by the user, and the processor 7 performs tracking control based on the adjusted tracking parameter P. The processor 7 also records the input value of the tracking parameter P in the history data 14 (step SB5).

[0030] The history data 14 created as described above is an operation log of the user interface 4, showing a history of operations of the user interface 4 by the surgeon to change the following parameter P. Fig. 5 is an example of the operation log 14 showing a history of changes to the base distance d, where the horizontal axis represents time from the start of surgery and the vertical axis represents the base distance d. In this example, the base distance d is changed from an initial value d0 to distances d1, d2, and d3. The base distances d (d0, d1, d2, and d3) may be recorded in association with the time from the start of surgery.

[0031] The processor 7 ends the intra-operative step SB in response to an end trigger input by the user to the user interface 4, for example (YES in step SB6), and then starts the post-operative step SC. As shown in Fig. 4D , the post-operative step SC includes step SC1 of storing the history data 14 in the storage unit 9 in association with the accompanying information, step SC2 of calculating a recommended value of the following parameter P from the history data 14, step SC3 of asking the user whether or not to change the following parameter P to the recommended value, step SC4 of changing the following parameter P based on the user's response, and step SC5 of storing the changed following parameter P in the storage unit 9 in association with the accompanying information 13.

[0032] The processor 7 stores the history data 14 created in the intraoperative process SB in the storage unit 9 in association with the accompanying information 13 (step SC1). The accompanying information 13 is at least a part of the surgery information input in step SA1 and includes at least surgeon information. The accompanying information 13 may be set by the user.

[0033] Next, the processor 7 calculates the average value of the following parameter P changed by the user as a recommended value (step SC2). In the case of the base distance d in FIG. 5, the recommended value is the average value of the distances d1, d2, and d3. The recommended value may be a value other than the average, such as the median, mode, or longest value of the following parameter P in the operation log 14, or a time-weighted average value Σyi·ti / Σti, where yi is the value of the following parameter P, and ti is the length of time the following parameter yi has been used.

[0034] Next, the processor 7 presents a query to the user by, for example, displaying an optical indication of "whether to change the tracking parameters" on the display device 5 or outputting a voice message of "whether to change the tracking parameters" (step SC3). The processor 7 may also query the user using any other means. The user inputs a response to the query into the control device 6 by operating the user interface 4.

[0035] If an answer to the effect of the change is received (YES in step SC3), the processor 7 changes the tracking parameter P to the recommended value (step SC4), and stores the changed tracking parameter P in the storage unit 9 in association with the accompanying information 13 (step SC5). As a result, the tracking parameter P adjusted by the surgeon is stored in the storage unit 9 in association with the surgeon's information. On the other hand, if an answer to the effect of the change is received (NO in step SC3), the processor 7 does not perform steps SC4 and SC5, and ends the post-operative process SC.

[0036] By executing the above steps SC1 to SC5, after surgery using the control device 6, the history data 14 associated with the accompanying information 13 and the adjusted following parameters P are stored in the storage unit 9. When the control device 6 is used after the adjusted following parameters P are stored in the storage unit 9, the processor 7 sets the following parameters P based on the surgery information in the pre-surgery step SA (steps SA2, SA4, SA5).

[0037] Specifically, if there is accompanying information 13 that matches the surgical information at least in terms of the surgeon's name (YES in step SA2), the processor 7 asks the user whether or not to apply the tracking parameters P associated with the accompanying information 13 that matches the surgical information to the upcoming surgery (step SA4), thereby suggesting to the surgeon that they use the tracking parameters P that the surgeon himself or herself has adjusted in the past.

[0038] The user operates the user interface 4 to input a response to the inquiry into the control device 6. If the response to apply is received (YES in step SA4), the processor 7 sets the associated tracking parameter P as the tracking parameter P for the upcoming surgery (step SA5). This allows the surgeon to use the tracking parameter P that he or she adjusted in a previous surgery for the upcoming surgery. On the other hand, if the response to not apply is received (NO in step SA4), the processor 7 sets the tracking parameter P to its initial value (step SA3).

[0039] In endoscopic surgery, it is important to maintain an appropriate field of view of the endoscope, and the appropriate control parameters for this vary depending on factors such as the surgeon, surgical procedure, target organ, surgical facility, and patient. Therefore, adjusting the control parameters requires the surgeon's time and effort. For example, if a scopist manually operates the endoscope, the surgeon must undergo training by performing multiple surgeries in advance, and the surgeon must instruct the scopist on how to operate the endoscope during surgery. Automatically controlling the field of view of the endoscope requires the surgeon to set the control parameters according to their preferences in advance, or to adjust the control parameters by operating a user interface during surgery.

[0040] According to this embodiment, history data 14 indicating the history of changes made to the following parameter P by the surgeon during surgery is stored in the storage unit 9 in association with the accompanying information 13, and a recommended value for the following parameter P is calculated from the history data 14. This makes it possible to obtain an appropriate following parameter P without requiring the surgeon to do any effort.

[0041] Furthermore, according to this embodiment, after surgery, the following parameters P changed to recommended values ​​are stored in the storage unit 9 in association with the accompanying information 13, and the surgeon is suggested to use the changed following parameters P in subsequent surgeries. Therefore, in subsequent surgeries, the surgeon can use the following parameters P that he or she adjusted himself or herself from the start of the surgery, thereby saving the effort of adjusting the following parameters P.

[0042] Furthermore, according to this embodiment, the processor 7 inquires of the user as to whether or not to change the following parameter P to a recommended value, and the user determines whether or not to change it. This allows the following parameter P to be changed only when the user wishes. The recommended value is not necessarily a preferable value, and it may be preferable not to change it to the recommended value. Similarly, the processor 7 inquires of the user as to whether or not to apply the changed following parameter P to the upcoming surgery, and the user determines whether or not to apply it. This allows the adjusted following parameter P to be applied to the upcoming surgery only when the user wishes.

[0043] Second Embodiment Next, an endoscopic system, a control method, and a control program according to a second embodiment of the present invention will be described. This embodiment differs from the first embodiment in the control parameters used to calculate recommended values. In this embodiment, configurations different from the first embodiment will be described, and configurations common to the first embodiment will be assigned the same reference numerals and will not be described again. Similar to the first embodiment, the endoscopic system 1 according to this embodiment includes an endoscope 2, a moving device 3, a user interface 4, a display device 5, and a control device 6.

[0044] The control parameters 12 include a unit change amount Δ of the following parameter P in addition to the following parameter P. The unit change amount Δ is the amount of change in the following parameter P per operation of the user interface 4. The processor 7 changes the following parameter P by the unit change amount Δ per operation of the user interface 4. Therefore, for example, in the case of a base distance d, the user can change the base distance d by the distance Δ each time he or she operates the user interface 4.

[0045] Next, a control method executed by the processor 7 will be described. Similar to the first embodiment, the control method according to this embodiment includes a pre-operative step SA, an intra-operative step SB, and a post-operative step SC. Figures 6A and 6B show the intra-operative step SB and the post-operative step SC, respectively, in this embodiment.

[0046] As shown in FIG. 6A, the intraoperative process SB includes step SB1, step SB12 for creating historical data 14 of the tracking parameter P during surgery in the memory unit 9, steps SB3 and SB4 for changing the tracking parameter P based on the operation of the user interface 4, and step SB15 for recording the adjustment amount of the tracking parameter P.

[0047] Processor 7 creates history data 14 (step SB12), accepts a change in the following parameter P (step SB3), and changes (step SB4) the following parameter P based on the operation of user interface 4. Specifically, processor 7 creates history data 14 in storage unit 9 for recording the amount of adjustment of the following parameter P (step SB12).

[0048] If the initial setting unit change amount Δ is too small, the user operates the user interface 4 multiple times in succession during one adjustment of the following parameter P. "Continuous operations" refers to, for example, the time interval between the i-th operation and the next (i+1)-th operation being equal to or less than a predetermined threshold. The adjustment amount is the amount of change in the following parameter P during one adjustment, for example, the amount of change in the following parameter P within a predetermined time period. The processor 7 may calculate the adjustment amount as the product of the unit change amount Δ and the number of successive operations of the user interface 4. After step SB4, the processor 7 calculates the adjustment amount of the following parameter P and records the adjustment amount in the history data 14 (step SB15).

[0049] 7 shows an example of the history data 14. In FIG. 7, in the first (t1) and third (t3) adjustments, the base distance d is changed by two consecutive operations by twice the unit change amount Δ. In the second adjustment (t2), the base distance d is changed by four consecutive operations by four times the unit change amount Δ.

[0050] As shown in FIG. 6B, the post-operative process SC includes step SC1, step SC12 of calculating a recommended value for the unit change amount Δ from the history data 14, step SC13 of asking the user whether or not to change the unit change amount Δ to the recommended value, step SC14 of changing the unit change amount Δ based on the user's response, and step SC15 of storing the unit change amount Δ in the memory unit 9 in association with the accompanying information 13.

[0051] The processor 7 calculates the most frequent adjustment amount in the history data 14 as the recommended value (step SC12). For example, in the example of FIG. 7, two consecutive operations are performed most frequently, so the most frequent value is twice the unit change amount Δ. The recommended value may be a value other than the most frequent value, such as the average, median, or greatest common denominator of the maximum and minimum adjustment amounts, an approximate value that minimizes the error from each adjustment amount, or a time-weighted average value Σyi·ti / Σti, where yi is the adjustment amount and ti is the length of time that the adjustment amount yi is used.

[0052] Next, similar to step SC3, the processor 7 presents a query to the user by, for example, displaying an optical display on the display device 5 or outputting a voice message (step SC13). The user inputs a response to the query into the control device 6 using the user interface 4.

[0053] If an answer to the effect of the change is received (YES in step SC13), the processor 7 changes the unit change amount Δ to the recommended value (step SC14), and stores the changed unit change amount Δ in the storage unit 9 in association with the accompanying information 13 (step SC15). On the other hand, if an answer to the effect of the change is received (NO in step SC13), the processor 7 does not perform steps SC14 and SC15, and ends the post-surgery process SC.

[0054] By executing the above steps SC1 and SC12 to SC15, after surgery using the control device 6, the adjusted unit change amount Δ associated with the accompanying information 13 is newly stored in the storage unit 9. When the control device 6 is used after the adjusted unit change amount Δ is stored in the storage unit 9, the processor 7 sets the unit change amount Δ based on the surgery information in the pre-surgery step SA (steps SA2, SA4, SA5).

[0055] Specifically, if there is accompanying information 13 that matches the surgical information at least in terms of the surgeon's name (YES in step SA2), the processor 7 asks the user whether or not to apply the unit change amount Δ associated with the accompanying information 13 that matches the surgical information to the upcoming surgery (step SA4), thereby suggesting to the surgeon that they use the unit change amount Δ that the surgeon themselves adjusted in the past.

[0056] If the answer to "apply" is received (YES in step SA4), processor 7 sets the associated unit change amount Δ as the unit change amount Δ for the upcoming surgery (step SA5). This allows the surgeon to use the unit change amount Δ that he or she adjusted in the previous surgery for the upcoming surgery. On the other hand, if the answer to "do not apply" is received (NO in step SA4), processor 7 sets the unit change amount Δ to the initial value (step SA3).

[0057] As described above, according to this embodiment, the history data 14 indicating the history of the adjustment amount of the following parameter P made by the surgeon during surgery is stored in the storage unit 9 in association with the accompanying information 13, and a recommended value for the unit change amount Δ is calculated from the history data 14. This makes it possible to obtain an appropriate unit change amount Δ without requiring the surgeon to do any effort.

[0058] Furthermore, according to this embodiment, after surgery, the unit change amount Δ changed to a recommended value is stored in the storage unit 9 in association with the accompanying information 13, and the use of the changed unit change amount Δ in subsequent surgeries is suggested to the surgeon. Therefore, in subsequent surgeries, the surgeon can use the unit change amount Δ that he or she adjusted himself or herself from the start of the surgery, thereby saving the effort of adjusting the unit change amount Δ.

[0059] Furthermore, according to this embodiment, the processor 7 inquires of the user as to whether or not to change the unit change amount Δ to the recommended value, and the user determines whether or not to change it. This allows the unit change amount Δ to be changed only when the user wishes. Similarly, the processor 7 inquires of the user as to whether or not to apply the changed unit change amount Δ to the upcoming surgery, and the user determines whether or not to apply it. This allows the adjusted unit change amount Δ to be applied to the upcoming surgery only when the user wishes.

[0060] Third Embodiment Next, an endoscopic system, a control method, and a control program according to a third embodiment of the present invention will be described. This embodiment differs from the first and second embodiments in the history data used to calculate the recommended value. In this embodiment, configurations different from the first embodiment will be described, and configurations common to the first embodiment will be assigned the same reference numerals and will not be described again. The endoscopic system according to this embodiment includes an endoscope 2, a moving device 3, a user interface 4, a display device 5, and a control device 6, similar to the first embodiment.

[0061] In this embodiment, the history data 14 is a group of images consisting of time-series images B during surgery, or a history of control parameters 12 detected from each image B of the group of images or information correlated therewith. Next, a control method executed by the processor 7 will be described. Similar to the first embodiment, the control method according to this embodiment includes a pre-operative step SA, an intra-operative step SB, and a post-operative step SC. Figures 8A and 8B show the intra-operative step SB and the post-operative step SC, respectively, in this embodiment.

[0062] 8A, the in-operative process SB includes step SB1, step SB22 for creating history data 14 in the storage unit 9, and steps SB3 and SB4. The processor 7 stores images B input from the endoscope 2 to the control device 6 in the storage unit 9, thereby creating an image group consisting of images B in time series during the in-operative process SB as history data 14 in the storage unit 9 (step SB22).

[0063] As shown in FIG. 8B, the post-operative process SC includes step SC1, step SC22 of calculating recommended values ​​of control parameters 12 from historical data 14, step SC23 of asking the user whether or not to change the control parameters 12 to the recommended values, step SC24 of changing the control parameters 12 based on the user's response, and step SC25 of storing the changed control parameters 12 in the memory unit 9 in association with accompanying information 13.

[0064] The processor 7 detects the control parameters 12 or information correlated with the control parameters 12, and the characteristics of the scenes from each image B as the history data 14 stored in the storage unit 9, and calculates recommended values ​​of the control parameters 12 for each scene (step SC22). The control parameters 12 are, for example, at least one of the tracking parameter P and the unit change amount Δ.

[0065] For example, the processor 7 detects a distance d' (see FIG. 3B ) from the endoscope 2 to a predetermined target as information correlated with the control parameter 12. The predetermined target is, for example, an object to be tracked in the image B, and the distance d' changes as the base distance d changes. The processor 7 calculates the distance d' by analyzing each image B using known means. For example, the processor 7 may calculate the distance d' by stereo measurement using the stereo image B. Alternatively, the processor 7 may calculate the distance d' from each image B using a learning model. The learning model is created by machine learning (e.g., deep learning) of various images B and distances d' and is stored in advance in the storage unit 9.

[0066] As shown in Figures 9A to 9C, surgery generally includes multiple scenes. Figure 9A shows a scene in which a membrane is incised using an electric scalpel 20A, and Figure 9B shows a scene in which a blood vessel D is dissected using forceps 20B. Examples of scene features are body tissue (blood vessels, organs, etc.) and treatment tools 20A and 20B in image B. The processor 7 detects the type of at least one of the body tissue and treatment tools 20A and 20B in each image B using, for example, a known image recognition technique.

[0067] 9C shows a scene in which a blood vessel is treated using forceps 20B. Another example of a scene feature is that the distance α between a predetermined body tissue and the distal end 20a of treatment tool 20B is equal to or less than a predetermined threshold. For example, processor 7 recognizes region E of blood vessel D and region G of treatment tool 20B in image B, calculates the distance α between the center of gravity of region E and the distal end 20a, and detects the distance α as a feature if it is equal to or less than the threshold.

[0068] Next, the processor 7 calculates, for each detected scene feature, an average value of the control parameter 12 or information correlated therewith, such as the average value of the distance d', as a recommended value. This allows a recommended value of the control parameter 12 appropriate for each scene to be calculated. The recommended value may be a value other than the average, such as the median, mode, or longest value of the control parameter 12 or information correlated therewith, or a time-weighted average value Σyi·ti / Σti, where yi is the value of the control parameter 12, and ti is the length of time that the control parameter yi has been used.

[0069] Next, similar to step SC3, the processor 7 presents a query to the user by, for example, displaying an optical display on the display device 5 or outputting a voice message (step SC23). The user inputs a response to the query into the control device 6 using the user interface 4.

[0070] If an answer to the effect of the change is received (YES in step SC23), processor 7 changes control parameters 12 to the recommended values ​​(step SC24), and stores the changed control parameters 12 in storage unit 9 in association with accompanying information 13 and the characteristics of the scene (step SC25). On the other hand, if an answer to the effect of the change is received (NO in step SC23), processor 7 does not perform steps SC24 and SC25, and ends the post-operative process SC.

[0071] When the control device 6 is used after the control parameters 12 are associated with the scene features and stored in the storage unit 9, the processor 7 sets the control parameters 12 based on the surgery information and the user's responses in the pre-surgery step SA (steps SA2 to SA5). If the adjusted control parameters 12 are set as the control parameters 12 for the upcoming surgery (step SA5), the processor 7 detects the scene features from the image B and controls the moving device 3 based on the control parameters 12 corresponding to the features in the intra-surgery step SB. As a result, during the surgery, the control parameters 12, such as the tracking parameter P such as the base distance d, are automatically changed to an appropriate value according to the scene.

[0072] As described above, according to this embodiment, intraoperative images B are stored in the storage unit 9 as history data 14 in association with the accompanying information 13, and recommended values ​​of the control parameters 12 for each scene are calculated from the history data 14. This allows appropriate control parameters 12 to be obtained without requiring the surgeon to do any effort.

[0073] Furthermore, according to this embodiment, after surgery, the control parameters 12 that have been changed to recommended values ​​are stored in the storage unit 9 in association with the accompanying information 13, and the surgeon is prompted as to whether or not to use the changed control parameters 12 in subsequent surgeries. Therefore, in subsequent surgeries, the surgeon can use the control parameters 12 that he or she adjusted himself or herself from the start of the surgery, thereby eliminating the need to adjust the control parameters 12.

[0074] Furthermore, according to this embodiment, the processor 7 inquires of the user as to whether or not to change the control parameters 12 to the recommended values, and the user determines whether or not to change them. This allows the control parameters 12 to be changed only when the user wishes. Similarly, the processor 7 inquires of the user as to whether or not to apply the changed control parameters 12 to the upcoming surgery, and the user determines whether or not to apply them. This allows the adjusted control parameters 12 to be applied to the upcoming surgery only when the user wishes.

[0075] In the present embodiment, the processor 7 stores the images B during the intra-operative step SB as the history data 14. However, instead of this, the processor 7 may store the control parameters 12 or information correlated with the control parameters 12, and scene features as the history data 14. That is, in the intra-operative step SB, the processor 7 detects the control parameters 12 or information correlated therewith, and scene features from each image B input from the endoscope 2 to the control device 6, and stores the detected control parameters 12 or information correlated therewith, and scene features in the storage unit 9 as the history data 14. In the post-operative step SC, the processor 7 calculates recommended values ​​of the control parameters 12 for each scene from the history data 14.

[0076] In each of the above embodiments, the processor 7 changes the control parameters 12 to the recommended values ​​based on the user's response. Alternatively, the processor 7 may automatically change the control parameters 12 to the recommended values ​​without querying the user about whether to change the control parameters 12. In this case, steps SC3, SC13, and SC23 are omitted in each of the embodiments. Similarly, in each of the embodiments, the processor 7 may apply the control parameters 12 associated with accompanying information that matches the surgery information to the upcoming surgery without querying the user. In this case, step SA4 is omitted in each of the embodiments.

[0077] In each of the above embodiments, the processor 7 calculates the recommended values ​​in the post-operative step SC. Alternatively, the recommended values ​​may be calculated in the pre-operative step SA. In this case, the processor 7 executes steps SC2 to SC4, SC12 to SC14, or SC22 to SC24 in the pre-operative step SA of a subsequent surgery. That is, in the pre-operative step SA, the processor 7 reads from the storage unit 9 the history data 14 associated with the accompanying information 13 that matches the surgery information, calculates recommended values ​​for the control parameters 12 from the history data 14, and asks the user whether to change the control parameters 12 to the recommended values. This also allows the surgeon to use the appropriate control parameters 12 that he or she adjusted in a previous surgery from the start of the surgery, thereby eliminating the need to adjust the control parameters 12.

[0078] There are multiple types of treatment tools 20 that can be used in surgery, and therefore there are multiple types of treatment tools 20 that can be tracked. The preferred control parameters 12 may differ for each type of treatment tool 20. Therefore, in each embodiment, the processor 7 may select the treatment tool 20 to which the adjusted control parameters 12 are to be applied based on the type of treatment tool 20.

[0079] 10A to 10C illustrate treatment tools 20 to which an adjusted following parameter P is applied. In FIG. 10A , an adjusted following parameter P (e.g., base distance) is applied to each type of treatment tool 20. In FIG. 10B , an adjusted following parameter P is applied to each group of treatment tools 20. Each group includes, for example, treatment tools 20 of the same type or with a similar usage method. In FIG. 10C , an adjusted following parameter P is applied to all types of treatment tools 20. The adjusted following parameter P may also be applied to an object other than the treatment tool 20. For example, the object may be an organ such as a blood vessel.

[0080] In each of the above embodiments, the storage unit 9 may store one or more pieces of history data 14 in association with each item. For example, the storage unit 9 may store one or more pieces of history data 14 of surgeries performed by surgeon A in association with surgeon A, and store one or more pieces of history data 14 of surgeries performed by surgeon B in association with surgeon B. In this case, the processor 7 may calculate recommended values ​​of the control parameters 12 from at least one piece of history data 14 for each item. For example, the processor 7 may calculate recommended values ​​for surgeon A from one or more pieces of history data 14 associated with surgeon A, and calculate recommended values ​​for surgeon B from one or more pieces of history data 14 associated with surgeon B. This configuration makes it possible to automatically create tracking parameters P appropriate for each condition, such as the surgeon, surgical procedure, target organ, facility, patient, etc.

[0081] In each of the above embodiments, the processor 7 searches for associated information that matches the surgical information in at least the surgeon's name. However, instead, the processor 7 may search for associated information that matches the surgical information in at least one arbitrary item. For example, in step SA2, the processor 7 may search for associated information that matches the surgical information in terms of the surgical procedure or patient. This configuration allows the surgeon to be suggested control parameters 12 according to various surgical conditions. In this case, as described above, the processor 7 may suggest control parameters 12 calculated from one or more history data 14 associated with each item.

[0082] In each of the above embodiments, the processor 7 calculates the recommended value from one piece of history data 14. Alternatively, the recommended value may be calculated from multiple pieces of history data 14. One example of the multiple pieces of history data 14 is history data 14 of multiple surgeries performed by the same surgeon. The multiple surgeries may be multiple recent surgeries performed by the same surgeon, or surgeries using the same surgical procedure at the same facility performed by the same surgeon. Another example of the multiple pieces of history data 14 is history data 14 of the first predetermined number of surgeries using the control device 6. The predetermined number may be one or multiple surgeries. In this case, the initial setting control parameters 12 are used for the predetermined number of surgeries, and the recommended value of the control parameters 12 is calculated after the predetermined number of surgeries have been performed.

[0083] In each of the above embodiments, the processor 7 proposes to the user control parameters 12 associated with the same surgeon as the surgeon in the surgical information, but instead, the processor 7 may propose to the user control parameters 12 associated with a surgeon different from the surgeon in the surgical information.

[0084] For example, the processor 7 may propose control parameters 12 calculated based on history data 14 of surgeries previously performed by a second surgeon as control parameters 12 for a surgery to be performed by a first surgeon. The first surgeon is the surgeon of the surgery information (i.e., the surgeon who will perform the surgery), and the second surgeon is a surgeon different from the first surgeon. With this configuration, the first surgeon can easily use appropriate control parameters 12 adjusted by the second surgeon for the surgery.

[0085] The second surgeon may be a surgeon with more surgical experience than the first surgeon. For example, the processor 7 may determine the second surgeon based on the surgeon's history (number of surgeries performed) in the surgeon information. The second surgeon is an experienced surgeon who has performed more surgeries than the first surgeon. With this configuration, the less experienced first surgeon can use the appropriate control parameters 12 adjusted by the more experienced second surgeon in a previous surgery, and reproduce the appropriate field of view F during the surgery by the second surgeon in the first surgeon's surgery.

[0086] The processor 7 may determine the second surgeon based on information other than the surgeon's career history. In one example, the processor 7 determines the second surgeon based on the facility to which the surgeon belongs. For example, the first surgeon is a surgeon at a branch hospital of a hospital, and the second surgeon is a surgeon at the main hospital of the hospital. In another example, the processor 7 determines the second surgeon based on the surgical field and specialty. For example, in a gallbladder surgery, the first surgeon specializes in colorectal surgery but is assigned to perform laparoscopic cholecystectomy, while the second surgeon is a general surgeon. In another example, the processor 7 determines the second surgeon based on patient information. For example, the second surgeon is a surgeon who has performed more surgeries on petite female patients than the first surgeon. The processor 7 may calculate the recommended value of the control parameter 12 from the history data 14 of past surgeries performed by multiple second surgeons. For example, the processor 7 may calculate the recommended value from the history data 14 associated with multiple second surgeons who have performed many surgeries on petite female patients.

[0087] In each of the above embodiments, in addition to or instead of after surgery, the processor 7 may inquire of the user whether to change the control parameters 12 during the current surgery when a change in the control parameters 12 based on the operation of the user interface 4 satisfies a predetermined condition. With this configuration, the control parameters 12 can be changed to appropriate values ​​during surgery, reducing the effort required for the surgeon to adjust the control parameters 12 during surgery.

[0088] 11A shows a control method according to a modified example of the first embodiment. The predetermined condition is that the number of changes to the control parameter 12 based on the operation of the user interface 4 is equal to or greater than a predetermined number N. That is, when the number of changes to the tracking parameter P reaches the predetermined number N (YES in step SB7), the processor 7 calculates a recommended value for the tracking parameter P from the history data 14 of the current surgery recorded up to that point (step SB8) and asks the user whether or not to change the tracking parameter P to the recommended value (step SB9). If the user responds that they wish to change the tracking parameter P (YES in step S9), the processor 7 changes the tracking parameter P to the recommended value (step SB10).

[0089] The predetermined condition may be other conditions. Another example of the predetermined condition is that the following parameter P is changed a predetermined number of times N or more within a predetermined time from the start of surgery. In this case, it is possible to prevent unnecessary suggestions for changing the recommended value due to the increase in the number of changes in the latter half of surgery. Another example of the predetermined condition is that after the following parameter P is changed, the following parameter P is continuously used for a predetermined period of time or more. Surgeons tend to continue using appropriate following parameters P for long periods of time. Therefore, it is possible to determine appropriate following parameters P for a surgeon based on the continued use of the same following parameter P for a long period of time.

[0090] 11B shows a control method according to a modified example of the second embodiment. The predetermined condition is that the number of adjustments of the predetermined adjustment amount of the tracking parameter P is equal to or exceeds a predetermined number N. That is, when the number of adjustments of the tracking parameter P by the predetermined adjustment amount reaches the predetermined number N (YES in step SB17), processor 7 calculates a recommended value for the unit change amount Δ from the history data 14 of the current surgery recorded up to that point (step SB18) and asks the user whether or not to change the unit change amount Δ to the recommended value (step SB19). If the user responds that the change should be made (YES in step SB19), processor 7 changes the unit change amount Δ to the recommended value (step SB20).

[0091] The predetermined condition may be other conditions. Another example of the predetermined condition is that the user interface 4 is continuously operated a predetermined number of times N or more within a predetermined time from the start of surgery, regardless of the adjustment amount. Another example of the predetermined condition is that a predetermined adjustment amount and an adjustment that is an integer multiple of the predetermined adjustment amount are performed a predetermined number of times N or more.

[0092] Although the embodiment of the present invention and its modified examples have been described above, the present invention is not limited to this and can be modified as appropriate within the scope of the present invention.

[0093] REFERENCE SIGNS LIST 1 Endoscope system 2 Endoscope 3 Moving device 4 User interface 6 Control device 7 Processor 9 Storage unit 11 Control program 12 Control parameters 13 Associated information 14 History data 20, 20A, 20B Treatment tool B Image C Specific area

Claims

1. An endoscope system that controls a field of view of an endoscope based on a control parameter, a storage unit that stores history data of the control parameters during surgery in association with accompanying information of the surgery, the accompanying information including information on at least one of a surgeon, a surgical procedure, and a patient of the surgery; a processor; the processor calculating recommended values ​​for the control parameters based on the historical data; An endoscope system, wherein the history data is an operation log of a user interface for changing the control parameters.

2. the control of the field of view of the endoscope is a tracking control for making the endoscope track a target so as to keep the target within a specific region within the field of view; The endoscope system according to claim 1 , wherein the control parameter is a tracking parameter related to at least one of a positional relationship between the endoscope and the specific region and a tracking speed of the endoscope relative to the tracking target.

3. the control of the field of view of the endoscope is a tracking control for making the endoscope track a target so as to keep the target within a specific region within the field of view; The endoscopic system according to claim 1 , wherein the control parameter is a change amount, by operation of a user interface, of a tracking parameter relating to at least one of a positional relationship between the endoscope and the specific region and a tracking speed of the endoscope relative to the tracking target.

4. The endoscope system according to claim 3 , wherein the change amount is a unit change amount of the tracking parameter per one operation of the user interface.

5. The endoscope system according to claim 3 , wherein the history data is a history of an adjustment amount of the tracking parameter by an operation of the user interface, and the adjustment amount is an amount of change of the tracking parameter within a predetermined time.

6. the storage unit stores one or more pieces of history data in association with each item included in the associated information; The endoscope system according to claim 1 , wherein the processor calculates the recommended value from at least one of the historical data for each of the items.

7. The processor, receiving surgical information relating to at least one of a surgeon, a surgical procedure, and a patient of an upcoming surgical procedure; The endoscopic system according to claim 1, wherein a recommended value calculated from the historical data associated with the accompanying information that matches the surgical information in at least one of the surgeon, surgical procedure, and patient is set as the control parameter in the upcoming surgery.

8. The processor, querying a user as to whether to change the control parameters to the calculated recommended values; The endoscope system according to claim 1 , wherein when a response to change the control parameter is received, the control parameter is changed to the calculated recommended value.

9. The endoscope system of claim 1 , wherein the processor automatically changes the control parameters to the calculated recommended values.

10. The endoscopic system of claim 1 , wherein the processor changes the control parameter to the recommended value calculated based on the historical data of the current surgery if the control parameter has been changed by the user a predetermined number of times or more during the current surgery.

11. The endoscope system according to claim 10 , wherein the processor changes the control parameter to the recommended value when the control parameter is changed a predetermined number of times or more within a predetermined time from the start of the surgery.

12. The endoscope system according to claim 1 , wherein the recommended value is a mean value, a median value, a mode value, a value with the longest usage time, or a time-weighted average value of the control parameter in the history data.

13. A moving device for changing the position and attitude of the endoscope is further provided. The endoscopic system of claim 1 , wherein the processor controls the movement device based on the control parameters.

14. an endoscope for acquiring images; The endoscope system according to claim 13 , further comprising: a user interface that accepts a user's operation for changing the control parameter.

15. 1. A control method for controlling a field of view of an endoscope based on a control parameter, comprising: The historical data of the control parameters during the surgery is stored in a storage unit in association with accompanying information of the surgery, the accompanying information including information on at least one of the surgeon, the surgical procedure, and the patient of the surgery; and calculating recommended values ​​of the control parameters based on the historical data; A control method, wherein the history data is an operation log of a user interface for changing the control parameters.

16. A control program for causing a computer to execute a control method for controlling a field of view of an endoscope based on a control parameter, comprising: The control method comprises: The historical data of the control parameters during the surgery is stored in a storage unit in association with accompanying information of the surgery, the accompanying information including information on at least one of the surgeon, the surgical procedure, and the patient of the surgery; and calculating recommended values ​​of the control parameters based on the historical data; A control program, wherein the history data is an operation log of a user interface for changing the control parameters.