Control device, endoscope system, storage medium, and program product

Abstract

A control device controls movement of an endoscope so that the endoscope follows a treatment instrument. The control device includes a processor. The processor acquires position information including a position of the treatment instrument (S2), acquires scene information associated with a treatment scene observed by the endoscope (S4), determines a shift parameter of a target point based on the scene information (S6), the shift parameter being a parameter that determines a position of the target point relative to a reference point defined within a field of view of the endoscope, sets the position of the target point relative to the reference point based on the shift parameter (S7), and controls movement of the endoscope based on the position of the target point and the position of the treatment instrument, thereby causing the endoscope to follow the treatment instrument to dispose the treatment instrument at the target point (S3).

Description

Technical Field

[0001] This invention relates to control devices, endoscope systems, storage media, and program products, and particularly to control devices, endoscope systems, storage media, and program products for controlling an endoscope to follow a treatment instrument.

[0002] This application claims priority to U.S. Provisional Application No. 63 / 076,408, filed provisionally in the United States on September 10, 2020, the contents of which are incorporated herein by reference. Background Technology

[0003] In laparoscopic surgery, the surgeon operates the instrument while observing the endoscopic image displayed on a monitor. When the instrument is located at the end of the endoscopic image or detached from it, the surgeon experiences increased pressure or finds it difficult to continue the surgery. Therefore, it is important to keep the instrument continuously positioned near the center of the endoscopic image. Consequently, a device has been proposed that continuously positions the instrument near the center of the endoscopic image by having the endoscope follow it (see, for example, Patent Document 1).

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 5-337118 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] The relationship between the area at the center of the endoscopic image that the surgeon wants to capture for the procedure and the procedure instrument varies depending on the procedure being performed. Therefore, as in Patent Document 1, when the endoscope is followed by the procedure instrument to position the procedure instrument in a fixed position relative to the endoscope, it is not possible to continuously capture the area that the surgeon wants to observe at the center of the endoscopic image.

[0009] For example, such as Figure 22A As shown, when dissecting biological tissue from the lower left to the upper right, although the surgeon wants to observe the area C in front of the direction of travel of the electrosurgical scalpel 6, area C is positioned at the upper right end or outside of the endoscopic image B. Figure 22B As shown, when using two forceps 6 to unfold biological tissue, even if the surgeon wants to observe the area C between the two forceps 6, the area C is positioned at the left end or outside of the endoscopic image B.

[0010] The present invention was made in view of the above circumstances, and its object is to provide a control device, an endoscope system, and a control method capable of positioning the treatment instrument in an appropriate position within an endoscopic image according to the treatment scenario.

[0011] Methods for solving problems

[0012] One aspect of the present invention is a control device for controlling the movement of an endoscope to follow a treatment instrument. The control device includes a processor that acquires position information including the position of the treatment instrument, acquires scene information associated with a treatment scene observed by the endoscope, determines an offset parameter for a target point based on the scene information, the offset parameter being a parameter that determines the position of the target point relative to a predetermined reference point within the field of view of the endoscope, sets the position of the target point relative to the reference point based on the offset parameter, and controls the movement of the endoscope based on the position of the target point and the position of the treatment instrument, thereby causing the endoscope to follow the treatment instrument and position the treatment instrument at the target point.

[0013] Another aspect of the invention is an endoscope system comprising: an endoscope; a moving device for moving the endoscope within a subject; and the aforementioned control device for controlling the moving device to cause the endoscope to follow a treatment instrument.

[0014] Another aspect of the present invention is a control method for controlling the movement of an endoscope to follow a treatment device, the control method comprising the steps of: obtaining position information including the position of the treatment device; obtaining scene information, the scene information being information associated with a treatment scene observed by the endoscope; determining an offset parameter of a target point based on the scene information, the offset parameter being a parameter determining the position of the target point relative to a predetermined reference point within the field of view of the endoscope; setting the position of the target point relative to the reference point based on the offset parameter; and controlling the movement of the endoscope based on the position of the target point and the position of the treatment device, thereby causing the endoscope to follow the treatment device to position the treatment device at the target point.

[0015] Invention Effects

[0016] According to the present invention, it has the effect of enabling the treatment device to be positioned appropriately within the endoscopic image according to the treatment scenario. Attached Figure Description

[0017] Figure 1 This is an external view of the overall structure of the endoscope system according to the first embodiment.

[0018] Figure 2A It is shown Figure 1 A block diagram of the overall structure of the endoscope system.

[0019] Figure 2B yes Figure 1 Functional block diagram of the control device of the endoscope system.

[0020] Figure 3A It is a diagram illustrating the reference points and target points set within the field of view of an endoscope.

[0021] Figure 3B It is a diagram illustrating the reference points and target points within an endoscopic image.

[0022] Figure 4A This is a diagram illustrating an example of a method for detecting the movement vector of a treatment device.

[0023] Figure 4B This is an explanation based on Figure 4A The offset parameters and the target point are determined by the movement vector.

[0024] Figure 5 This is another example of a method for detecting the movement vector of a treatment device.

[0025] Figure 6A This is an example of an endoscopic image with the scene stripped away.

[0026] Figure 6B This is another example of an endoscopic image with a scene stripped away.

[0027] Figure 7 This is a flowchart of the control method of the first embodiment.

[0028] Figure 8 This is a diagram illustrating an example of a parameter table showing the correspondence between the movement vector and offset parameters of a treatment device.

[0029] Figure 9 This is a flowchart of a variation of the control method of the first embodiment.

[0030] Figure 10 This is a functional block diagram of the control device of the endoscope system according to the second embodiment.

[0031] Figure 11 This is an example of an endoscopic image of a unfolding scene.

[0032] Figure 12 This is a flowchart of the control method of the second embodiment.

[0033] Figure 13 This is a flowchart of the control method in the third embodiment.

[0034] Figure 14A This is an example of an endoscopic image illustrating the movement of the stripping line and the treatment instrument.

[0035] Figure 14BThis is another example of an endoscopic image illustrating the movement of the stripping line and the treatment instrument.

[0036] Figure 15 This is a flowchart of the control method in the fourth embodiment.

[0037] Figure 16 This is a flowchart of the control method in the fifth embodiment.

[0038] Figure 17 This is a flowchart of a variation of the control method of the fifth embodiment.

[0039] Figure 18A This is an example diagram showing the position and orientation of an endoscope.

[0040] Figure 18B It is shown Figure 18A Endoscopic images of the target point's position and orientation.

[0041] Figure 19A This is another example of the position and orientation of an endoscope.

[0042] Figure 19B It is shown Figure 19A Endoscopic images of the target point's position and orientation.

[0043] Figure 20A It is a diagram illustrating two-dimensional or three-dimensional target points set within the field of view of an endoscope.

[0044] Figure 20B It is a diagram illustrating two-dimensional or three-dimensional target points within an endoscopic image.

[0045] Figure 21A yes Figure 1 An external view of the overall structure of a modified example of an endoscope system.

[0046] Figure 21B yes Figure 1 An external view of the overall structure of another variation of the endoscope system.

[0047] Figure 22A This is an example of an endoscopic image of a previous dissection scene.

[0048] Figure 22B This is an example of an endoscopic image of an unfolding scene. Detailed Implementation

[0049] (First Implementation)

[0050] The control device, endoscope system, and control method of the first embodiment of the present invention will be described with reference to the accompanying drawings.

[0051] like Figure 1 As shown, the endoscope system 10 of this embodiment is used to insert an endoscope 2 and one or more treatment instruments 6 into the body of a patient P who is the subject of the examination, and to perform surgery by observing the treatment instruments 6 with the endoscope 2 while treating the treatment site with the treatment instruments 6, for example, for laparoscopic surgery.

[0052] like Figure 1 and Figure 2A As shown, the endoscope system 10 includes: an endoscope 2; a moving device 3 that moves the endoscope 2 within the body of the patient P; an endoscope processor 4 connected to the endoscope 2 and processing the endoscopic images acquired by the endoscope 2; a control device 1 connected to the moving device 3 and the endoscope processor 4 and controlling the moving device 3; and a display device 5 connected to the endoscope processor 4 and displaying the endoscopic images.

[0053] Endoscope 2 is, for example, a rigid endoscope, and includes an imaging unit 2a with an imaging element that acquires an endoscopic image B. The imaging unit 2a is, for example, a three-dimensional camera disposed at the front end of endoscope 2, acquiring a stereoscopic image including the front end 6a of the treatment device 6 as the endoscopic image B (for example, see reference...). Figure 3B Additionally, for example, the camera unit 2a is an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, which converts light received from a specified area into an electrical signal through photoelectric conversion, thereby generating an image of the specified area. The stereoscopic image E, which is the endoscope image, is generated by image processing of two images with parallax by the endoscope processor 4, etc.

[0054] Endoscopic image B is sent from endoscope 2 to endoscope processor 4, where necessary processing is performed. The image is then sent from endoscope processor 4 to display device 5 for display. The surgeon observes the endoscopic image B displayed on display device 5 while operating the inserted surgical instrument 6. Display device 5 can be any display such as a liquid crystal display (LCD) or organic EL display. Display device 5 may also include a speaker or other sound device.

[0055] In addition to the display device 5, a terminal can also be provided for user use, communicating with the control device 1 and the endoscope processor 4 via a communication network, and displaying the endoscopic image E on the terminal. The terminal is not particularly limited and can be a laptop computer, tablet computer, or smartphone, etc.

[0056] The moving device 3 has a robotic arm 3a (including an electric endoscope holder) that is connected to the base of the endoscope 2 and performs three-dimensional control of the position and orientation of the endoscope 2. Figure 1 The moving device 3 has a robotic arm 3a with multiple joints 3b. Through the movement of the joints 3b, the endoscope 2 can be moved in three dimensions, thereby changing the position and orientation of the endoscope 2 in three dimensions.

[0057] like Figure 2A As shown, the control device 1 includes at least one processor 1a (similar to a central processing unit), a memory 1b, a storage unit 1c, and a user interface 1d. The control device 1 can be, for example, a desktop computer, a tablet computer, a laptop computer, a smartphone, or a mobile phone.

[0058] Processor 1a can be a single-processor, multi-processor, or multi-core processor. Processor 1a reads and executes the program stored in storage unit 1c.

[0059] The memory 1b is, for example, a semiconductor memory containing ROM (read-only memory) or RAM (random access memory) regions. The memory 1b may also store data required for processing by the processor 1a in the same way as the storage unit 1c described later (i.e., it may also operate as a "storage unit").

[0060] The storage unit 1c is a non-volatile recording medium containing semiconductor memory such as a hard disk or flash memory, storing programs and data required for the processor 1a to perform processing. The processor 1a performs processing according to the program read into the memory 1b, thereby realizing the functions of the units 11, 12, 13, and 14 described later. Some of the functions of the control device 1 can also be implemented using dedicated logic circuits and hardware such as FPGA (Field Programmable Gate Array), SoC (System-on-a-Chip), ASIC (Application Specific Integrated Circuit), and PLD (Programmable Logic Device).

[0061] The control device 1 has a manual mode and a follow mode, and switches between manual mode and follow mode based on the instructions of the operator such as the surgeon, for example, based on the operator's voice.

[0062] The manual mode is a mode in which the operator manually operates the endoscope 2. In the manual mode, for example, the operator can remotely operate the robotic arm 3a or the electric retainer by operating the operating device (not shown) connected to the control device 1.

[0063] The following mode is a mode in which the endoscope 2 automatically follows the treatment device 6 that is set as the object to be followed, by the control device 1 controlling the moving device 3.

[0064] like Figure 3A and Figure 3B As shown, in the following mode, the control device 1 obtains the three-dimensional position of the tip 6a of the treatment device 6, and controls the moving device 3 based on the three-dimensional position of the tip 6a and the three-dimensional position of the target point T within the field of view F. Thus, the control device 1 controls the movement of the endoscope 2, causing the endoscope 2 to follow the treatment device 6 so that the target point T is positioned at the tip 6a.

[0065] exist Figure 3A as well as Figure 3B In the image, the direction parallel to the optical axis A of endoscope 2 is the X direction, the direction orthogonal to the optical axis A and corresponding to the horizontal direction of the endoscope image B is the Y direction, and the direction orthogonal to the optical axis A and corresponding to the vertical direction of the endoscope image B is the Z direction.

[0066] The target point T is initially set as a reference point O on optical axis A, located a predetermined distance D away from the tip 2b of endoscope 2 in a direction parallel to optical axis A. The control device 1, based on the treatment scenario observed by endoscope 2, causes the target point T to shift three-dimensionally from reference point O to other positions within the field of view F. Therefore, the position of the tip 6a within endoscopic image B varies between the center of endoscopic image B and a position shifted from the center, depending on the treatment scenario.

[0067] Specifically, such as Figure 2B As shown, the control device 1 includes: a position information acquisition unit 11 that acquires position information; a scene information acquisition unit 12 that acquires scene information; an offset parameter determination unit 13 that determines an offset parameter based on the scene information; and a control unit 14 that controls the position and orientation of the endoscope 2 based on the offset parameter and the position information.

[0068] The position information acquisition unit 11 acquires position information of an object present in the endoscopic image B from the endoscope processor 4. The position information includes at least the three-dimensional position of the tip 6a of the treatment device 6. Therefore, the endoscope processor 4 performs a process to calculate the position information, including the three-dimensional position of the tip 6a, based on the endoscopic image B. The process of calculating the position information can also be performed by the position information acquisition unit 11.

[0069] Scene information is information associated with the treatment scene observed by the endoscope 2. The movement of the treatment instrument 6 varies depending on the treatment scene. For example, in a dissection scene where biological tissue is peeled off using the treatment instrument 6, the surgeon moves the treatment instrument 6 slowly. The scene information acquisition unit 12 acquires the three-dimensional movement vector (velocity vector) V of the treatment instrument 6, representing its movement, as scene information.

[0070] For example, the scene information acquisition unit 12, such as Figure 4A As shown, the movement vector V is detected based on the endoscopic image B, or as... Figure 5 As shown, the movement vector V is detected based on the movement of the endoscope 2. The scene information acquisition unit 12 can also acquire the movement vector V using any method other than these.

[0071] exist Figure 4A In this method, the motion vector V is detected based on two or more endoscopic images B taken at different times. Specifically, the motion vector V is calculated according to the following formula.

[0072] V = P1 - P2

[0073] P1 is the three-dimensional position vector (x1, y1, z1) of the front end 6a in the endoscopic image B at time t, and P2 is the three-dimensional position vector (x2, y2, z2) of the front end 6a in the endoscopic image B at time t+Δt after time t. The endoscopic image B is input to the control device 1 directly from the endoscope 2 or via the endoscope processor 4.

[0074] exist Figure 5 In this method, the movement vector V is detected based on the movement of the endoscope 2 following the treatment device 6. That is, the endoscope 2 follows the treatment device 6 such that the tip 6a of the treatment device 6 and the tip 2b of the endoscope 2 maintain a predetermined positional relationship. Therefore, the movement of the tip 2b of the endoscope 2 and the movement of the tip 6a of the treatment device 6 are in a certain relationship.

[0075] The offset parameter determination unit 13 determines the offset parameters based on the movement vector V. The offset parameters are parameters that determine the position of the target point T relative to the reference point O, such as parameters that represent the offset direction and offset distance of the target point T relative to the reference point O.

[0076] Specifically, such as Figure 6A and Figure 6B As shown, the offset parameter determination unit 13 determines, based on the direction of the movement vector V, the offset parameter that will position the region C in front of the treatment device 6 in the direction of movement of the treatment device 6 at the center of the field of view F (i.e., on the optical axis A). Figure 6A and Figure 6B In the image, the arrow indicates the direction of movement of the treatment device 6.

[0077] For example, the offset parameter determination unit 13 calculates a three-dimensional vector in the opposite direction to the movement vector V as the offset parameter by substituting the movement vector V into a predetermined function F(V). The magnitude of the three-dimensional vector can be fixed or determined based on the magnitude of the movement vector V.

[0078] For example, such as Figure 4B As shown, F(V) = -k*V, where k is a coefficient. In this case, the larger the movement vector V, the larger the three-dimensional vector F(V), and the greater the offset distance of the target point T relative to the reference point O.

[0079] Here, the offset parameter determination unit 13 determines whether the magnitude |V| of the movement vector V, i.e., the speed of the handling device 6, is below a predetermined threshold α. If |V| is below the threshold α, the offset parameter determination unit 13 determines the offset parameter and outputs the offset parameter to the control unit 14. On the other hand, if |V| is greater than the threshold α, the offset parameter determination unit 13 does not determine the offset parameter.

[0080] The control unit 14 calculates the three-dimensional position of the target point T, which is used to position the area C in front of the treatment device 6 in the direction of movement of the device 6 at the center of the field of view F, based on the offset parameters. For example, the control unit 14 calculates the position after offsetting the three-dimensional vector (-k*V) from the reference point O in the direction of the three-dimensional vector (-k*V) as the three-dimensional position of the target point T. Then, the control unit 14 sets the target point T at the calculated three-dimensional position, causing the target point T to be offset from the reference point O.

[0081] Furthermore, the control unit 14 receives information on the three-dimensional position of the front end 6a of the processing device 6 from the position information acquisition unit 11.

[0082] Next, the control unit 14 calculates the position and orientation of the tip 2b of the endoscope 2, which is positioned at the three-dimensional location of the target point T at the tip 6a. Then, the control unit 14 calculates the amount of motion (e.g., the rotation angle of each joint 3b) of the moving device 3 used to position the tip 2b at the calculated position and orientation, and causes the moving device 3 to move with the calculated amount of motion. As a result, the tip 2b of the endoscope 2 follows the tip 6a of the treatment instrument 6, and the target point T moves toward the tip 6a.

[0083] Next, the control method executed by the control device 1 will be explained.

[0084] When switching to follow mode, the processor 1a of the control device 1 executes... Figure 7 The steps S1 to S7 shown cause the endoscope 2 to automatically follow the treatment device 6.

[0085] When the follow mode starts, the control unit 14 initially sets the target point T as the reference point O, which is the center of the field of view F of the endoscope 2 (step S1).

[0086] Next, the position information acquisition unit 11 acquires position information including the three-dimensional position of the front end 6a of the processing device 6 (step S2).

[0087] Next, the control unit 14 causes the tip 2b of the endoscope 2 to follow the tip 6a of the treatment device 6, so that the tip 6a is positioned at the target point T (step S3). Specifically, the control unit 14 controls the moving device 3 based on the three-dimensional position of the tip 6a and the target point T, so that the endoscope 2 moves, and the target point T within the field of view F is positioned at the position of the tip 6a.

[0088] The processes of steps S4 to S7 for adjusting the position of the target point T are performed in parallel with the control of the endoscope 2 to follow the treatment device 6.

[0089] First, the scene information acquisition unit 12 acquires the movement vector V of the processing device 6 (step S4).

[0090] If the magnitude of the movement vector |V| is greater than the threshold α (No in step S5), the processing in steps S6 and S7 is not performed, and the control unit 14 maintains the target point T as the reference point O (step S1). Therefore, the endoscope 2 follows the treatment device 6 so that the tip 6a is positioned at the center within the endoscopic image B.

[0091] On the other hand, when the magnitude of the movement vector |V| is below the threshold α ("Yes" in step S5), the offset parameter determination unit 13 determines the offset parameter for positioning the area C in front of the treatment device 6 in the movement direction at the reference point O based on the movement vector V (step S6). Next, the control unit 14 sets the target point T at a position offset from the reference point O in the opposite direction to the movement vector V based on the offset parameter (step S7). Therefore, the endoscope 2 follows the treatment device 6, such that the tip 6a of the treatment device 6 is positioned at the target point T offset from the center of the endoscopic image B.

[0092] When dissecting biological tissue using the instrument 6, the surgeon moves the instrument 6 slowly, thus reducing the speed |V| of the instrument 6. Therefore, as... Figure 6A or Figure 6B As shown, when the surgeon begins dissection using instrument 6, the target point T shifts from the reference point O at the center of the field of view F in the opposite direction to the movement direction of instrument 6. Consequently, the tip 6a shifts from the center of the endoscopic image B in the opposite direction to the dissection direction, and the area C to be dissected is then positioned at the center of the endoscopic image B. When dissection is complete and the speed of instrument 6 exceeds a threshold α, the target point T returns to the reference point O, and the tip 6a is positioned at the center of the endoscopic image B.

[0093] Thus, the operation of the treatment device 6 varies depending on the treatment scenario. According to this embodiment, by shifting the target point T three-dimensionally from the reference point O according to the movement vector of the treatment device 6, the tip 6a of the treatment device 6 in the endoscopic image B can be positioned in a position suitable for the current treatment scenario.

[0094] In particular, in dissection scenarios where the magnitude of the movement vector |V| is below the threshold α, the front end 6a is positioned after being offset from the reference point O in the opposite direction to the movement direction of the treatment instrument 6, and the area C in front of the treatment instrument 6 in the movement direction, i.e., the area to be dissected next, is positioned at the center of the endoscopic image B. Therefore, the surgeon can easily observe the dissected area C.

[0095] In this embodiment, the offset parameter determination unit 13 uses the function F(V) to determine the offset parameter, but it can also use the parameter table E that corresponds to the movement vector V to determine the offset parameter instead.

[0096] Figure 8 An example of parameter table E is shown. In parameter table E, three speed zones (E1, E2, and E3) are defined based on the magnitude of the movement vector |V|: low speed, medium speed, and high speed. For each zone, offset distances δ1, δ2, and δ3 are assigned. For example, the relationship between offset distance δ1 in zone E1, offset distance δ2 in zone E2, and offset distance δ3 in zone E3 is δ1 < δ2 < δ3. Figure 8 In this case, |V| is equivalent to the medium speed region E2, so the position after shifting from the reference point O to the opposite direction of the movement vector V by an offset distance δ2 becomes the target point T.

[0097] Figure 9 This illustrates the control method when using parameter table E. For example... Figure 9 As shown, step S5 is omitted, and the offset parameter is repeatedly determined regardless of |V|.

[0098] (Second Implementation)

[0099] Next, the control device, endoscope system, and control method of the second embodiment of the present invention will be described.

[0100] This embodiment differs from the first embodiment in that it determines the offset parameter based on the movement vector V and the type of treatment device 6. In this embodiment, structures that differ from the first embodiment will be described, while structures common to the first embodiment will be labeled with the same reference numerals and their descriptions will be omitted.

[0101] The endoscope system 10 of this embodiment includes a control device 1, an endoscope 2, a moving device 3, an endoscope processor 4, and a display device 5.

[0102] like Figure 10 As shown, in addition to the position information acquisition unit 11, scene information acquisition unit 12, offset parameter determination unit 13, and control unit 14, the control device 1 also includes a scene estimation unit 15. The scene estimation unit 15 has the same function as the other units 11, 12, 13, and 14, and is implemented by the processor 1a.

[0103] In addition to acquiring the movement vector V, the scene information acquisition unit 12 also acquires the type of the treatment device 6 set as the object to be followed as scene information. For example, the scene information acquisition unit 12 identifies the type of the treatment device 6 in the endoscopic image B by using image recognition based on artificial intelligence, thereby acquiring the type of the treatment device from the endoscopic image B. The scene information acquisition unit 12 can also acquire the type of the treatment device 6 based on information related to the type of the treatment device 6 input by the operator to the control device 1 or identification information set on the treatment device 6.

[0104] The scene estimation unit 15 estimates the treatment scene observed by the endoscope 2 based on the type of treatment instrument 6. Specifically, if the type of treatment instrument 6 is a unfolding device such as forceps, the scene estimation unit 15 estimates the treatment scene as an unfolding scene where biological tissue is unfolded using the treatment instrument 6. If the type of treatment instrument 6 is a dissection device such as an electrosurgical scalpel, the scene estimation unit 15 estimates the treatment scene as a dissection scene where biological tissue is dissected using the treatment instrument 6.

[0105] The offset parameter determination unit 13 determines the offset parameters based on the movement vector V and the handling scenario.

[0106] Specifically, in cases where the processing scenario is stripped of its context, such as... Figure 6A and Figure 6B As shown, the offset parameter determination unit 13 determines the offset parameter of the region C to be stripped from the treatment device 6, that is, the region C in front of the treatment device 6 in the direction of movement of the treatment device 6, which is positioned at the center of the field of view F.

[0107] When the scenario being handled is an expanded scenario, such as Figure 11 As shown, the offset parameter determination unit 13 determines the offset parameter of the region C expanded by the treatment device 6, that is, the region C behind the treatment device 6 in the direction of movement of the treatment device 6, at the center of the field of view F.

[0108] For example, a function F(V) or parameter table E is prepared in advance for each processing scenario and stored in the storage unit 1c. In the case of a stripping scenario, the offset parameter determination unit 13 selects the function F(V) or parameter table E for the stripping scenario, and uses the selected function F(V) or parameter table E to calculate a three-dimensional vector in the opposite direction to the movement vector V as an offset parameter. In the case of an unfolding scenario, the offset parameter determination unit 13 selects the function F(V) or parameter table E for the unfolding scenario, and uses the selected function F(V) or parameter table E to calculate a three-dimensional vector in the same direction as the movement vector V as an offset parameter.

[0109] Next, the control method executed by the control device 1 will be explained.

[0110] When switching to follow mode, the processor 1a of the control device 1 executes... Figure 12 Steps S2 to S12 shown cause the endoscope 2 to automatically follow the treatment device 6, which is the object being followed.

[0111] In this embodiment, the scene information acquisition unit 12 acquires the movement vector V (step S4), and then acquires the type of the processing device 6 (step S8).

[0112] Next, the scene estimation unit 15 estimates the treatment scene observed by the endoscope 2 based on the type of treatment device 6 (step S9).

[0113] Next, the offset parameter determination unit 13 determines the offset parameters based on the processing scene and the movement vector V (steps S10 to S12).

[0114] Specifically, in the case of an expanded scene ("expanded scene" in step S10), the offset parameter determination unit 13 calculates the offset parameter using the function F(V) for the expanded scene or the parameter table E (step S11). Thus, as... Figure 11 As shown, the target point T is offset from the reference point O at the center of the field of view F in the same direction as the movement direction of the treatment device 6, the front end 6a of the treatment device 6 is offset from the center of the endoscopic image B in the same direction as the unfolding direction, and the area C unfolded by the treatment device 6 is positioned at the center of the endoscopic image B.

[0115] On the other hand, in the case of a stripping scene ("stripping scene" in step S10), the offset parameter determination unit 13 calculates the offset parameter using the function F(V) or parameter table E for the stripping scene (step S12). Thus, as... Figure 6A and Figure 6BAs shown, the target point T is shifted from the center of the field of view F in the opposite direction to the movement direction of the treatment device 6, and the front end 6a of the treatment device 6 is shifted from the center of the endoscopic image B in the opposite direction to the stripping direction. The area C stripped by the treatment device 6 is located at the center of the endoscopic image B.

[0116] During surgery, various treatment scenarios can be observed through the endoscope 2. The type of treatment instrument 6 used varies depending on the treatment scenario. According to this embodiment, the current treatment scenario is estimated based on the type of treatment instrument 6, and the target point T is three-dimensionally offset from the reference point O based on the movement vector V and the treatment scenario. As a result, the tip 6a of the treatment instrument 6 in the endoscopic image B can be positioned appropriately for the current treatment scenario.

[0117] Specifically, during the unfolding phase, the area C unfolded by the treatment instrument 6 is positioned at the center of the endoscopic image B, thus allowing the surgeon to easily observe the unfolded area C. During the dissection phase, the area C to be dissected by the treatment instrument 6 is positioned at the center of the endoscopic image B, thus allowing the surgeon to easily observe the area C to be dissected.

[0118] In addition, the scene information acquisition unit 12 can also acquire any other information that can be recognized using artificial intelligence as the processing scene, instead of the type of processing device 6.

[0119] For example, the scene information acquisition unit 12 can also use known image recognition technology to identify the anatomical features of the subject in the endoscopic image B, and obtain information such as the type, location and orientation of the anatomical features as scene information. The scene estimation unit 15 estimates the treatment scene based on the scene information.

[0120] Alternatively, the scene information acquisition unit 12 may also detect scene changes such as bleeding in the subject within the endoscopic image B using known image recognition technology, acquire the detected information as scene information, and the scene estimation unit 15 may estimate the handling scene based on the scene information.

[0121] Furthermore, the scenario estimation is not limited to the above-described implementation method. The scenario information acquisition unit 12 can acquire any information obtained during the operation as scenario information, and the scenario estimation unit 15 can also estimate the treatment scenario based on the scenario information.

[0122] (Third Implementation)

[0123] Next, the control device, endoscope system, and control method of the third embodiment of the present invention will be described.

[0124] This embodiment differs from the first and second embodiments in that it determines the offset parameter based on the movement vector V and the type and operating state of the treatment device 6. In this embodiment, structures that differ from the first and second embodiments will be described, while structures common to the first and second embodiments will be labeled with the same reference numerals and their descriptions will be omitted.

[0125] The endoscope system 10 of this embodiment includes a control device 1, an endoscope 2, a moving device 3, an endoscope processor 4, and a display device 5.

[0126] The control device 1, like the second embodiment, includes a position information acquisition unit 11, a scene information acquisition unit 12, an offset parameter determination unit 13, a control unit 14, and a scene estimation unit 15.

[0127] In addition to acquiring the movement vector V and the type of the processing device 6, the scene information acquisition unit 12 also acquires the operating status of the processing device 6, which is the object being followed, as scene information. For example, the control device 1 is connected to the drive device (not shown) that drives the processing device 6. The processing device 6 is activated by supplying power from the drive device. The scene information acquisition unit 12 receives a signal from the drive device indicating whether the processing device 6 is activated or not.

[0128] The scene information acquisition unit 12 can also acquire the operating status through other means. For example, the scene information acquisition unit 12 can also acquire the operating status of the treatment device 6 from the endoscopic image B. When the treatment device 6 is an electrosurgical scalpel, the color of the activated electrosurgical scalpel 6 changes due to high heat compared to when it is not activated. Therefore, it is possible to identify whether the treatment device 6 is activated or not based on the endoscopic image B.

[0129] Similar to the second embodiment, the scene estimation unit 15 estimates the treatment scene based on the type of treatment device 6. Furthermore, if the estimated treatment scene is a stripping scene, the scene estimation unit 15 estimates a more detailed treatment scene based on the operating state of the treatment device 6. Specifically, if the treatment device 6 is activated, the scene estimation unit 15 estimates a scene where the treatment device 6 is activated, i.e., a scene where biological tissue is being stripped using the treatment device 6. On the other hand, if the treatment device 6 is not activated, the scene estimation unit 15 estimates a scene where the treatment device 6 is not activated, i.e., another scene within the stripping scene.

[0130] The offset parameter determination unit 13 determines the offset parameters based on the movement vector V and the handling scenario.

[0131] For example, similar to the second embodiment, a function F(V) or parameter table E is prepared in advance for each disposal scenario and stored in the storage unit 1c. That is, a function F(V) or parameter table E for unfolding a scenario, a function F(V) or parameter table E for stripping a scenario (activated), and a function F(V) or parameter table E for stripping a scenario (not activated) are prepared. The offset parameter determination unit 13 selects the function F(V) or parameter table E for the disposal scenario estimated by the scene estimation unit 15, and uses the selected function F(V) or parameter table E to calculate a three-dimensional vector as an offset parameter.

[0132] Next, the control method executed by the control device 1 will be explained.

[0133] When switching to follow mode, the processor 1a of the control device 1 executes... Figure 13 Steps S2 to S16 shown cause the endoscope 2 to automatically follow the treatment device 6, which is the object being followed.

[0134] In this embodiment, the scene information acquisition unit 12 acquires the movement vector V (step S4), acquires the type of the processing device 6 (step S8), and then acquires the working status of the processing device 6 (step S13).

[0135] Next, the scene estimation unit 15 estimates the treatment scene observed by the endoscope 2 based on the type of treatment device 6 (step S9). If the treatment scene is a stripping scene ("stripping scene" in step S10), the scene estimation unit 15 then estimates whether the treatment scene is a scene in which the treatment device 6 is activated or a scene in which it is not activated based on the working state of the treatment device 6 (step S14).

[0136] Next, the offset parameter determination unit 13 determines the offset parameters based on the processing scene and the movement vector V (steps S11, S15, S16).

[0137] Specifically, in the case of an unfolded scene ("unfolded scene" in step S10), similarly to the third embodiment, the offset parameter determination unit 13 uses the function F(V) or parameter table E for unfolded scenes to calculate the offset parameter (step S11).

[0138] If the processing device 6 in the stripping scene is in an active scene ("Yes" in step S14), the offset parameter determination unit 13 uses the function F(V) or parameter table E for the stripping scene (active) to calculate the offset parameter (step S15).

[0139] If the processing device 6 in the stripping scene is not in an active scene (No in step S14), the offset parameter determination unit 13 uses the function F(V) or parameter table E for the stripping scene (not active) to calculate the offset parameter (step S16).

[0140] There are multiple scenarios within a single type of treatment scenario, and sometimes the scenario changes during the use of the same treatment instrument 6. For example, in a peeling scenario, there is a scenario before peeling begins, in which the inactive electrosurgical scalpel 6 is moved to position itself relative to the biological tissue; a scenario during peeling, in which the activated electrosurgical scalpel 6 is moved slowly to peel away the biological tissue; and a scenario after peeling ends, in which the inactive electrosurgical scalpel 6 is moved away from the biological tissue.

[0141] According to this embodiment, a more detailed current treatment scenario can be estimated based on the type and working status of the treatment device 6, and the front end 6a of the treatment device 6 in the endoscopic image B can be positioned in a position more suitable for the current treatment scenario.

[0142] (Fourth Implementation)

[0143] Next, the control device, endoscope system, and control method of the fourth embodiment of the present invention will be described.

[0144] This embodiment differs from the first to third embodiments in that it determines the offset parameters based on the movement vector V and the anatomical information of the biological tissue. In this embodiment, structures that differ from the first to third embodiments will be described, while structures common to the first to third embodiments will be labeled with the same reference numerals and their descriptions will be omitted.

[0145] The endoscope system 10 of this embodiment includes a control device 1, an endoscope 2, a moving device 3, an endoscope processor 4, and a display device 5.

[0146] The control device 1, like the second embodiment, includes a position information acquisition unit 11, a scene information acquisition unit 12, an offset parameter determination unit 13, a control unit 14, and a scene estimation unit 15.

[0147] In addition to acquiring the movement vector V, the scene information acquisition unit 12 also acquires anatomical information of the biological tissue within the endoscopic image B as scene information. The anatomical information is information about the anatomical structure of the biological tissue associated with the treatment of the instrument 6. In one example, artificial intelligence-based image recognition technology is used in acquiring the anatomical information. For example, such as... Figure 14A and Figure 14BAs shown, the scene information acquisition unit 12 identifies the arrangement of organs and blood vessels in the endoscopic image B, and identifies the dissection line L that should be dissected by the treatment instrument 6 as anatomical information. The scene information acquisition unit 12 can also identify the dissection line L if a mark representing the dissection line L is applied to the surface of the biological tissue.

[0148] The scene estimation unit 15 estimates the handling scene based on the movement vector V and the anatomical information.

[0149] Specifically, such as Figure 14A As shown, when using the treatment instrument 6 to peel off biological tissue, the surgeon moves the treatment instrument 6 along the peeling line L, so the direction of movement of the treatment instrument 6 is consistent with the length direction of the peeling line L. When the direction of the movement vector V is consistent with the length direction of the peeling line L, the scene estimation unit 15 estimates that the treatment scene is a peeling scene of biological tissue using the treatment instrument 6.

[0150] On the other hand, such as Figure 14B As shown, when the treatment device 6 performs actions other than peeling, for example, when the treatment device 6 is moved in a direction intersecting the peeling line L to unfold the part to be peeled, the direction of movement of the treatment device 6 is not consistent with the length direction of the peeling line L. When the direction of the movement vector V is inconsistent with the length direction of the peeling line L, the scene estimation unit 15 estimates that the treatment scene is a scene other than the peeling scene.

[0151] Whether the direction of the moving vector V is consistent with the length direction of the peeling line L can be determined by, for example, whether the angle between the direction of the moving vector V and the direction of the peeling line L is a specified value.

[0152] The offset parameter determination unit 13 determines the offset parameters based on the movement vector V and the handling scenario.

[0153] Specifically, when the treatment scenario is a stripped-out scenario, the offset parameter determination unit 13 determines the offset parameter of arranging the area C in front of the treatment device 6 at the center of the field of view F in the moving direction of the treatment device 6.

[0154] When the scene being processed is a scene outside the scene itself, the offset parameter determination unit 13 determines the offset parameter for configuring the target point T at the reference point O.

[0155] Next, the control method executed by the control device 1 will be explained.

[0156] When switching to follow mode, the processor 1a of the control device 1 executes... Figure 15 Steps S2 to S18, as shown, cause the endoscope 2 to automatically follow the treatment device 6, which is the object being followed.

[0157] In this embodiment, the scene information acquisition unit 12 acquires the movement vector V (step S4), and then acquires the anatomical information in the endoscopic image B (step S17).

[0158] Next, the scene estimation unit 15 estimates the treatment scene observed by the endoscope 2 based on the movement vector V and the anatomical information (step S9). Specifically, if the direction of the movement vector V is consistent with the length direction of the stripping line L, the scene estimation unit 15 estimates the treatment scene as a stripping scene. On the other hand, if the direction of the movement vector V is inconsistent with the length direction of the stripping line L, the scene estimation unit 15 estimates the treatment scene as another scene.

[0159] Next, the offset parameter determination unit 13 determines the offset parameters based on the processing scene and the movement vector V (steps S10, S12, S18).

[0160] Specifically, in the case of a stripping scenario ("stripping scenario" in step S10), the offset parameter determination unit 13 calculates the offset parameter using the function F(V) or parameter table E for the stripping scenario (step S12). As a result, the target point T is offset from the center of the field of view F in a direction opposite to the moving direction of the treatment device 6 (step S7), the tip 6a of the treatment device 6 is offset from the center of the endoscopic image B in a direction opposite to the stripping direction, and the area C stripped by the treatment device 6 is positioned at the center of the endoscopic image B.

[0161] On the other hand, in other scenarios ("other scenarios" in step S10), the offset parameter determination unit 13 determines the offset parameter for positioning the target point T at the reference point O (step S18). As a result, the target point T is set at the center of the field of view F, and the front end 6a of the treatment device 6 is positioned at the center of the endoscopic image B.

[0162] Thus, according to this embodiment, the current treatment scenario can be estimated based on the anatomical information within the endoscopic image B and the movement vector V representing the movement of the treatment device 6. Furthermore, based on the treatment scenario, it is determined whether to shift the target point T from the reference point O, thereby enabling the front end 6a of the treatment device 6 within the endoscopic image B to be positioned appropriately for the current treatment scenario.

[0163] Specifically, in a dissection scenario where the surgeon moves the treatment instrument 6 along the dissection line L, the surgeon can easily observe the area C to be dissected because the area C to be dissected by the treatment instrument 6 is positioned at the center of the endoscopic image B.

[0164] When the scene is separated from other scenes, the target point T is set as the reference point O, and the tip 6a of the instrument 6 is positioned at the center within the endoscopic image B. This allows the surgeon to easily observe the tip 6a of the instrument 6.

[0165] In this embodiment, the type of treatment device 6 in the second embodiment can also be used as scene information. For example, if the treatment scene is estimated to be another scene based on the movement vector V and the anatomical information, the treatment scene can then be estimated based on the type of treatment device 6.

[0166] Furthermore, in this embodiment, the type and operating status of the treatment device 6 of the third embodiment can also be used as scene information. For example, if the treatment scene is estimated to be another scene based on the movement vector and anatomical information, the treatment scene can then be estimated based on the type and operating status of the treatment device 6.

[0167] (Fifth Implementation)

[0168] Next, the control device, endoscope system, and control method of the fifth embodiment of the present invention will be described.

[0169] This embodiment differs from the first to fourth embodiments in that it determines the offset parameter based on the operator's input. In this embodiment, structures that differ from the first to fourth embodiments will be described, while structures common to the first to fourth embodiments will be labeled with the same reference numerals and their descriptions will be omitted.

[0170] The endoscope system 10 of this embodiment includes a control device 1, an endoscope 2, a moving device 3, an endoscope processor 4, and a display device 5.

[0171] The control device 1 is similar to the first embodiment, and includes a position information acquisition unit 11, a scene information acquisition unit 12, an offset parameter determination unit 13, and a control unit 14.

[0172] User interface 1d can have any input device such as a mouse, keyboard or touch panel. The operator can use user interface 1d to input the position of the front end 6a of the handling device 6 to the control device 1.

[0173] For example, such as Figure 3A As shown, for the position of the front end 6a, input the offset angles φy and φz. The offset angle φy is the angle between the line segment connecting the positions of the front ends 2b and 6a of endoscope 2 in the XY plane and the optical axis A. The offset angle φz is the angle between the line segment connecting the positions of the front ends 2b and 6a of endoscope 2 in the XZ plane and the optical axis A. Based on the offset angle φy, determine the distance δy in the Y direction from the reference point O to the position of the front end 6a at each position in the X direction. Based on the offset angle φz, determine the distance δz in the Z direction from the reference point O to the position of the front end 6a at each position in the X direction.

[0174] The scene information acquisition unit 12 acquires the position of the front end 6a input by the user interface 1d as scene information.

[0175] The offset parameter determination unit 13 determines the offset parameters based on the position of the input front end 6a. For example, the offset parameter determination unit 13 calculates the three-dimensional position (D, δy, δz) determined by the distance D and the offset angles φy and φz as the offset parameters.

[0176] The control unit 14 sets the target point T as a three-dimensional position (D, δy, δz). Therefore, the position of the front end 6a input by the operator is set as the target point T.

[0177] Next, the control method executed by the control device 1 will be explained.

[0178] When switching to follow mode, the processor 1a of the control device 1 executes... Figure 16 The steps S1 to S20 shown cause the endoscope 2 to automatically follow the treatment device 6, which is the object being followed.

[0179] Similar to the first embodiment, by performing steps S1 to S3, the tip 2b of the endoscope 2 follows the tip 6a of the treatment device 6, so that the tip 6a of the treatment device 6 is positioned at the reference point O within the field of view F.

[0180] When the operator wants to move the position of the front end 6a from the center of the endoscope image B to another position according to the treatment scenario, the operator uses the user interface 1d to input the desired position of the front end 6a into the control device 1.

[0181] If the operator inputs the position of the tip 6a into the control device 1 ("Yes" in step S19), the scene information acquisition unit 12 acquires the position input by the operator (step S20), the offset parameter determination unit 13 determines the offset parameter based on the input position (step S6), and the control unit 14 sets the target point T at the position input by the operator (step S7). As a result, the tip 6a of the treatment instrument 6 in the endoscopic image B moves from the center to the target point T specified by the operator.

[0182] Thus, according to this embodiment, the target point T is shifted towards the input position of the front end 6a of the treatment device 6 by the operator's input. Therefore, the operator can set the target point T at any position suitable for the current treatment scenario at any time, and the front end 6a in the endoscopic image B can be shifted to any position at any time.

[0183] In this embodiment, such as Figure 17 As shown, the scene information acquisition unit 12 can also acquire the position of the front end 6a of the processing device 6 when the input of the adjustment of the indicated target point T is started as scene information.

[0184] In this case, the operator moves the tip 6a of the treatment device 6 to the desired position within the endoscopic image B and inputs an instruction to start adjustment to the control device 1 using the user interface 1d (step S21). In response to the input, the scene information acquisition unit 12 acquires the position of the tip 6a at the time the instruction was input (step S22). For example, the scene information acquisition unit 12 stores the endoscopic image B at the time the instruction was input and calculates the three-dimensional position of the tip 6a of the treatment device 6 relative to the tip 2b of the endoscope 2 based on the endoscopic image B.

[0185] If the operator inputs an instruction to end the adjustment of target point T ("Yes" in step S23), the adjustment of target point T ends, and target point T returns to reference point O (step S1).

[0186] In this embodiment, the scene information acquisition unit 12 acquires the position of the front end 6a of the treatment device 6 as scene information, but it may also acquire the position and orientation of the endoscope 2 as scene information instead. For example, the scene information acquisition unit 12 acquires information about the current position and orientation of the endoscope 2 from the moving device 3.

[0187] Figures 18A to 19B Examples of the position and orientation of endoscope 2 are shown. In laparoscopic surgery, the position and orientation of endoscope 2 vary depending on the site of the procedure and the procedure being performed, i.e., depending on the procedure scenario.

[0188] exist Figure 18A and Figure 19A In this process, endoscope 2 is inserted into the abdominal cavity through a hole H formed in the body wall, and the position and orientation of endoscope 2 are changed with the position of hole H as a fulcrum. The position and orientation of endoscope 2 are, for example, the position and orientation in a coordinate system fixed relative to the subject P with the fulcrum as the reference.

[0189] exist Figures 18A to 19B In a modified example, multiple offset parameters corresponding to the position and orientation of the endoscope 2 are preset, or set by the operator such as the surgeon during the operation. The offset parameter determination unit 13 selects the offset parameters corresponding to the position and orientation of the endoscope 2. Therefore, as... Figure 18B and Figure 19B As shown, a target point T is set at a position corresponding to the position and orientation of endoscope 2.

[0190] According to this structure, scene information can be obtained without the need for operator input.

[0191] In the above embodiments, the target point T is offset relative to the reference point O in a three-dimensional direction, but it can also be replaced by offsetting the target point T relative to the reference point O in a two-dimensional or one-dimensional direction.

[0192] For example, in one variation, the scene information acquisition unit 12 may detect a two-dimensional movement vector V along the YZ plane perpendicular to the optical axis A, and the offset parameter determination unit 13 may determine a two-dimensional offset parameter for offsetting the target point T relative to the reference point O in a direction parallel to the YZ plane. In other variations, the scene information acquisition unit 12 may detect a one-dimensional movement vector V in the Z direction parallel to the optical axis A, and the offset parameter determination unit 13 may determine a one-dimensional offset parameter for offsetting the target point T relative to the reference point O only in the X direction.

[0193] In the above embodiments, the reference point O is a point on the optical axis A of the field of view F, but the reference point can be set at any position within the field of view F, or it can be a point outside the optical axis A.

[0194] In the above embodiments, the target point T is a point within the field of view F, but it can also be a two-dimensional or three-dimensional region. In this case, the control device 1 can also start the endoscope 2 to follow the treatment device 6 when the front end 6a of the treatment device 6 extends to the outside of the region of the target point T.

[0195] For example, such as Figure 20A as well as Figure 20B As shown, the target point T is a cuboid region with defined dimensions dx, dy, and dz in the X, Y, and Z directions. In the first to fourth embodiments, the target point T is set as the region centered on the three-dimensional position calculated based on the movement vector V. In the fifth embodiment, the target point T is set as the region centered on the position input by the operator.

[0196] In the above embodiments, the position information acquisition unit 11 acquires position information from the endoscopic image B, but other means may be used instead to acquire position information.

[0197] In a modified example, the position information acquisition unit 11 may also acquire position information from a three-dimensional position measuring device disposed outside the body. For example, the three-dimensional position measuring device may also measure the position of the first mark installed on the endoscope 2 and the position of the second mark installed on the treatment device 6, and calculate the three-dimensional position of the front end 6a of the treatment device 6 relative to the front end 2b of the endoscope 2 based on the measured two positions.

[0198] In other variations, the position information acquisition unit 11 may also acquire position information from a three-dimensional scanning device installed on the endoscope 2. The three-dimensional scanning device may also determine the three-dimensional position of the treatment instrument 6 and biological tissues within the field of view F of the endoscope 2 by scanning with light or ultrasound.

[0199] In another variation, such as Figure 21A and Figure 21BAs shown, the position information acquisition unit 11 can also acquire position information of the endoscope 2 and the treatment device 6 from the first moving device 3 that moves the endoscope 2 and the second moving device 31 that moves the treatment device 6, respectively. The second moving device 31, like the first moving device 3, holds the treatment device 6 by a robotic arm or an electric retainer, and changes the position and orientation of the treatment device 6 in three dimensions under the control of the control device 101. The treatment device 6 can be as follows... Figure 21A As shown, it connects to the front end of the robotic arm and becomes an integral part of the robotic arm; it can also be like... Figure 21B As shown, it is separate from the robotic arm and is held by the robotic arm.

[0200] In the above embodiments, the position information acquisition unit 11 acquires position information including the three-dimensional position of the treatment device 6, but it may also acquire position information including the two-dimensional position of the treatment device 6 instead.

[0201] For example, the treatment device 6, the target point T, and the reference point O may be two-dimensional positions on the image plane (YZ plane) of the endoscopic image B. The control unit 14 causes the target point T to shift two-dimensionally relative to the reference point O in the Y and Z directions, and causes the endoscope 2 to follow the treatment device 6 two-dimensionally in the Y and Z directions.

[0202] Explanation of reference numerals in the attached figures

[0203] 1 Control device

[0204] 2 Endoscope

[0205] 3 mobile devices

[0206] 6. Treatment equipment

[0207] 6a frontend

[0208] 10 Endoscopic Systems

[0209] A-axis

[0210] B-Endoscopic images

[0211] F-view

[0212] L-shaped stripping line (anatomical information)

[0213] Target point T

[0214] O reference point

[0215] Patient P, subject

[0216] V-vector (scene information)

Claims

1. A control device that controls the movement of an endoscope to cause the endoscope to follow a treatment instrument, wherein, The control device has a processor. The processor obtains location information including the location of the treatment device. The processor acquires scene information, which is information associated with the treatment scene observed by the endoscope. The processor determines the offset parameter of the target point based on the scene information. This offset parameter determines the position of the target point relative to a predetermined reference point within the field of view of the endoscope. The processor sets the position of the target point relative to the reference point based on the offset parameter. The processor controls the movement of the endoscope based on the position of the target point and the position of the treatment device, thereby causing the endoscope to follow the treatment device to position the treatment device at the target point.

2. The control device according to claim 1, wherein, The processor obtains the movement vector of the treatment device as the scene information, and determines the offset parameter based on the movement vector.

3. The control device according to claim 2, wherein, The processor determines, based on the direction of the movement vector, the offset parameter at which the area in front of the treatment device in the direction of movement of the treatment device is positioned at the center of the field of view.

4. The control device according to claim 1, wherein, The processor estimates the handling scenario based on the scenario information, and determines the offset parameter based on the estimated handling scenario.

5. The control device according to claim 4, wherein, The processor obtains the type of the treatment device as the scene information, and estimates the treatment scene based on the type of the treatment device.

6. The control device according to claim 5, wherein, If the treatment scenario is estimated to be a stripping scenario, the processor determines the offset parameter of the area to be stripped by the treatment device at the center of the field of view.

7. The control device according to claim 5, wherein, If the disposal scenario is estimated to be an unfolding scenario, the processor sets the offset parameter that will configure the area unfolded by the disposal device at the center of the field of view.

8. The control device according to claim 4, wherein, The processor obtains the working status of the treatment device as the scene information, and estimates the treatment scene based on the working status of the treatment device.

9. The control device according to claim 4, wherein, The processor obtains anatomical information of biological tissues within the field of view as scene information, and estimates the treatment scenario based on the anatomical information.

10. The control device according to claim 1, wherein, The processor acquires the position information, which includes the three-dimensional position of the treatment device.

11. A control device that controls the movement of an endoscope to cause the endoscope to follow a treatment instrument, wherein, The control device has a processor. The processor obtains positional information from the image inside the subject, estimates the treatment scenario based on the image, and determines the position of the treatment instrument within the endoscope's field of view based on the treatment scenario. In the case of a dissection scenario, the target point located at the front end of the treatment device is shifted from the reference point at the center of the endoscope's field of view to the opposite direction of the movement of the treatment device, so as to position the area to be dissected by the treatment device at the center of the endoscopic image.

12. A control device that controls the movement of an endoscope to cause the endoscope to follow a treatment instrument, wherein, The control device has a processor. The processor obtains positional information from the image inside the subject, estimates the treatment scenario based on the image, and determines the position of the treatment instrument within the endoscope's field of view based on the treatment scenario. In the case of a dissection scenario, the target point located at the tip of the treatment instrument is shifted from a reference point at the center of the endoscope's field of view in a direction opposite to the movement direction of the treatment instrument, so as to position the area to be dissected by the treatment instrument at the center of the endoscopic image. In cases where the treatment scenario is a scenario other than a stripping scenario, the target point located at the front end of the treatment device is positioned at a reference point at the center of the endoscope's field of view, so that the front end of the treatment device is positioned at the center of the endoscope image.

13. An endoscope system, wherein, This endoscopic system has the following features: Endoscope; A moving device that moves the endoscope within the body being examined; and The control device according to any one of claims 1-12 controls the moving device to cause the endoscope to follow the treatment instrument.

14. The endoscopic system according to claim 13, wherein, The endoscope captures images of the body being examined by taking a photograph.

15. The endoscopic system according to claim 13, wherein, The mobile device includes a robotic arm that is connected to the endoscope and controls the position and orientation of the endoscope.

16. The endoscopic system according to claim 15, wherein, The processor calculates the position and orientation of the endoscope, which is positioned at the front end of the treatment device, and controls the robotic arm based on the calculated position and orientation of the endoscope.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the following steps: Obtain location information including the location of the treatment equipment; Obtain scene information, which is information related to the treatment scene observed by the endoscope; The offset parameter of the target point is determined based on the scene information. This offset parameter is a parameter that determines the position of the target point relative to a reference point defined within the field of view of the endoscope. The position of the target point relative to the reference point is set according to the offset parameter; as well as The movement of the endoscope is controlled according to the position of the target point and the position of the treatment device, thereby causing the endoscope to follow the treatment device to position the treatment device at the target point.

18. The computer-readable storage medium according to claim 17, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the movement vector of the treatment device as the scene information, and determining the offset parameter based on the movement vector.

19. The computer-readable storage medium according to claim 18, wherein, When the computer program is executed by the processor, it also performs the following steps: determining, based on the direction of the movement vector, the offset parameter at which the area in front of the treatment device in the direction of movement of the treatment device is positioned at the center of the field of view.

20. The computer-readable storage medium of claim 17, wherein, When the computer program is executed by the processor, it also performs the following steps: estimating the handling scenario based on the scenario information, and determining the offset parameter based on the estimated handling scenario.

21. The computer-readable storage medium according to claim 20, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the type of the treatment device as the scenario information, and estimating the treatment scenario based on the type of the treatment device.

22. The computer-readable storage medium according to claim 21, wherein, When the computer program is executed by the processor, it also performs the following steps: if the treatment scenario is estimated to be a stripping scenario, it determines the offset parameter at which the area to be stripped by the treatment device is positioned at the center of the field of view.

23. The computer-readable storage medium according to claim 21, wherein, When the computer program is executed by the processor, it also performs the following steps: if the disposal scenario is estimated to be an unfolding scenario, setting the offset parameter at the center of the field of view of the area to be unfolded by the disposal device.

24. The computer-readable storage medium of claim 20, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the working state of the treatment device as the scene information, and estimating the treatment scene based on the working state of the treatment device.

25. The computer-readable storage medium of claim 20, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining anatomical information of biological tissues within the field of view as the scene information, and estimating the treatment scene based on the anatomical information.

26. The computer-readable storage medium according to claim 17, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the position information including the three-dimensional position of the treatment device.

27. A computer program product comprising a computer program that, when executed by a processor, performs the following steps: Obtain location information including the location of the treatment equipment; Obtain scene information, which is information related to the treatment scene observed by the endoscope; The offset parameter of the target point is determined based on the scene information. This offset parameter is a parameter that determines the position of the target point relative to a reference point defined within the field of view of the endoscope. The position of the target point relative to the reference point is set according to the offset parameter; as well as The movement of the endoscope is controlled according to the position of the target point and the position of the treatment device, thereby causing the endoscope to follow the treatment device to position the treatment device at the target point.

28. The computer program product according to claim 27, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the movement vector of the treatment device as the scene information, and determining the offset parameter based on the movement vector.

29. The computer program product according to claim 28, wherein, When the computer program is executed by the processor, it also performs the following steps: determining, based on the direction of the movement vector, the offset parameter at which the area in front of the treatment device in the direction of movement of the treatment device is positioned at the center of the field of view.

30. The computer program product according to claim 27, wherein, When the computer program is executed by the processor, it also performs the following steps: estimating the handling scenario based on the scenario information, and determining the offset parameter based on the estimated handling scenario.

31. The computer program product according to claim 30, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the type of the treatment device as the scenario information, and estimating the treatment scenario based on the type of the treatment device.

32. The computer program product according to claim 31, wherein, When the computer program is executed by the processor, it also performs the following steps: if the treatment scenario is estimated to be a stripping scenario, it determines the offset parameter at which the area to be stripped by the treatment device is positioned at the center of the field of view.

33. The computer program product according to claim 31, wherein, When the computer program is executed by the processor, it also performs the following steps: if the disposal scenario is estimated to be an unfolding scenario, setting the offset parameter at the center of the field of view of the area to be unfolded by the disposal device.

34. The computer program product according to claim 30, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the working state of the treatment device as the scene information, and estimating the treatment scene based on the working state of the treatment device.

35. The computer program product according to claim 30, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining anatomical information of biological tissues within the field of view as the scene information, and estimating the treatment scene based on the anatomical information.

36. The computer program product according to claim 27, wherein, When the computer program is executed by the processor, it also performs the following steps: obtaining the position information including the three-dimensional position of the treatment device.

Images