Medical system, control device, control method, and control program

Abstract

To provide a control device, a control method, and a control program that enable continuous observation by avoiding dirt even when dirt adheres to a distal-end lens.SOLUTION: An endoscope acquires an image D including a target of interest, and includes at least one processor. The processor acquires the position of the target of interest within a visual field of the endoscope, calculates the position of dirt on the endoscope within the visual field, sets a target region B at a position different from the position of dirt, and controls the endoscope so as to follow the target of interest such that the target of interest is arranged within the target region, on the basis of the position of the target region and the position of the target of interest.SELECTED DRAWING: Figure 7A

Description

[Technical Field]

[0001] The present invention relates to a medical system, a control device, a control method, and a control program. [Background technology]

[0002] Conventionally, a medical system is known in which an image including an object is acquired by an endoscope and the endoscope is made to track a target of interest, such as a treatment tool, using a predetermined three-dimensional area set within the field of view of the endoscope (see, for example, Patent Document 1). Also known is a technology that detects a treatment scene and offsets the position of the predetermined three-dimensional area to make it easier to see the site of dissection, thereby making the endoscope track the target of interest (see, for example, Patent Document 2). Furthermore, a technology is known that analyzes image information acquired before and after rotating the tip lens of a surgical camera to detect dirt adhering to the lens (see, for example, Patent Document 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2022 / 54428 [Patent Document 2] International Publication No. 2022 / 54882 [Patent Document 3] Japanese Patent Application Laid-Open No. 2013-197652 Summary of the Invention [Problem to be solved by the invention]

[0004] When performing a procedure while tracking an object of interest with an endoscope, debris generated during the procedure may adhere to the lens at the tip of the endoscope, reducing visibility of the object of interest. When the visibility of the object of interest is reduced due to dirt, it is desirable to be able to continue the procedure by tracking the object of interest with the endoscope while minimizing the effects of the dirt. [Means for solving the problem]

[0005] One aspect of the present invention is a control device that controls the movement of an endoscope, wherein the endoscope acquires an image including an object of interest and is equipped with at least one processor, which acquires the position of the object of interest within the field of view of the endoscope, calculates the position of dirt on the endoscope within the field of view, sets a target area at a position different from the position of the dirt, and, based on the position of the target area and the position of the object of interest, causes the endoscope to track the object of interest so that the object of interest is positioned within the target area.

[0006] Another aspect of the present invention is a medical system comprising an endoscope that acquires an image including an object of interest, a movement device that moves the endoscope, and a control device that controls the movement device based on the position of the object of interest in the image, wherein the control device comprises at least one processor that acquires position information including the position of the object of interest within the field of view of the endoscope, calculates the position of dirt on the endoscope within the field of view, sets a target area at a position different from the position of the dirt, and causes the endoscope to track the object of interest so that the object of interest is positioned within the target area based on the position of the target area and the position of the object of interest.

[0007] Another aspect of the present invention is a control method including a processor acquiring the position of an object of interest within a field of view of an endoscope, calculating the position of dirt on the endoscope within the field of view, setting a target area at a position different from the position of the dirt, and, based on the position of the target area and the position of the object of interest, causing the endoscope to track the object of interest so that the object of interest is positioned within the target area.

[0008] Another aspect of the present invention is a control program that causes a processor to acquire the position of an object of interest within the field of view of an endoscope, calculate the position of dirt on the endoscope within the field of view, set a target area at a position different from the position of the dirt, and, based on the position of the target area and the position of the object of interest, cause the endoscope to track the object of interest so that the object of interest is positioned within the target area. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an external view of a medical system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram of the medical system of FIG. 1. [Figure 3] FIG. 1 is a diagram showing a three-dimensional target region set within the field of view of an endoscope. [Figure 4] 4 is an endoscopic image showing an example of a cross section of the target area of ​​FIG. 3. [Figure 5A] 4 is a diagram illustrating the size of the target region on the endoscopic image at the depth X1 in FIG. 3. FIG. [Figure 5B] 4 is a diagram illustrating the size of the target region on the endoscopic image at the depth X2 in FIG. 3. FIG. [Figure 5C] 4 is a diagram illustrating the size of the target region on the endoscopic image at a depth X3 in FIG. 3. FIG. [Figure 6A] FIG. 10 is a diagram showing an example in which the positions of the centers of gravity of two stains that are close to each other are calculated as the positions of the stains. [Figure 6B] FIG. 10 is a diagram showing an example in which the position of the center of gravity of two stains that are close to each other is calculated as the position of each stain. [Figure 7A] FIG. 10 is a diagram showing an example of a method for setting a new target area when there is one stain in the field of view. [Figure 7B] 10A and 10B are diagrams illustrating an example of a method for setting a new target area when there are multiple stains within the field of view. [Figure 8] 2 is a flowchart of a control method executed by the control device of the medical system of FIG. 1. [Figure 9A]10A and 10B are diagrams illustrating another example of a method for setting a new target area when there are multiple stains within the field of view. [Figure 9B] FIG. 10 is a diagram showing yet another example of a method for setting a new target area when there are multiple stains within the field of view. [Figure 10] 10A and 10B are diagrams showing an example of movement of the endoscope when the field of view of the endoscope is set to an upward field of view. [Figure 11A] FIG. 10 is a diagram showing an example of an endoscopic image when the field of view of the endoscope is set to an upward field of view. [Figure 11B] FIG. 10 is a diagram showing an example of an endoscopic image when the field of view of the endoscope is set to a downward field of view. [Figure 12A] FIG. 10 is a diagram showing an example of an endoscopic image in which the field of view of the endoscope is set to an upward field of view, there is no dirt, and the target area is positioned in the center of the upper half. [Figure 12B] 12B is a diagram showing an example of an endoscopic image in which dirt is attached in FIG. 12A. FIG. [Figure 12C] 12C is a diagram showing an example of an endoscopic image illustrating the movement of the target area due to the dirt in FIG. 12B. FIG. [Figure 12D] 12D is a diagram showing an example of an endoscopic image when the endoscope is moved so that the tip of the treatment tool is positioned in the new target region in FIG. 12C. FIG. [Figure 13A] FIG. 10 is a diagram showing an example of an endoscopic image in which the field of view of the endoscope is set to the left half, there is no dirt, and the target area is positioned in the center of the left half. [Figure 13B] 13B is a diagram showing an example of an endoscopic image in which dirt is attached in FIG. 13A. FIG. [Figure 13C] 13C is a diagram showing an example of an endoscopic image illustrating the movement of a target area due to the dirt in FIG. 13B. FIG. [Figure 13D] FIG. 13D is a diagram showing an example of an endoscopic image when the endoscope is moved so that the tip of the treatment tool is positioned in the new target region in FIG. 13C. [Figure 14] FIG. 10 is a diagram showing an example of an endoscopic image in which a plurality of candidate regions are set in advance. [Figure 15]This is a perspective view showing an example of the distribution of weights on an endoscopic image to set a new target area using the gradient of weights assigned to dirt, etc., in a manner similar to the potential gradient in the population potential method. DETAILED DESCRIPTION OF THE INVENTION

[0010] A medical system 10, a control device 4, a control method, and a control program according to one embodiment of the present invention will be described with reference to the drawings. As shown in Figure 1, the medical system 10 of this embodiment includes an endoscope 1 and a treatment tool 2 to be inserted into the body of a patient P, a moving device 3 that holds the endoscope 1 and moves it within the body, a control device 4 that is connected to the endoscope 1 and the moving device 3 and controls the moving device 3, and a display device 5 that displays an endoscopic image D.

[0011] The endoscope 1 is, for example, a rigid endoscope having a tip lens, and is equipped with an imaging unit 1a (see FIG. 2) that has an image sensor and acquires an endoscopic image D (see FIG. 4). The endoscope 1 acquires an endoscopic image D including a tip 2a (hereinafter also referred to as a target object) of a treatment tool 2 using the imaging unit 1a, and transmits the endoscopic image D to the control device 4. The imaging unit 1a is, for example, a three-dimensional camera provided at the tip of the endoscope 1, and acquires a stereo image including information on the three-dimensional position of the tip 2a of the treatment tool 2 as the endoscopic image D.

[0012] The moving device 3 is a robot arm having multiple joints 3a, and holds the base end of the endoscope 1 at the tip of the robot arm. In one example, as shown in Fig. 3, the moving device 3 has three degrees of freedom of movement: linear movement back and forth along the X axis, rotation around the Y axis (pitch), and rotation around the Z axis (yaw), and preferably also has a degree of freedom of movement of rotation around the X axis (roll). The X axis is an axis on the same straight line as the optical axis A of the endoscope 1, and the Y axis and Z axis are axes that are perpendicular to the optical axis A and extend in directions corresponding to the horizontal and vertical directions of the endoscopic image D, respectively.

[0013] As shown in FIG. 2, the control device 4 includes at least one processor 4a such as a central processing unit, a memory 4b, a storage unit 4c, an input interface 4d, an output interface 4e, and a network interface 4f. The endoscopic images D transmitted from the endoscope 1 are sequentially input to the control device 4 via the input interface 4d, sequentially output to the display device 5 via the output interface 4e, and displayed on the display device 5. An operator such as a surgeon operates the treatment tool 2 inserted inside the body while observing the endoscopic images D displayed on the display device 5, and uses the treatment tool 2 to treat the affected area inside the body.

[0014] The memory 4b is made up of a volatile storage device such as a RAM (random access memory), and is used as a working area for the processor 4a. The storage unit 4c is a computer-readable non-transitory storage medium, such as a known magnetic disk, optical disk, flash memory, or ROM (read-only memory).

[0015] The storage unit 4c stores programs and data necessary for the processor 4a to execute processes. The functions of the control device 4, which will be described later, are realized by loading programs into the memory 4b and executing them with the processor 4a. Some of the functions of the control device 4 may be realized by dedicated logic circuits or the like.

[0016] The control device 4 has a manual mode and a tracking mode. The manual mode is a mode in which an operator such as a surgeon manually operates the endoscope 1, and the tracking mode is a mode in which the control device 4 automatically makes the endoscope 1 track the distal end 2 a of the treatment tool 2.

[0017] The control device 4 switches between manual mode and tracking mode based on instructions from the operator. For example, the control device 4 is equipped with artificial intelligence capable of recognizing human voice, and switches to manual mode when it recognizes the voice for "manual mode," and switches to tracking mode when it recognizes the voice for "tracking mode." The control device 4 may switch between manual mode and tracking mode in accordance with the on / off of a manual operation switch (not shown) provided on the endoscope 1.

[0018] In the manual mode, for example, an operator such as a surgeon can remotely control the moving device 3 by operating an operation device (not shown) connected to the control device 4. In the tracking mode, the control device 4 controls the moving device 3 based on the three-dimensional position of the tip 2a of the treatment tool 2, thereby causing the endoscope 1 to three-dimensionally track the tip 2a of the treatment tool 2 so that the tip 2a of the treatment tool 2 moves toward the center of the target area B.

[0019] Specifically, the control device 4 recognizes the treatment tool 2 in the endoscopic image D and calculates the three-dimensional position of the tip 2a of the treatment tool 2 using the endoscopic image D. Next, the control device 4 operates each joint 3a so that the optical axis A of the endoscope 1 moves in a direction intersecting the optical axis A toward the tip 2a of the treatment tool 2 and the tip of the endoscope 1 moves in the depth direction along the optical axis A toward a position a predetermined observation distance away from the tip 2a.

[0020] The target area B is a predetermined three-dimensional area set within the field of view F of the endoscope 1 and has dimensions in the mutually orthogonal X, Y, and Z directions. The X direction is the depth direction parallel to the optical axis A of the endoscope 1. The Y and Z directions are directions orthogonal to the optical axis A and parallel to the horizontal and vertical directions of the endoscopic image D, respectively.

[0021] As shown in Figures 3 and 4, target area B is located at a position away from the tip of the endoscope 1 in the X direction, and is set within a portion of the field of view F in the X direction. Furthermore, target area B has a three-dimensional shape whose cross section becomes smaller as it approaches the tip of the endoscope 1. In an initial state where no particular designation is made, target area B on the endoscopic image D is a central area including the center of the endoscopic image D. The shape of the cross section of target area B perpendicular to the optical axis A may be rectangular, circular, or elliptical, or may be other shapes such as a polygon. Target area B may be displayed superimposed on the endoscopic image D, or may not be displayed.

[0022] In one example, the cross-sectional shape of the target area B is similar to the shape of the endoscopic image D. For example, if the endoscopic image D is rectangular, the cross-sectional shape of the target area B is also rectangular. Since the target area B displayed on the endoscopic image D may interfere with the observation of the endoscopic image D, it is preferable that the target area B is not displayed. If the target area B has a similar shape to the endoscopic image D, the surgeon can easily recognize the position of the hidden target area B.

[0023] Generally, the field of view F of the endoscope 1 is cone-shaped with an apex at or near the tip of the endoscope 1. It is preferable that the target region B is frustum-shaped with a common apex with the field of view F of the endoscope 1. With such a target region B, as shown in Figures 5A to 5C, the apparent size and position of the target region B on the endoscopic image D are constant regardless of the positions X1, X2, and X3 in the X direction.

[0024] The processor 4a processes the endoscopic image D sent from the endoscope 1, detects dirt adhering to the distal lens of the endoscope 1 using a known method, and calculates the position of the detected dirt in the YZ direction. The position of the dirt is the pixel position (coordinates) where the detected dirt is located on the endoscopic image D, and is calculated, for example, according to the following dirt detection rule.

[0025] <Dirt detection rules> (a) If the stain covers a single pixel or a range of multiple consecutive pixels and the number of pixels (area) is equal to or less than a first threshold, the stain is deemed not to exist. (b) When the stain covers a range of multiple consecutive pixels and the number of pixels is greater than the first threshold, the position of the stain is the pixel position that is the center of gravity of the range of multiple pixels.

[0026] (c) As shown in Figure 6A, when a stain spans multiple discontinuous pixel ranges, if the distance d between each stain range H1, H2 is equal to or less than a second threshold, it is considered to be a single stain range.Then, the pixel position that is the center of gravity OH (representative value) according to the area ratio of those stain ranges H1, H2 is determined to be the stain position.In the figure, for simplicity, stains are shown as circles, but the detected stains can be any shape.

[0027] (d) As shown in Figure 6B, if the dirt extends over multiple non-contiguous pixel ranges and the distance d between each dirt range H1, H2 is greater than the second threshold, the pixel position that is the center of gravity OH of each dirt range H1, H2 is determined to be the position of the dirt.

[0028] The processor 4a calculates the position of the tip 2a of the treatment tool 2 based on the endoscopic image D sent from the endoscope 1. Furthermore, the processor 4a sets the position of a new target region B based on the calculated position OH of the stain. The position of the new target area B is set, for example, in accordance with the following target position setting rules.

[0029] <Target position setting rules> (e) When only a single stain area OH is detected, the processor 4a calculates the lengths of four straight lines L1 to L4 connecting the stain position, which is the center of gravity position OH of the stain area H, to the four corners (edges of the field of view) of the endoscopic image D, as shown in Fig. 7A. Then, the processor 4a sets the center position LA of the longest straight line (L2 in this case) of the four straight lines L1 to L4 as the position (center position) of a new target area B.

[0030] (f) When multiple stain areas H are detected, the processor 4a calculates the center positions LA1, LA2 of the longest straight lines LMAX, LMAX2 for each stain area H1, H2 in the same manner as above, as shown in Fig. 7B. The processor 4a then sets the median value LO of the center positions LA1, LA2 of all the longest straight lines LMAX, LMAX2 as the position of a new target area B.

[0031] The processor 4a calculates the drive amount of each joint 3b of the moving device 3 based on the position of the set target area B and the calculated position of the tip 2a of the treatment tool 2, and operates the moving device 3. As a result, the target area B is set at a position different from the positions of the stains H, H1, and H2, and the endoscope 1 is moved following the treatment tool 2 so that the detected tip 2a of the treatment tool 2 is positioned in the set target area B.

[0032] Next, the operation of the medical system 10 according to this embodiment will be described below with reference to the drawings. The surgeon performs treatment by operating the treatment tool 2 inserted into the body of the patient P while observing the endoscopic image D displayed on the display device 5. During treatment, the surgeon or other operator switches from manual mode to follow-up mode or from follow-up mode to manual mode, for example, by voice.

[0033] As shown in FIG. 8, when the mode is switched to the follow-up mode in step S1, the processor executes the control method of steps S2 to S10 to control the mobile device 3 in the follow-up mode. In the control method according to this embodiment, first, the processor 4a calculates the position of the distal end 2a of the treatment tool based on the endoscopic image D (step S2). Next, the processor 4a performs processing to detect dirt on the distal end lens of the endoscope 1 based on the endoscopic image D (step S3) and determines whether or not dirt is present (step S4). If it is determined that dirt is present on the distal end lens, the processor 4a calculates the position of the dirt (step S5).

[0034] Next, the processor 4a calculates the position of a new target area B based on the position of the stain and in accordance with the target position setting rule (step S6), and sets the target area B at the calculated position (step S7). Then, the moving device 3 is operated so that the tip 2a of the treatment tool 2 is positioned within the set target area B (step S8). It is determined whether the tip 2a of the treatment tool 2 is positioned within the new target area B (step S9), and if not, the process from step S8 is repeated.

[0035] Then, it is determined whether or not an operator such as a surgeon has switched from the follow-up mode to the manual mode, for example, by voice (step S10), and if the mode has not been switched, the process from step S2 is repeated. Thus, according to this embodiment, the new target area B is set in an uncontaminated area within the field of view, so even if dirt adheres to the distal lens, it is possible to avoid the dirt and capture the distal end 2 a of the treatment tool 2. This has the advantage of ensuring a wide field of view around the distal end 2 a of the treatment tool 2 that is not obstructed by dirt, making it easier for the surgeon to continue the treatment.

[0036] Furthermore, according to this embodiment, if the stain covers a single pixel or a range of multiple consecutive pixels and the number of pixels is equal to or less than the first threshold, the stain is deemed not to be present. This prevents unnecessary processing of moving the target area B due to small stains that do not obstruct the field of view, and also prevents unnecessary movement of the target area B, making treatment easier.

[0037] In addition, if the stain covers a range of multiple consecutive pixels and the number of pixels is greater than the first threshold, the stain's position is determined to be the pixel position that is the center of gravity of the range of multiple pixels, so that a relatively large block of stain can be processed as a single stain. Furthermore, if multiple stain ranges are close to each other, they are considered to be a single stain, which simplifies processing.

[0038] Furthermore, according to this embodiment, the center position of the new target area B is set to the center position LA of the longest straight line LMAX among the straight lines L1 to L4 connecting the dirt detected within the field of view to the four corners of the endoscopic image D. Furthermore, if multiple dirt areas are detected, the median value LO of the center positions LA1, LA2 of the longest straight lines LMAX1, LMAX2 for each dirt area is set as the position of the new target area B. This makes it possible to position the new target area B in an unsoiled area within the field of view using a simple method.

[0039] In this embodiment, the center position of the new target area B is set at the midpoint of the longest straight line LMAX among the straight lines connecting the dirt detected in the field of view with the four corners of the endoscopic image D. Alternatively, the new target area B may be set at a position on the straight line LMAX other than the midpoint of the straight line LMAX. Furthermore, the new target area B is not limited to the longest straight line LMAX, and may be set on any straight line of a predetermined length or more.

[0040] Furthermore, as shown in FIG. 9A, when there are multiple stains H1, H2, and H3 that are relatively far apart, the center position of a new target region B may be set at the center of a circle that passes through the positions of the stains H1, H2, and H3. Instead of a circle, the target region B may be set at the center of an ellipse or, as shown in FIG. 9B, a polygon that includes one of the corners of the endoscopic image D. This allows the new target region B to be set in a relatively wide area where the field of view is not obstructed by stains, allowing treatment to be continued while observing the distal end 2a of the treatment tool 2 over a wider field of view. Note that, while the example described here uses stains H1, H2, and H3, the number of stains is not limited to three. The target region B can be set as in this method regardless of the number of stains present.

[0041] In addition, in order to prevent the new target region B from going beyond the endoscopic image D, the coordinates that the calculated center position of the new target region B can take may be limited to a predetermined range in the center of the endoscopic image D.

[0042] 3, the field of view F of the endoscope 1 is cone-shaped with its apex at or near the tip of the endoscope 1, so the upper half of the field of view F is a field of view looking up at the observation target, such as tissue to be treated, and the lower half of the field of view F is a field of view looking down at the observation target. Depending on the type of treatment, it may be desirable to perform it with a field of view looking up or a field of view looking down.

[0043] That is, when treatment is desired in an upward field of view, the processor 4a operates the movement device 3 to move the endoscope 1 so as to capture the observation target in the upper part of the field of view, based on instructions from an operator such as a surgeon, as shown in Fig. 10. According to Fig. 10, even if the treatment tool 2 is hidden by tissue when the target area is positioned in the center of the field of view, it is possible to observe the treatment tool 2 while avoiding the tissue by setting the field of view to an upward field of view.

[0044] On the other hand, when treatment is desired from a top-down view, the processor 4a operates the movement device 3 to move the endoscope 1 so as to capture the observation target at the bottom of the view field, based on instructions from an operator such as a surgeon. As a result, the target area B is positioned at the center of the upper half of the view field F, as shown in Fig. 11A, and when treatment is desired from a top-down view, the target area B is positioned at the center of the lower half of the view field F, as shown in Fig. 11B, for example.

[0045] In these cases, when moving the target area B in response to detection of dirt, it is preferable to set the new target area B so that the field of view looking up or down continues. Therefore, in order to deal with these cases, for example, a settable area that can be set as the target area B may be stored in the memory 4b, and the processor 4a may set a new target area B within the settable area.

[0046] That is, when treatment is performed using an upward field of view, the upper half of the field of view is stored in memory 4b as a settable area, and a new target area B is limited to the upper half of the field of view, as shown in Fig. 12A. In this state, if dirt is detected as shown in Fig. 12B, target area B is set to a position in the upper half of the field of view, mainly moved left and right (to the right in the figure), as shown in Fig. 12C. Then, as shown in Fig. 12D, endoscope 1 is moved so that tip 2a of treatment tool 2 is positioned within the newly set target area B.

[0047] Similarly, when performing treatment using a downward view, the lower half of the field of view is stored in memory as the settable area, and the new target area B is limited to the lower half of the field of view. Therefore, the target area B is set to move mainly left and right in the lower half of the field of view.

[0048] 13A, when performing a procedure such as incising tissue while moving a treatment tool from left to right within an endoscopic image D, it is necessary to ensure a wide field of view at the incision destination (area E) ahead in the direction of movement of the treatment tool 2 (to the right in the figure). In such a case, the processor 4a operates the movement device 3 to move the endoscope 1 in response to an instruction from the operator so as to capture the tip 2a of the treatment tool 2 on the left side of the field of view, and the target area B is positioned in the center of the left half of the field of view.

[0049] Therefore, when moving target area B in response to detection of dirt, the left half of the field of view is stored in memory 4b as a settable area, and the new target area B is also set to move mainly up and down within the left half of the field of view. That is, when dirt is detected as shown in Fig. 13B, target area B is set to a position in the left half of the field of view that has moved mainly up and down (upward in the figure), as shown in Fig. 13C. Then, as shown in Fig. 13D, endoscope 1 is moved so that tip 2a of treatment tool 2 is positioned within the newly set target area B.

[0050] Similarly, in a procedure that requires a wide field of view to the left of the field of view, the right half of the field of view is stored in memory 4b as a settable area, and the new target area B is also set to move mainly up and down within the right half of the field of view.

[0051] By doing so, even if dirt adheres to the tip lens, the tip 2a of the treatment tool 2 can be captured while avoiding the dirt, while ensuring the same conditions of the field of view. The same effect can be achieved by storing the settable area and restricting the new target area B to within the settable area, but restricting the direction of movement of the target area B in response to the detection of dirt. Also, the settable area is not limited to the upper half, lower half, left half, or right half of the field of view, but may be set to any of the fields of view divided into three or more parts horizontally or vertically.

[0052] Alternatively, instead of storing a settable area, a plurality of candidate areas Ba that can be set as the target area B within the field of view may be stored in advance in memory 4b, and processor 4a may set one of the candidate areas Ba that does not contain dirt as a new target area B. In the example shown in Fig. 14, as indicated by the nine black circles, the center positions of the candidate areas Ba are set at nine equally spaced locations within the field of view: three on the left and right and three on the top and bottom.

[0053] Since a new target region B is selected from the candidate regions Ba stored in advance, it is possible to prevent part of the target region B from extending outside the endoscopic image D, thereby maintaining ease of observation. Although an example has been shown in which the candidate regions Ba do not overlap each other, the candidate regions Ba may instead overlap each other or may be arranged with a gap between them.

[0054] Furthermore, similar to the potential gradient in the population potential method, a weight may be assigned to each position within the endoscopic image D, and a weight gradient may be set such that the smaller the weight, the further away from the stain, thereby setting a new target region in the region with the smallest weight. For example, as shown in FIG. 15, by assigning weights that are highest at the calculated stain position and decreasing toward its periphery, a weight gradient that decreases the further away from the stain can be set. The thin lines in FIG. 15 indicate weight contours. Furthermore, for example, in the case of the above-mentioned upward field of view, by setting a weight gradient that decreases toward the top in the lower half of the field of view, it is possible to prevent a new target region from being set in the lower half of the field of view, in the same way as with stains.

[0055] Furthermore, after moving the target area B, the target area B may be reset to the center of the field of view when the endoscope 1 is removed from the body of the patient P or when the removed endoscope 1 is reinserted into the body of the patient P. If the field of view was offset to provide an upward field of view, a downward field of view, or a field of view that is wider in the left and right directions immediately before the endoscope 1 is removed from the body of the patient P, the position of the reset target area B may be returned to the original offset direction.

[0056] In the above embodiment, the endoscope 1 acquires a three-dimensional stereo image as the endoscopic image D, but instead, a two-dimensional endoscopic image D may be acquired. In this case, for example, the position of the tip 2 a of the treatment tool 2 in the X direction may be measured by another distance measuring means such as a distance sensor provided at the tip of the endoscope 1.

[0057] In the above embodiment, the target of interest tracked by the endoscope 1 is the treatment tool 2, but the target of interest is not limited to this and may be any object that appears in the endoscopic image D during surgery. For example, the target of interest may be a lesion, an organ, a blood vessel, a marker, medical material such as gauze, or a medical instrument other than the treatment tool 2.

[0058] Although the embodiments of the present invention and their modifications have been described in detail above with reference to the drawings, the specific configuration of the present invention is not limited to the above-described embodiments and modifications, and various design modifications are possible within the scope of the gist of the present invention. Furthermore, the components shown in the above-described embodiments and modifications can be combined as appropriate. For example, the subject may be a lumen other than the large intestine, or may be an organ other than a lumen that can be the target of endoscopic examination. The region of interest may be set according to the subject. [Explanation of symbols]

[0059] 1. Endoscope 2 Treatment tools 2a Tip (target) 3. Mobile Devices 4. Control device 4a processor 4b Memory 10. Healthcare System D Endoscopic image (image)

Claims

1. A control device for controlling movement of an endoscope, the endoscope acquires an image including an object of interest; at least one processor; The processor: acquiring the position of the target object within the field of view of the endoscope; Calculating the position of the contamination of the endoscope within the field of view; A target area is set at a position different from the position of the stain, a control device that causes the endoscope to track the target object of interest so that the target object of interest is positioned within the target area based on the position of the target area and the position of the target object of interest.

2. the processor: The control device according to claim 1 , wherein the center of the target area is set on a straight line connecting the position of the stain and an edge of the field of view.

3. The control device of claim 2 , wherein the edge of the field of view is the edge farthest from the location of the contamination.

4. the processor: Based on the distance between the plurality of stains present in the field of view, If the distance is equal to or less than a predetermined threshold, a representative position of the plurality of stains is calculated as the position of the stain; The control device according to claim 1 , wherein when the distance is greater than the threshold value, the control device calculates the positions of the plurality of stains as the positions of the stains.

5. the processor: The control device according to claim 4 , wherein when the positions of the plurality of stains are calculated, the center of the target area is set to the median of the centers of a plurality of straight lines connecting the positions of the respective stains to the edge of the field of view.

6. the processor: The control device according to claim 4 , wherein the center of the target area is set at the centroid position of a polygon, circle, or ellipse that passes through the positions of the plurality of stains.

7. the processor: Calculating the area of ​​the stain; The control device according to claim 1 , wherein the position of the stain is calculated only when the area of ​​the stain is greater than a predetermined threshold.

8. at least one memory; The memory stores a settable area that can be set as the target area within the field of view, The control device according to claim 1 , wherein the processor sets the target area within the settable area.

9. at least one memory; the memory stores a plurality of candidate areas that can be set as the target area within the field of view; The control device according to claim 1 , wherein the processor sets the target area to any of the candidate areas that does not include the calculated location of the stain.

10. The control device according to claim 1, wherein the processor creates a weight gradient by assigning weights that decrease from the calculated position of the dirt toward the periphery within the field of view, and sets the center of the target area at the position where the weight is lowest.

11. an endoscope for acquiring an image including a target of interest; a moving device for moving the endoscope; a control device that controls the moving device based on the position of the target object in the image; Equipped with The control device at least one processor; The processor: acquiring position information including the position of the target within the field of view of the endoscope; Calculating the position of the contamination of the endoscope within the field of view; A target area is set at a position different from the position of the stain, and a medical system that causes the endoscope to track the target object of interest so that the target object of interest is positioned within the target area based on the position of the target area and the position of the target object of interest.

12. The processor: Obtaining the location of an object of interest within the field of view of the endoscope; Calculating the position of the contamination of the endoscope within the field of view; Setting a target area at a position different from the position of the stain; and causing the endoscope to track the target of interest so that the target of interest is positioned within the target area based on the position of the target area and the position of the target of interest.

13. Obtaining the location of an object of interest within the field of view of the endoscope; Calculating the position of the contamination of the endoscope within the field of view; Setting a target area at a position different from the position of the stain; and causing the endoscope to track the target of interest so that the target of interest is positioned within the target area, based on the position of the target area and the position of the target of interest.

Images