Medical systems, control methods, and control programs

Abstract

This system provides a medical system that can automatically adjust to an appropriate and stable endoscopic view without user intervention. [Solution] The medical system comprises an endoscope that acquires images including treatment instruments, a mobile device that moves the endoscope, a processor that controls the mobile device, and a storage unit that stores a plurality of control parameters, including known field-of-view parameters for each type of treatment instrument to be recognized. The processor recognizes treatment instruments in the images acquired by the endoscope, determines whether the recognized treatment instrument is the object to be recognized, and if it is determined that the treatment instrument is the object to be recognized, it controls the mobile device based on the known field-of-view parameter corresponding to the type of treatment instrument among the plurality of control parameters stored in the storage unit. If it is determined that the treatment instrument is not the object to be recognized, it controls the mobile device based on at least one predetermined control parameter among the plurality of control parameters stored in the storage unit.

Description

Technical Field

[0001] The present invention relates to a medical system, a control method, and a control program.

Background Art

[0002] Conventionally, in laparoscopic surgery and the like, it has been desired to obtain an appropriate endoscopic view such as a close-up view or an overhead view according to the procedure, and a medical system capable of adjusting the view of the endoscope is known (see, for example, Patent Document 1). The medical system described in Patent Document 1 switches the view of the endoscope according to the type of the treatment instrument by recognizing the type of the treatment instrument.

[0003] Also, a medical image processing device that corrects a recognition result according to the reliability of the recognition result of a subject in an image acquired by an endoscope is known (see, for example, Patent Document 2). The medical image processing device described in Patent Document 2 switches a plurality of reception modes for receiving correction of the recognition result by the user based on the reliability of the recognition result, and displays the recognition result corrected according to the reception mode.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the medical system described in Patent Document 1 has problems such as incorrect recognition of the type of treatment instrument, resulting in the setting of inappropriate parameters for the field of view, or frequent switching of the endoscopic field of view settings due to the recognition of multiple types as candidate treatment instruments. Furthermore, the medical image processing device described in Patent Document 2 has the problem that the user has to correct the recognition results, making the process cumbersome.

[0006] The present invention has been made in view of the above circumstances, and aims to provide a medical system, control method, and control program that can automatically adjust to an appropriate and stable endoscopic field of view without user intervention. [Means for solving the problem]

[0007] To achieve the above objective, the present invention provides the following means. A first aspect of the present invention is a medical system comprising: an endoscope that acquires an image including a treatment instrument; a moving device for moving the endoscope; a processor that controls the moving device; and a storage unit that stores a plurality of control parameters including known field-of-view parameters for each type of treatment instrument to be recognized, wherein the processor recognizes the treatment instrument in the image acquired by the endoscope, determines whether the recognized treatment instrument is the one to be recognized, controls the moving device based on the known field-of-view parameter corresponding to the type of treatment instrument recognized from the plurality of control parameters stored in the storage unit if it is determined that the treatment instrument is the one to be recognized, and controls the moving device based on at least one predetermined control parameter from the plurality of control parameters stored in the storage unit if it is determined that the treatment instrument is not the one to be recognized.

[0008] A second aspect of the present invention is a control method for controlling the movement of an endoscope that acquires an image including a treatment instrument based on the position of the treatment instrument, wherein the method recognizes the treatment instrument in the image acquired by the endoscope, determines whether the recognized treatment instrument is a target for recognition, controls the movement of the endoscope based on a known field of view parameter corresponding to the type of the recognized treatment instrument from among a plurality of control parameters stored in the memory, and controls the movement of the endoscope based on at least one predetermined parameter from among a plurality of control parameters stored in the memory if it is determined that the treatment instrument is not a target for recognition.

[0009] A third aspect of the present invention is a control program for controlling the movement of an endoscope that acquires an image including a treatment instrument based on the position of the treatment instrument, wherein the program causes a computer to perform the following actions: recognize the treatment instrument in the image acquired by the endoscope, determine whether the recognized treatment instrument is a target for recognition, control the movement of the endoscope based on a known field of view parameter corresponding to the type of the recognized treatment instrument from among a plurality of control parameters stored in the memory, and control the movement of the endoscope based on at least one predetermined control parameter from among the plurality of control parameters stored in the memory if it is determined that the treatment instrument is not a target for recognition. [Effects of the Invention]

[0010] The medical system, control method, and control program according to the present invention have the effect of automatically adjusting to an appropriate and stable endoscopic field of view without user intervention. [Brief explanation of the drawing]

[0011] [Figure 1] This is an external view of a medical system according to an embodiment of the present invention. [Figure 2]Figure 1 is a block diagram of the medical system. [Figure 3] This is a flowchart of a control method according to an embodiment of the present invention. [Figure 4] This figure shows an example of an endoscopic image set to a distant endoscopic field of view relative to the treatment instrument. [Figure 5] This figure shows an example of an endoscopic image set to a close endoscopic field of view relative to the treatment instrument. [Figure 6] This figure shows an example of an endoscopic image where the endoscopic field of view changes frequently, and an example of an endoscopic image where the endoscopic field of view is set to a constant level. [Figure 7] This is a flowchart of a control method according to an embodiment of the present invention. [Figure 8] This figure shows an example of how the endoscopic image appears on the monitor when a medical instrument is being changed. [Figure 9] This figure shows an example of an endoscopic image of a treatment instrument with a spiral-shaped tip. [Figure 10] This figure shows an example of an endoscopic image that shows the shaft of a medical instrument. [Figure 11] This figure shows an example of an endoscopic image illustrating the periodic movement history of the tip of a treatment instrument. [Figure 12] This figure shows an example of an endoscopic image in which the intestinal tract is visible around a medical instrument. [Figure 13] This is an external view of a medical system in which a thin instrument is inserted into a trocca. [Figure 14] This is an external view of a medical system in which a thick instrument is inserted into a trocca. [Figure 15] This graph shows an example of the processor's confidence level in recognizing each treatment tool when the confidence level for one type of treatment tool is significantly higher than that of the other treatment tools. [Figure 16] This graph shows an example of the confidence levels of each treatment instrument when the confidence level of the instrument with the highest confidence level is not significantly different from that of the other instruments. [Figure 17] This graph shows an example of the confidence level for each medical device when none of the devices are being recognized. [Figure 18] This is a flowchart of a control method according to an embodiment of the present invention. [Figure 19] This is a graph showing an example of the confidence level of each treatment tool when the confidence level of the treatment tool with the highest confidence level repeatedly has large temporal variations. [Figure 20] This is a flowchart of a control method according to a modified example of an embodiment of the present invention. [Figure 21] This is a flowchart of a control method according to another modified example of an embodiment of the present invention. [Figure 22] This is a flowchart of a control method according to an embodiment of the present invention.

Mode for Carrying Out the Invention

[0012] 〔First Embodiment〕 A medical system, a control method, and a control program according to the first embodiment of the present invention will be described below with reference to the drawings. As shown in FIG. 1, the medical system 10 according to the present embodiment includes an endoscope 1 and a treatment tool 2 inserted into the body of a patient S, a moving device 3 that moves the endoscope 1 inside the body, a control device 5 that controls the moving device 3, a video system center 7, and a monitor 9.

[0013] The endoscope 1 is, for example, a rigid endoscope, and as shown in FIG. 2, includes an imaging unit 1a having an imaging element or the like that acquires an endoscope image. The endoscope 1 acquires an endoscope image including the tip of the treatment tool 2 by the imaging unit 1a. The endoscope image acquired by the endoscope 1 is transmitted to the video system center 7. The imaging unit 1a is, for example, a three-dimensional camera provided at the tip of the endoscope 1, and acquires a stereo image including information on the three-dimensional position of the tip of the treatment tool 2 as the endoscope image.

[0014] As shown in Figure 1, the mobile device 3 includes a robotic arm 3a having multiple joints 3b, and the tip of the robotic arm 3a holds the base end of the endoscope 1. In one example, the robotic arm 3a has three degrees of freedom of motion: linear movement forward and backward along the X-axis, rotation around the Y-axis (pitch), and rotation around the Z-axis (yaw), and preferably further has a degree of freedom of motion of rotation around the X-axis (roll). The X-axis is an axis on the same straight line as the optical axis of the endoscope 1, and the Y-axis and Z-axis are axes that are perpendicular to the optical axis of the endoscope 1 and extend in directions corresponding to the lateral and vertical directions of the endoscope image, respectively.

[0015] The video system center 7 is connected to the endoscope 1, the control device 5, and the monitor 9. The video system center 7 includes an input interface into which endoscopic images from the endoscope 1 are sequentially input, and an output interface (not shown) for sequentially outputting the endoscopic images to the monitor 9 and the control device 5.

[0016] As shown in Figure 2, the control device 5 comprises at least one processor 5a, such as a central processing unit, a memory 5b, such as RAM (Random Access Memory), used as a workspace for the processor 5a, and a storage unit 5c. The control device 5 may be, for example, a desktop computer, a tablet computer, a laptop computer, or a smartphone.

[0017] The memory unit 5c is a computer-readable, non-temporary recording medium such as a ROM (Read-Only Memory) or a hard disk. The memory unit 5c stores programs and data necessary for the processor 5a to execute processing. The processor 5a realizes the functions described later by executing the programs read from the memory unit 5c into memory 5b. Some functions of the processor 5a may be realized by dedicated logic circuits or the like.

[0018] The memory unit 5c stores multiple control parameters, including known field-of-view parameters for each type of treatment instrument 2 that the processor 5a recognizes, and constant field-of-view parameters applicable to multiple types of treatment instruments 2. The control parameters include, for example, the tracking speed, tracking distance, tracking sensitivity, and tracking direction for the endoscope 1 to track the tip of the treatment instrument 2. The tracking speed is the speed at which the endoscope 1 tracks the tip of the treatment instrument 2. The tracking distance is the distance of the endoscope 1 to the tip of the treatment instrument 2 to capture the tip of the treatment instrument 2 at an appropriate position in the depth direction of the endoscopic image. The tracking sensitivity is the sensitivity of the endoscope 1 to tracking the movement of the tip of the treatment instrument 2. The tracking direction is the direction in which the endoscope 1 is moved relative to the tip of the treatment instrument 2 to capture the tip of the treatment instrument 2 at a desired position, such as the center of the endoscopic image. The memory unit 5c also stores a control program that causes the processor 5a to execute a control method that controls the movement of the endoscope 1 based on the position of the treatment instrument 2.

[0019] As shown in Figure 3, the control method includes: step SA4, which recognizes the treatment instrument 2 in the endoscopic image acquired by the endoscope 1; step SA5, which determines whether the recognized treatment instrument 2 is the target of recognition; step SA6, which controls the mobile device 3 based on known field-of-view parameters corresponding to the type of recognized treatment instrument 2 from among a plurality of control parameters stored in the memory unit 5c, if it is determined that the treatment instrument 2 is the target of recognition; and step SA7, which controls the mobile device 3 based on constant field-of-view parameters stored in the memory unit 5c, if it is determined that the treatment instrument 2 is not the target of recognition.

[0020] The processor 5a executes the control method in response to a tracking start command and terminates the control method in response to a tracking end command. The tracking start command and tracking end command are input to the control device 5 by an operator, such as a physician, operating an input device, such as a switch (not shown), provided on the endoscope 1.

[0021] Processor 5a determines whether or not the treatment instrument 2 can be detected in the endoscopic image acquired by the endoscope 1. General technologies such as AI (Artificial Intelligence, not shown) image recognition are used to determine whether or not the treatment instrument 2 can be detected. If the confidence level of the AI's determination that the treatment instrument 2 can be recognized in the endoscopic image is higher than a predetermined recognition threshold, processor 5a determines that the treatment instrument 2 can be detected. On the other hand, if the confidence level of the AI's determination is below the predetermined recognition threshold, processor 5a determines that the treatment instrument 2 cannot be detected.

[0022] Furthermore, when processor 5a determines that it can detect the treatment tool 2, it determines whether the recognized treatment tool 2 is the target of recognition. The determination of whether the treatment tool 2 is the target of recognition also utilizes general technologies such as AI-based image recognition. For example, if the recognition result of the treatment tool 2 by the AI ​​changes fewer than a predetermined number of times within a predetermined time, processor 5a determines that the recognized treatment tool 2 is the target of recognition. On the other hand, if the recognition result by the AI ​​changes more than a predetermined number of times within a predetermined time, processor 5a determines that the recognized treatment tool 2 is not the target of recognition.

[0023] The processor 5a has a normal mode and a constant mode. The normal mode is a mode in which the mobile device 3 is controlled based on known field of view parameters corresponding to the type of treatment instrument 2, from among a plurality of control parameters stored in the memory unit 5c, for the recognized treatment instrument 2. The constant mode is a mode in which the mobile device 3 is controlled based on predetermined constant field of view parameters stored in the memory unit 5c for the recognized treatment instrument 2.

[0024] Next, the operation of the medical system 10, the control method, and the control program will be explained with reference to the flowchart in Figure 3. The operator performs treatment on the affected area inside the patient S by manipulating the treatment instrument 2 inserted into the patient S's body while observing the endoscopic image displayed on the monitor 9. During the treatment, the processor 5a determines whether or not a tracking start command has been input (step SA1). If it is not determined that a tracking start command has been input, the system waits until a tracking start command is input.

[0025] If it is determined that a tracking start command has been entered, the processor 5a determines whether or not a tracking end command has been entered (step SA2). If it is determined that a tracking end command has not been entered, the processor 5a acquires an endoscopic image of the body currently being treated, which is taken by the endoscope 1 (step SA3).

[0026] Next, the processor 5a determines whether or not the treatment instrument 2 can be detected in the acquired endoscopic image (step SA4). If the AI's confidence level in recognizing the treatment instrument 2 in the endoscopic image is higher than a predetermined recognition threshold, the processor 5a determines that the treatment instrument 2 can be detected. On the other hand, if the AI's confidence level is below the predetermined recognition threshold, the processor 5a determines that the treatment instrument 2 cannot be detected, and the process returns to step SA2.

[0027] If it is determined that instrument 2 can be detected, the processor 5a determines whether the instrument 2 recognized in the endoscopic image is the target of recognition (step SA5). For example, if the AI's recognition result of instrument 2 changes fewer than Y times within X seconds, the processor 5a determines that instrument 2 is the target of recognition. On the other hand, if the AI's recognition result changes Y or more times within X seconds, the processor 5a determines that instrument 2 is not the target of recognition. X seconds and Y times can be any values.

[0028] If it is determined that the treatment instrument 2 is the target of recognition, the processor 5a sets the system to normal mode. As a result, the mobile device 3 is controlled based on the known field of view parameters corresponding to the type of recognized treatment instrument 2 stored in the memory unit 5c, and the endoscope 1 begins tracking the treatment instrument 2 in normal mode (step SA6).

[0029] In this case, the endoscopic field of view can be set to one suitable for the treatment instrument 2 currently in use, as shown in Figure 4 or Figure 5, for example. Figure 4 shows an endoscopic image with a grasping forceps as treatment instrument 2, and the endoscopic field of view is set to a distant one based on the known field of view parameters of the grasping forceps currently in use. Figure 5 shows an endoscopic image with an electrosurgical unit as treatment instrument 2, and the endoscopic field of view is set to a closer one based on the known field of view parameters of the electrosurgical unit currently in use.

[0030] On the other hand, if it is determined that the treatment instrument 2 is not the object to be recognized, the processor 5a sets it to a fixed mode. As a result, the mobile device 3 is controlled based on predetermined fixed field-of-view parameters stored in the memory unit 5c, and the endoscope 1 starts tracking the treatment instrument 2 in the fixed mode (step SA7).

[0031] Since the constant field of view parameter is a control parameter applicable to multiple types of treatment instruments 2, even if the treatment instrument 2 cannot be recognized as the target of recognition, it is possible to avoid the inconvenience of setting an inappropriate endoscopic field of view or frequent switching of the endoscopic field of view, as shown in Figure 6, and to set an appropriate endoscopic field of view. The two figures on the left of Figure 6 show an example of an endoscopic image in which a distant field of view and a close-up field of view are frequently switched, and the figure on the right of Figure 6 shows an example of an endoscopic image set to an appropriate constant field of view.

[0032] Next, in either step SA6 or step SA7, the process returns to step SA2. If the processor 5a does not determine in step SA2 that a tracking termination command has been input, steps SA3 through SA6 or SA7 are repeated. On the other hand, if the processor 5a determines in step SA2 that a tracking termination command has been input, the control method terminates.

[0033] As described above, according to the medical system 10, control method, and control program of this embodiment, if the recognized treatment instrument 2 is one of the items to be recognized, the mobile device 3 is controlled based on known field-of-view parameters corresponding to that treatment instrument 2. On the other hand, if the treatment instrument 2 is not one of the items to be recognized, the mobile device 3 is controlled based on predetermined field-of-view parameters applicable to multiple types of treatment instruments 2. This allows for automatic adjustment to an appropriate and stable endoscopic field of view without user intervention, not only when the treatment instrument 2 is one of the items to be recognized, but also when the treatment instrument 2 is not one of the items to be recognized.

[0034] Furthermore, if the recognition result of the treatment instrument 2 by the processor 5a changes more than a predetermined number of times within a predetermined time, it is considered that multiple types of treatment instruments 2 are recognized as candidates. In that case, the mobile device 3 is controlled by the processor 5a based on certain field-of-view parameters that are suitable for multiple types of treatment instruments 2, thereby avoiding the inconvenience of the field of view setting of the endoscope 1 being frequently switched and enabling the setting of a stable endoscopic field of view.

[0035] [Second Embodiment] Next, a medical system, control method, and control program according to a second embodiment of the present invention will be described below with reference to the drawings. The medical system 10 according to this embodiment differs from the first embodiment in that, when the treatment instrument 2 is removed while the system is set to a certain mode by the processor 5a, the setting of the certain mode is released. In describing this embodiment, parts that share the same configuration as the medical system, control method, and control program according to the first embodiment described above are denoted by the same reference numerals and their descriptions are omitted.

[0036] The processor 5a turns on the constant mode maintenance flag when the system is set to constant mode, and turns off the constant mode maintenance flag when the system is deactivated. Furthermore, when the system is set to constant mode, the processor 5a determines whether the currently used treatment instrument 2 has been removed. This determination is made, for example, by checking whether the treatment instrument 2 cannot be recognized in the endoscopic image for a predetermined period of time or longer.

[0037] Specifically, processor 5a starts counting when the treatment instrument 2 is no longer recognized in the endoscopic image acquired by endoscope 1. If the state of not being able to recognize treatment instrument 2 continues for a predetermined time or longer after the start of counting, it determines that treatment instrument 2 has been removed. When processor 5a determines that treatment instrument 2 has been removed, it cancels the constant mode. If treatment instrument 2 is recognized before the predetermined time has elapsed since the start of counting, the count is reset.

[0038] The operation of the medical system 10, control method, and control program according to this embodiment, configured as described above, will be explained below with reference to the flowchart in Figure 7. In this embodiment, when the control method is started, the processor 5a cancels the setting of the constant mode and turns off the constant mode maintenance flag (step SB1), and then proceeds to step SA1. Also, when an endoscopic image is acquired in step SA3, the processor 5a determines whether or not the constant mode is set (step SB3-1). If it is determined that the constant mode is not set, the process proceeds to step SA4.

[0039] If it is determined in step SA4 that the treatment instrument 2 can be detected, the processor 5a determines whether the constant mode maintenance flag is OFF or OFF (step SB4). If it is determined that the constant mode maintenance flag is OFF, the processor 5a determines whether the recognized treatment instrument 2 is the target of recognition (step SA5). If it is determined that it is the target of recognition, the processor 5a starts tracking the treatment instrument 2 with the endoscope 1 in normal mode (step SA6), and then returns to step SA2.

[0040] On the other hand, if it is determined in step SB4 that the constant mode maintenance flag is not OFF, that is, that it is set to constant mode, and if it is determined in step SA5 that the treatment instrument 2 is not the target of recognition, in either case the processor 5a starts tracking the endoscope 1 for the treatment instrument 2 in constant mode (step SA7), and then returns to step SA2.

[0041] In step SB3-1, if it is determined that the system is set to a constant mode, the processor 5a determines whether the currently used instrument 2 has been removed (step SB3-2). If the state in which instrument 2 cannot be recognized in the endoscopic image does not continue for Z seconds or more, the processor 5a determines that instrument 2 has not been removed. In this case, the system remains set to a constant mode and proceeds to step SA4. Z seconds can be any value.

[0042] On the other hand, if the state in which the treatment instrument 2 cannot be recognized in the endoscopic image continues for Z seconds or longer, the processor 5a determines that the treatment instrument 2 has been removed. In this case, the processor 5a cancels the setting of the constant mode and turns off the constant mode maintenance flag (step SB3-3), and then proceeds to step SA4.

[0043] If the state in which the treatment instrument 2 cannot be recognized continues for a predetermined period of time or longer, it is possible that the treatment instrument 2 being used has been replaced with another treatment instrument 2, as shown in Figure 8, for example. Therefore, according to the medical system 10, control method, and control program of this embodiment, even if a constant mode is set, if the state in which the treatment instrument 2 cannot be recognized continues for a predetermined period of time or longer, the constant mode setting is canceled, so that when the treatment instrument 2 is replaced with a treatment instrument 2 of the type being recognized, the endoscopic field of view can be immediately and appropriately reset to one suitable for the replaced treatment instrument 2.

[0044] [Third Embodiment] Next, a medical system, control method, and control program according to a third embodiment of the present invention will be described below with reference to the drawings. The medical system 10 according to this embodiment differs from the first embodiment in that the processor 5a determines whether or not the treatment instrument 2 is the object to be recognized based on the characteristic quantities of the treatment instrument 2 in the endoscopic image. In describing this embodiment, parts that share the same configuration as the medical system, control method, and control program according to the first embodiment described above are denoted by the same reference numerals and their descriptions are omitted.

[0045] The memory unit 5c stores known feature quantities for each type of treatment tool 2 that is being recognized. These known feature quantities are, for example, known shape data relating to the shape of the tip of the treatment tool 2, and may be the shape of the tip of a forceps or the shape of the tip of an electrosurgical unit.

[0046] The processor 5a detects image shape data, particularly the shape of the tip of the treatment instrument 2 recognized in the endoscopic image, as image features. Then, the processor 5a determines whether or not the treatment instrument 2 is the target of recognition based on the comparison result between the detected image shape data and the known shape data stored in the memory unit 5c.

[0047] The operation of the medical system 10, control method, and control program configured in this manner will be explained with reference to the flowchart in Figure 3. In this embodiment, in step SA5, the processor 5a detects the image shape data of the treatment instrument 2 recognized in the endoscopic image. For example, if the tip 2a of the treatment instrument 2 has a spiral shape, as shown in the endoscopic image in Figure 9, image shape data in which the treatment instrument mask is discontinuous is detected. The processor 5a then compares the detected image shape data with known shape data corresponding to the type of recognized treatment instrument 2 stored in the storage unit 5c.

[0048] If the difference between the detected image shape data and the known shape data is smaller than a predetermined feature threshold, the processor 5a determines that the recognized treatment tool 2 is the target of recognition. On the other hand, if the difference is larger than or equal to the predetermined feature threshold, the processor 5a determines that the recognized treatment tool 2 is unknown, i.e., not the target of recognition.

[0049] According to the medical system 10, control method, and control program of this embodiment, by using the characteristic quantities of the treatment instrument 2 in the endoscopic image as judgment criteria, it is possible to determine with greater accuracy whether or not the treatment instrument 2 is the target of recognition. In particular, since the shape of the tip of the treatment instrument 2 differs greatly depending on the type of treatment instrument 2, by using the shape of the treatment instrument 2 as judgment criteria, the determination can be made more simply and accurately.

[0050] In this embodiment, the shape of the treatment tool 2 is used as a criterion for judgment, but instead, the color of the treatment tool 2 may be used as a criterion for judgment. In this case, the memory unit 5c may store known color data, particularly regarding the color of the shaft of the treatment instrument 2, as known features. The known color data may be a representation of the color information of the shaft of the treatment instrument 2 being recognized using the three components RGB (Red, Green, Blue), for example, R=1 to 20, G=50 to 60, and B=200 to 240.

[0051] Furthermore, the processor 5a may detect image color data, particularly the color of the shaft 2b of the treatment instrument 2 recognized in the endoscopic image shown in Figure 10, as an image feature. The image color data may be, for example, obtained by detecting the RGB components of each pixel in the endoscopic image.

[0052] Furthermore, the processor 5a may determine whether the treatment tool 2 is the target of recognition based on the comparison result between the detected image color data and the known color data corresponding to the type of recognized treatment tool 2 stored in the storage unit 5c, similar to the case where the shape of the treatment tool 2 is used as a criterion. For example, if a certain number or more of the detected image color data fall within the range of the three RGB components of the known color data, it may be determined that the recognized treatment tool 2 is the target of recognition.

[0053] In this modified example, the RGB three-component representation method is used to quantify the color information of pixels. However, alternatively, a representation method using color measurement information such as HSV (Hue Saturation Value) or CMYK (Cyan, Magenta, Yellow, Key plate) may be used.

[0054] In this embodiment, both the shape and color of the treatment tool 2 may be used as criteria for judgment. In this case, if both the difference between the detected image shape data and the known shape data, or the difference between the image color data and the known color data, are smaller than a predetermined feature threshold, the processor 5a may determine that the treatment tool 2 is the object to be recognized. On the other hand, if either the difference between the detected image shape data and the known shape data, or the difference between the image color data and the known color data, is larger than a predetermined feature threshold, the processor 5a may determine that the treatment tool 2 is not the object to be recognized.

[0055] In this embodiment, the processor 5a may also use the time-series trajectory of the treatment tool 2 as a basis for judgment. In this case, the memory unit 5c may store known trajectory data relating to the time-series trajectory of the treatment tool 2 as known features. The known trajectory data may be, for example, the periodic motion trajectory specific to the tip of each treatment tool 2 in the treatment in which each treatment tool 2 is used.

[0056] Furthermore, the processor 5a may also detect image trajectory data as image features relating to the time-series trajectory of the treatment instrument 2, particularly its tip, that it recognizes in the endoscopic image. For example, as shown in the endoscopic image in Figure 11, the image trajectory data may include a record of the periodic movement history of the tip of one of the two forceps used as the treatment instrument 2.

[0057] Furthermore, the processor 5a may determine whether the recognized treatment tool 2 is the target of recognition based on the comparison result between the detected image trajectory data and known trajectory data corresponding to the type of recognized treatment tool 2 stored in the storage unit 5c, similar to the case where the shape or color of the treatment tool 2 is used as a basis for judgment.

[0058] Since the general behavior of the treatment instrument 2 is determined by the type of treatment instrument 2, according to this modified example, by using the time-series trajectory of the treatment instrument 2 in the endoscopic image as a basis for judgment, it is possible to easily and accurately determine whether the recognized treatment instrument 2 is the target of recognition.

[0059] In this embodiment, the processor 5a may use the subject present around the treatment tool 2 as a basis for judgment. In this case, the memory unit 5c may store known subject data relating to subjects present around the treatment instrument 2 as known features. The known subject data may be, for example, organs present at the treatment location where each treatment instrument 2 is used.

[0060] Furthermore, the processor 5a may also detect image subject data as image features related to subjects surrounding the treatment instrument 2 recognized in the endoscopic image. For example, as shown in the endoscopic image in Figure 12, if the intestinal tract C is visible behind the stapler, which is the treatment instrument 2, the intestinal tract C may be detected as image subject data.

[0061] Furthermore, the processor 5a may determine whether the recognized treatment tool 2 is the target of recognition based on the comparison result between the detected image subject data and known subject data corresponding to the type of recognized treatment tool 2 stored in the storage unit 5c, similar to the case where the shape or color of the treatment tool 2 is used as a basis for judgment.

[0062] Since the location within the body cavity where the instrument 2 is used differs depending on the type of instrument 2, this modified version allows for a simple and accurate determination of whether or not the instrument 2 is the object being recognized by using the surrounding objects in the endoscopic image as a basis for judgment.

[0063] [Fourth Embodiment] Next, a medical system, control method, and control program according to the fourth embodiment of the present invention will be described below with reference to the drawings. The medical system 10 according to this embodiment differs from the first embodiment in that the processor 5a determines whether or not the treatment instrument 2 is the object to be recognized based on the diameter of the recognized treatment instrument 2. In describing this embodiment, parts that share the same configuration as the medical system, control method, and control program according to the first embodiment described above are denoted by the same reference numerals and their descriptions are omitted.

[0064] The medical system 10 includes a trocker 11 fixed to the body surface of the patient S, penetrating the body surface, as shown in Figures 13 and 14. The trocker 11 is a cylindrical member having a through-hole into which a treatment instrument 2 can be inserted. By inserting the treatment instrument 2 into the through-hole of the trocker 11 fixed to the body surface, the treatment instrument 2 can be held in a state inserted into the body cavity. The trocker 11 is equipped with a sensor 13 that detects the diameter of the treatment instrument 2 inserted into the through-hole.

[0065] Sensor 13 is a touch sensor capable of measuring the continuous contact time of a treatment tool 2 that contacts the inner surface of a through-hole in the trocker 11. When the treatment tool 2 is inserted into the through-hole of the trocker 11, sensor 13 measures the continuous contact time between the treatment tool 2 and the inner surface of the through-hole. The continuous contact time measured by sensor 13 is sequentially transmitted to processor 5a. The memory unit 5c stores known diameter dimension data for each type of treatment tool 2 that is being recognized. The known diameter dimension data may include, for example, whether the diameter is thin or thick.

[0066] The processor 5a detects the approximate diameter of the treatment tool 2 based on the continuous contact time between the treatment tool 2 and the inner surface of the through-hole, which is transmitted from the sensor 13. For example, as shown in Figure 13, a treatment tool 2 that hardly contacts the inner surface of the through-hole of the trocker 11 has a short continuous contact time. A treatment tool 2 with a continuous contact time shorter than a predetermined contact time is determined to have a small diameter. On the other hand, as shown in Figure 14, a treatment tool 2 that is always in contact with the inner surface of the through-hole of the trocker 11 has a long continuous contact time. A treatment tool 2 with a continuous contact time longer than a predetermined contact time is determined to have a large diameter.

[0067] The processor 5a determines whether the treatment tool 2 is the one being recognized based on the comparison result between the diameter of the treatment tool 2 detected by the sensor 13 and the known diameter data corresponding to the type of recognized treatment tool 2 stored in the memory unit 5c.

[0068] Next, the operation of the medical system 10, control method, and control program configured in this manner will be explained with reference to the flowchart in Figure 3. In this embodiment, in step SA5, the processor 5a detects the approximate diameter of the treatment instrument 2 currently inserted into the trocker 11 based on the continuous contact time sent from the sensor 13. The processor 5a then compares the detected approximate diameter of the treatment instrument 2 with the known diameter data corresponding to the recognized type of treatment instrument 2 stored in the memory unit 5c.

[0069] If the detected approximate diameter matches the known diameter data, the processor 5a determines that the treatment tool 2 is the object being recognized. On the other hand, if the detected approximate diameter does not match the known diameter data, the processor 5a determines that the recognized treatment tool 2 is unknown, i.e., not the object being recognized.

[0070] Since the diameter of the treatment tool 2 may differ depending on the type of treatment tool 2, according to this embodiment, by using the diameter of the treatment tool 2 detected by the sensor 13 as a basis for judgment, it is possible to easily and accurately determine whether or not the treatment tool 2 is the target of recognition.

[0071] In this embodiment, instead of a touch sensor, a sensor capable of measuring the specific diameter of the treatment tool 2 may be used as the sensor 13. In this case, the known diameter data may be the numerical value of the specific diameter of the treatment tool 2. If the difference between the measured value of the diameter of the treatment tool 2 measured by the sensor 13 and the known diameter data associated with the recognized treatment tool 2 is smaller than a predetermined threshold, it is determined that the recognized treatment tool 2 is the target of recognition. On the other hand, if the difference is larger than the predetermined threshold, it is determined that the treatment tool 2 is not the target of recognition.

[0072] [Fifth Embodiment] Next, a medical system, control method, and control program according to the fifth embodiment of the present invention will be described below with reference to the drawings. The medical system 10 according to this embodiment differs from the first embodiment in that the processor 5a determines whether or not the treatment tool 2 is the object to be recognized based on the confidence level of the recognition result of the treatment tool 2. In describing this embodiment, parts that share the same configuration as the medical system, control method, and control program according to the first embodiment described above are denoted by the same reference numerals and their descriptions are omitted.

[0073] When the processor 5a recognizes multiple treatment instruments 2 in the endoscopic image, it calculates a confidence level P for the recognition result of each treatment instrument 2. The confidence level P of the recognition result may be calculated, for example, based on analysis information of the endoscopic image, or based on information from sensors provided on the treatment instrument 2 or trocker. The analysis information of the endoscopic image may be, for example, information obtained from feature extraction of the endoscopic image, such as the treatment instrument 2 or organs, treatment (task), or surgical scene, or information obtained from the operational behavior of the treatment instrument 2. Then, the processor 5a determines whether the recognized treatment instrument 2 is the target of recognition based on the comparison result obtained by comparing the difference in confidence levels P between the treatment instruments 2.

[0074] For example, as shown in Figure 15, among the treatment tools 2A, 2B, and 2C output as the recognition result of treatment tool 2, if the confidence level P of treatment tool 2A, which has the highest confidence level P, is higher than or equal to a predetermined first threshold P_th, and the difference ΔP between the confidence level P of treatment tool 2A and all other recognized treatment tools 2B and 2C is greater than or equal to a predetermined second threshold ΔP_th, then it is unlikely that treatment tool 2A is not the target of recognition, and the recognition can be clearly narrowed down to treatment tool 2A. In this case, the processor 5a determines that treatment tool 2A is currently in use and is the target of recognition.

[0075] On the other hand, as shown in Figure 16, for example, if the confidence level P of the treatment tool 2A, which has the highest confidence level P, is higher than or equal to a predetermined first threshold P_th, but the difference ΔP between the confidence level P of any of the other recognized treatment tools 2B, 2C is smaller than a predetermined second threshold ΔP_th, then it is possible that treatment tool 2A is not the target of recognition, and therefore it cannot be narrowed down to treatment tool 2A. Also, as shown in Figure 17, if the confidence levels P of all treatment tools 2A, 2B, and 2C are all lower than a predetermined first threshold P_th, then it is highly likely that none of the treatment tools 2A, 2B, and 2C are the target of recognition. In these cases, the processor 5a determines that none of the multiple recognized treatment tools 2 are the target of recognition.

[0076] Next, the operation of the medical system 10 configured in this manner, the control method, and the control program will be explained with reference to Figures 15 to 17 and the flowchart in Figure 18. If multiple treatment tools 2 are detected in step SA4, the processor 5a determines whether there is one or more treatment tools 2 whose recognition confidence P is higher than or equal to the first threshold P_th (step SC5-1).

[0077] If it is determined that there is one or more types, the processor 5a determines whether the difference ΔP between the confidence level P of the treatment instrument 2 with the highest confidence level P and the confidence levels P of all other treatment instruments 2 is greater than or equal to a predetermined second threshold ΔP_th (step SC5-2). As shown in Figure 15, if the difference ΔP between the confidence level P of the treatment instrument 2A with the highest confidence level P and the confidence levels P of all other treatment instruments 2B, 2C is greater than or equal to a predetermined second threshold ΔP_th, the processor 5a determines that treatment instrument 2A is currently in use and is the one to be recognized. In this case, the processor 5a starts tracking the endoscope 1 with the treatment instrument 2 in normal mode (step SA6).

[0078] On the other hand, in step SC5-1, if there is no treatment instrument 2 with a confidence level P of recognition result higher than or equal to the first threshold P_th, as shown in Figure 17, and in step SC5-2, if the difference ΔP between the confidence level P of treatment instrument 2A with the highest confidence level P and the confidence level P of other treatment instruments 2B, as shown in Figure 16, is smaller than a predetermined second threshold ΔP_th, then in both cases, the processor 5a determines that none of the recognized treatment instruments 2 are the target of recognition. In this case, the processor 5a starts tracking the endoscope 1 with the treatment instruments 2 in a constant mode (step SA7).

[0079] According to this embodiment, when multiple treatment instruments 2 are recognized, the field of view of the endoscope 1 suitable for the treatment instrument 2 currently in use can be efficiently provided. Furthermore, by using the difference ΔP in the confidence level P of the recognition results between the treatment instruments 2 as a criterion, it is possible to accurately determine whether or not a treatment instrument 2 is the target of recognition.

[0080] In this embodiment, if in step SC5-2 it is determined that the difference ΔP of confidence levels P is smaller than a predetermined second threshold ΔP_th, the mobile device 3 may be controlled using, for example, the median value of the known field of view parameters corresponding to multiple treatment instruments 2 whose confidence levels P are higher than or equal to Pmax-ΔP_th, instead of a constant field of view parameter. As the median value, for example, a simple average or a weighted average may be adopted. Pmax is the confidence level P of the treatment instrument 2 with the highest confidence level P. The weighted average can be calculated using, for example, the following formula: Σ{(Known field parameter * weight w_i) / Σw_i) corresponding to the type of treatment instrument 2 with confidence level P equal to or greater than Pmax-ΔP_th} w_i = f(Pmax, ΔP_th, confidence level Pi of treatment device 2 where confidence level P is greater than or equal to Pmax - ΔP_th)

[0081] Furthermore, in this embodiment, the mode is switched based on the confidence level P of the recognition result of the treatment instrument 2. However, instead, the mode for tracking the treatment instrument 2 may be switched based on the confidence level of the recognition result of, for example, the type of treatment, surgical task (e.g., blunt dissection, suturing, etc.), or surgical scene (e.g., processing of the ○○ mesenter, processing of the △△ blood vessel, etc.). In this case, for example, a normal mode that differs for each type of treatment, surgical task, or surgical scene, and a constant mode may be switched.

[0082] This embodiment can be modified as follows. The processor 5a may determine whether or not a treatment tool 2 is the object to be recognized based on the comparison results obtained by comparing the temporal variation of the confidence level P for each treatment tool 2 that it has calculated.

[0083] For example, as shown in Figure 15, among the treatment tools 2A, 2B, and 2C output as the recognition result of treatment tool 2, if the confidence level P of treatment tool 2A, which has the highest confidence level P, remains consistently above a predetermined first threshold P_th within a predetermined fluctuation time, it may be possible to narrow down the possibilities to treatment tool 2A. In this case, the processor 5a may determine that treatment tool 2A is the one currently in use and is the one being recognized.

[0084] On the other hand, as shown in Figure 19, for example, if the confidence level P of the treatment tool 2A with the highest confidence level P repeatedly rises above a predetermined first threshold P_th or falls below a predetermined first threshold P_th within a predetermined fluctuation time, a phenomenon known as chattering, then it is possible that treatment tool 2A is not the target of recognition, and therefore it cannot be narrowed down to treatment tool 2A. Also, in the case shown in Figure 17, it is highly likely that none of the treatment tools 2A, 2B, and 2C are the target of recognition. In these cases, the processor 5a may determine that none of the multiple recognized treatment tools 2 are the target of recognition.

[0085] Next, the operation of the medical system 10 configured in this manner, the control method, and the control program will be explained with reference to Figures 15, 17, 19, and the flowchart in Figure 20. In step SC5-1, if it is determined that there is one or more treatment tools whose recognition confidence level P is higher than or equal to the first threshold P_th, the processor 5a determines whether the confidence level P of the treatment tool 2 with the highest confidence level P remains higher than or equal to a predetermined first threshold P_th within a predetermined variation time (step SD5-2).

[0086] As shown in Figure 15, if the confidence level P of the treatment instrument 2A with the highest confidence level P remains consistently above a predetermined first threshold P_th, the processor 5a determines that the treatment instrument 2A is currently in use and is the target of recognition. In this case, the processor 5a starts tracking the endoscope 1 with the treatment instrument 2 in normal mode (step SA6).

[0087] On the other hand, in step SC5-1, if there are no treatment instruments 2 with a confidence level P of recognition result higher than or equal to the first threshold P_th, as shown in Figure 17, and in step SD5-2, if the confidence level P of treatment instrument 2A with the highest confidence level P repeatedly crosses the first threshold P_th and fluctuates significantly over time, as shown in Figure 19, in both cases the processor 5a determines that none of the multiple recognized treatment instruments 2 are the target of recognition. In this case, the processor 5a starts tracking the endoscope 1 with the treatment instruments 2 in a constant mode (step SA7).

[0088] According to this modified example, by using the temporal variation in the confidence level of the recognition result for each treatment tool 2 as a basis for judgment, it is possible to accurately determine whether or not treatment tool 2 is the object being recognized.

[0089] In this modified example, in step SD5-2, if it is determined that the confidence level P of the treatment tool 2 with the highest confidence level P repeatedly crosses the first threshold P_th and exhibits large temporal fluctuations, the mobile device 3 may be controlled using, for example, an intermediate value such as the simple average or weighted average of the known field of view parameters corresponding to each of the multiple treatment tools 2 whose confidence level P of recognition result is higher than or equal to the first threshold P_th, instead of a constant field of view parameter.

[0090] In this embodiment, the processor 5a may determine whether or not a treatment tool 2 is the object to be recognized based on the comparison result obtained by comparing both the difference ΔP in confidence levels P between the calculated treatment tools 2 and the temporal variation of the confidence level P for each treatment tool 2.

[0091] In this modified example, as shown in the flowchart of Figure 21, for example, in step SC5-1, if it is determined that there is one or more treatment instruments whose confidence level P of recognition result is higher than or equal to the first threshold P_th, the processor 5a determines the difference ΔP of the confidence levels P (step SC5-2). As shown in Figure 15, if it is determined that the difference ΔP between the confidence level P of treatment instrument 2A, which has the highest confidence level P, and the confidence levels P of all other treatment instruments 2B and 2C is greater than or equal to a predetermined second threshold ΔP_th, the processor 5a determines the temporal variation of the confidence level P (step SD5-2). As shown in Figure 15, if it is determined that the confidence level P of treatment instrument 2A, which has the highest confidence level P, remains consistently higher than or equal to a predetermined first threshold P_th within a predetermined variation time, the processor 5a determines that treatment instrument 2A is currently in use and is the target of recognition. In this case, the processor 5a starts tracking the endoscope 1 with the treatment instrument 2 in normal mode (step SA6).

[0092] On the other hand, in step SC5-1, if there is no treatment instrument 2 whose confidence level P of the recognition result is higher than or equal to the first threshold P_th, as shown in Figure 17; in step SC1, if the difference ΔP between the confidence level P of treatment instrument 2A, which has the highest confidence level P, and the confidence level P of other treatment instruments 2B, as shown in Figure 16, is smaller than a predetermined second threshold ΔP_th; and in step SD5-2, if the confidence level P of treatment instrument 2A, which has the highest confidence level P, repeatedly crosses the first threshold P_th within a predetermined fluctuation time, as shown in Figure 19, then in all cases, the processor 5a determines that none of the multiple recognized treatment instruments 2 are the target of recognition. In this case, the processor 5a starts tracking the endoscope 1 with the treatment instruments 2 in a constant mode (step SA7).

[0093] According to this modified example, by using both the temporal variation in the confidence level P of the recognition result for each treatment tool 2, and the difference ΔP in the confidence level P of the recognition results between treatment tools 2 as criteria, it is possible to determine with greater accuracy whether or not a treatment tool 2 is the object being recognized.

[0094] [Sixth Embodiment] Next, a medical system, control method, and control program according to the sixth embodiment of the present invention will be described below with reference to the drawings. The medical system 10 according to this embodiment differs from the first embodiment in that the processor 5a controls the mobile device 3 based on a field of view parameter common to the group to which the treatment tool 2 belongs, when the treatment tool 2 is determined not to be the object of recognition. In describing this embodiment, parts that share the same configuration as the medical system, control method, and control program according to the first embodiment described above are denoted by the same reference numerals and their descriptions are omitted.

[0095] The memory unit 5c stores common field-of-view parameters as control parameters for each group into which multiple treatment instruments 2 are divided according to type. For example, if the treatment instruments 2 are divided into a forceps group consisting of multiple forceps with different model numbers and an electrosurgical unit group consisting of multiple electrosurgical units with different model numbers, the memory unit 5c stores common field-of-view parameters common to the forceps group and common field-of-view parameters common to the electrosurgical unit group, respectively.

[0096] If the AI's recognition result of the treatment tool 2 changes fewer than Y times within X seconds, processor 5a determines that the recognized treatment tool 2 is the target of recognition. However, if the recognition result changes Y or more times within X seconds, processor 5a determines that the recognized treatment tool 2 is not the target of recognition. If processor 5a determines that the recognized treatment tool 2 is not the target of recognition, it then determines, for each group containing multiple treatment tools 2 recognized within X seconds, whether the sum of the recognition times of all treatment tools 2 recognized within X seconds is longer than a predetermined recognition time threshold.

[0097] If the total recognition time is longer than a predetermined recognition time threshold, the processor 5a controls the mobile device 3 based on the common field of view parameters stored in the memory unit 5c that correspond to the group whose total recognition time is longer than the predetermined recognition time threshold. On the other hand, if the total recognition time is shorter than the predetermined recognition time threshold, the processor 5a controls the mobile device 3 based on the constant field of view parameters stored in the memory unit 5c.

[0098] Next, the operation of the medical system 10, control method, and control program configured in this manner will be explained with reference to the flowchart in Figure 22. If, in step SA5, the AI's recognition result changes Y or more times within X seconds, and it is determined that the treatment tool 2 is not the target of recognition, the processor 5a determines whether the sum of the recognition times of all treatment tools 2 recognized within X seconds is longer than N seconds for each group containing the treatment tool 2 recognized within X seconds (step SE5-1). N seconds can be any value. The processor 5a also determines which group has a longer sum of recognition times (step SE5-2).

[0099] If the total recognition time of the treatment instruments 2 in the forceps group is N seconds or more (step SE5-1 "YES") and is longer than the total recognition time of the treatment instruments 2 in the electrosurgical group (step SE5-2 "YES"), the processor 5a controls the mobile device 3 based on the common field of view parameters corresponding to the forceps group. This initiates the endoscope 1 tracking the treatment instruments 2 in the common mode for the forceps group (step SE7-1). On the other hand, if the total recognition time of the treatment instruments 2 in the electrosurgical group is N seconds or more (step SE5-1 "YES") and is longer than the total recognition time of the treatment instruments 2 in the forceps group (step SE5-2 "NO"), the processor 5a controls the mobile device 3 based on the common field of view parameters corresponding to the electrosurgical group. This initiates the endoscope 1 tracking the treatment instruments 2 in the common mode for the electrosurgical group (step SE7-2). In step SE5-1, if the total recognition time of the treatment instrument 2 for any group is less than N seconds, the processor 5a starts tracking the endoscope 1 with the treatment instrument 2 in a constant mode (step SA7).

[0100] According to the medical system 10, control method, and control program of this embodiment, even if the recognized treatment instrument 2 is not the one intended for recognition, the mobile device 3 is controlled based on control parameters common to treatment instruments 2 of the same type, thereby enabling it to be set to a more appropriate field of view of the endoscope 1.

[0101] In this embodiment, step SE5-2 determines which group has a longer total recognition time. However, instead, for example, it may be possible to determine which group contains the treatment tool 2 with the highest confidence level of recognition results within X seconds, and to adopt the common field of view parameter corresponding to the group containing the treatment tool 2 with the highest confidence level.

[0102] Although embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and design changes and the like are also included within the scope of the gist of the present invention. For example, the present invention is not limited to being applied to the above embodiments and modified examples, but may also be applied to embodiments that appropriately combine these embodiments and modified examples, and is not particularly limited.

[0103] Furthermore, in each of the above embodiments and its modifications, the mobile device 3 is controlled in a constant mode based on a constant field of view parameter applicable to multiple types of treatment tools 2. However, the invention is not limited to this, and the mobile device 3 may be controlled by the processor 5a based on at least one predetermined control parameter among the control parameters stored in the memory unit 5c. For example, in a constant mode, the mobile device 3 may be controlled using a known field of view parameter corresponding to one of the predetermined types of treatment tools 2. In this case, the number of control parameters to be stored in the memory unit 5c can be reduced because there is no need to prepare control parameters for treatment tools 2 that are not the target of recognition, separately from the known field of view parameters. This is effective when the recognized type of treatment tool 2 is similar to one of the types of treatment tools 2 that are the target of recognition. Also, for example, in a constant mode, the average value, median value, or other calculable value of a plurality of predetermined control parameters may be used.

[0104] Furthermore, in each of the above embodiments and its modifications, the processor 5a separately determines whether or not the treatment instrument 2 can be detected in the endoscopic image (step SA4) and whether or not the recognized treatment instrument 2 is the target of recognition (step SA5) using general technologies such as AI-based image recognition. However, instead, these determinations may be made together using general technologies such as AI-based image recognition. [Explanation of symbols]

[0105] 1 Endoscope 2. Treatment tools 3. Mobile device 5a processor 5c storage section 10 Healthcare System 13 sensors

Claims

1. An endoscope that acquires images including the treatment instruments, A moving device for moving the endoscope, A processor that controls the mobile device, The system includes a storage unit that stores a plurality of control parameters, including known field-of-view parameters for each type of treatment tool being recognized, The aforementioned processor, The procedure instrument is recognized in the image acquired by the endoscope. Determine whether the recognized treatment device is the object of recognition, When it is determined that the treatment instrument is the object to be recognized, the mobile device is controlled based on the known field of view parameter corresponding to the type of treatment instrument recognized from among the plurality of control parameters stored in the memory unit. A medical system that, when it is determined that the treatment device is not the object to be recognized, controls the mobile device based on at least one predetermined control parameter among a plurality of control parameters stored in the memory unit.

2. The medical system according to claim 1, wherein, when the processor determines that the treatment instrument is not the object to be recognized, it controls the mobile device based on a certain field of view parameter, which is different from the known field of view parameter, among a plurality of control parameters.

3. The medical system according to claim 1, wherein the processor determines that the treatment instrument is not the object to be recognized, and controls the mobile device based on one of the known field of view parameters.

4. The medical system according to claim 1, wherein the processor determines that the treatment device is not the object to be recognized if the recognition result of the treatment device changes more than a predetermined number of times within a predetermined time.

5. The aforementioned processor, If it is determined that the treatment device is not the object to be recognized, the device is set to a constant mode in which it controls the moving device based on one of the multiple control parameters stored in the memory unit. The medical system according to claim 1, wherein, while set to the constant mode, if the state in which the treatment instrument in the image cannot be recognized continues for a predetermined period of time or longer, the setting of the constant mode is released.

6. The memory unit stores known characteristic quantities for each type of treatment tool that is the target of recognition. The aforementioned processor, The image features of the treatment device in the aforementioned image are detected, The medical system according to claim 1, which determines whether or not the treatment tool is the object to be recognized based on the result of comparing the detected image features with the known features corresponding to the type of treatment tool recognized and stored in the memory unit.

7. The known feature is at least one of known color data relating to the color of the treatment tool and known shape data relating to the shape of the treatment tool. The aforementioned processor, At least one of the image color data relating to the color of the treatment tool in the image and the image shape data relating to the shape of the treatment tool is detected as the image feature quantity. If the difference between the detected image features and the known features corresponding to the type of treatment tool stored in the memory unit is smaller than a predetermined feature threshold, it is determined that the treatment tool is the target of recognition. The medical system according to claim 6, wherein if the difference is greater than or equal to a predetermined feature threshold, it is determined that the treatment device is not the object to be recognized.

8. The known feature is known trajectory data relating to the time-series trajectory of the treatment tool. The aforementioned processor, Image trajectory data relating to the time-series trajectory of the treatment instrument in the image is detected as the image feature quantity. If the difference between the detected image features and the known features corresponding to the type of treatment tool stored in the memory unit is smaller than a predetermined feature threshold, it is determined that the treatment tool is the target of recognition. The medical system according to claim 6, wherein if the difference is greater than or equal to a predetermined feature threshold, it is determined that the treatment device is not the object to be recognized.

9. The known feature is known subject data relating to a subject present around the treatment tool. The processor detects image subject data relating to the subject surrounding the treatment tool in the image as the image feature quantity. If the difference between the detected image features and the known features corresponding to the type of treatment tool stored in the memory unit is smaller than a predetermined feature threshold, it is determined that the treatment tool is the target of recognition. The medical system according to claim 6, wherein if the difference is greater than or equal to a predetermined feature threshold, it is determined that the treatment device is not the object to be recognized.

10. The device is equipped with a sensor that detects the diameter of the treatment tool, The storage unit stores known diameter data relating to the diameter dimensions of each type of treatment tool that is to be recognized. The medical system according to claim 1, wherein the processor determines whether or not the treatment tool is one to be recognized, based on the result of comparing the diameter of the treatment tool detected by the sensor with the known diameter data corresponding to the type of treatment tool recognized and stored in the storage unit.

11. The aforementioned processor, When multiple treatment devices are recognized, the confidence level of the recognition result for each treatment device is calculated. The medical system according to claim 1, which determines whether or not a treatment tool is the object of recognition based on the result of comparing at least one of the temporal variation of the confidence level for each of the treatment tools and the difference in confidence levels between the treatment tools.

12. The aforementioned processor, If the confidence level of the treatment tool with the highest confidence level is higher than or equal to a predetermined first threshold, and the difference between that confidence level and that of the other treatment tools is greater than or equal to a predetermined second threshold, then it is determined that the treatment tool with the highest confidence level is the one currently in use and is the one being recognized. The medical system according to claim 11, which determines that none of the multiple recognized treatment devices are the target of recognition if the confidence level of the treatment device with the highest confidence level is higher than or equal to the predetermined first threshold, but the difference between that confidence level and that of the other treatment devices is less than the predetermined second threshold, or if the confidence levels of all the treatment devices are lower than the predetermined first threshold.

13. The aforementioned processor, If the confidence level of the treatment tool with the highest confidence level remains consistently above a predetermined first threshold within a predetermined fluctuation time, it is determined that the treatment tool with the highest confidence level is currently in use and is the one being recognized. The medical system according to claim 11, which determines that none of the multiple recognized treatment devices are the target of recognition if the confidence level of the treatment device with the highest confidence level repeatedly rises above a predetermined first threshold and falls below a predetermined first threshold within a predetermined fluctuation time, or if the confidence levels of all the treatment devices are below the predetermined first threshold.

14. The aforementioned processor, If the confidence level of the treatment tool with the highest confidence level remains consistently above a predetermined first threshold within a predetermined fluctuation time, and the difference between that confidence level and that of the other treatment tools is greater than or equal to a predetermined second threshold, then it is determined that the treatment tool with the highest confidence level is the one currently in use and the one being recognized. The medical system according to claim 11, which determines that none of the multiple recognized treatment devices are the target of recognition if the confidence level of the treatment device with the highest confidence level is higher than or equal to the predetermined first threshold, but the difference between it and the confidence levels of the other treatment devices is less than the predetermined second threshold, if the confidence level of the treatment device with the highest confidence level repeatedly rises above or equal to the predetermined first threshold and falls below the predetermined first threshold within the predetermined fluctuation time, or if the confidence levels of all the treatment devices are lower than the predetermined first threshold.

15. The storage unit stores, as control parameters, common field of view parameters common to all treatment instruments within the same group, for each group into which the multiple treatment instruments are divided according to type. The aforementioned processor, If it is determined that the recognized treatment device is not the target of recognition, then for each group containing the recognized treatment device, it is determined whether the sum of the recognition times of all treatment devices recognized within the predetermined time is longer than a predetermined recognition time threshold. If the sum of the recognition times is longer than or equal to the predetermined recognition time threshold, the mobile device is controlled based on the common field of view parameters stored in the storage unit, which correspond to the group whose sum of recognition time is longest than the predetermined recognition time threshold, or the group that includes the treatment instrument with the highest confidence in the recognition result of the treatment instrument. The medical system according to claim 4, wherein the mobile device is controlled based on at least one predetermined control parameter among a plurality of control parameters stored in the memory unit when the sum of the recognition times is shorter than the predetermined recognition time threshold.

16. A control method for controlling the movement of an endoscope that acquires images including a treatment instrument, based on the position of the treatment instrument, The procedure instrument is recognized in the image acquired by the endoscope. Determine whether the recognized treatment device is the object of recognition, When it is determined that the aforementioned treatment instrument is the object to be recognized, the movement of the endoscope is controlled based on the known field of view parameter corresponding to the type of the recognized treatment instrument, from among the multiple control parameters stored in the memory unit. A control method that controls the movement of the endoscope based on at least one predetermined control parameter among a plurality of control parameters stored in the memory unit when it is determined that the treatment instrument is not the object to be recognized.

17. A control program for controlling the movement of an endoscope that acquires images including a treatment instrument, based on the position of the treatment instrument, The procedure instrument is recognized in the image acquired by the endoscope. Determine whether the recognized treatment device is the object of recognition, When it is determined that the aforementioned treatment instrument is the object to be recognized, the movement of the endoscope is controlled based on the known field of view parameter corresponding to the type of the recognized treatment instrument, from among the multiple control parameters stored in the memory unit. A control program that causes a computer to control the movement of the endoscope based on at least one predetermined control parameter among a plurality of control parameters stored in the memory unit, when it is determined that the treatment instrument is not the object to be recognized.

Images