Control device and method of operating the control device

Abstract

A control device (100) includes an information acquisition unit (140), a determination unit (160), and a control unit (170). The information acquisition unit acquires an endoscope image (IMG), which is an image captured by an endoscope (10). On the basis of the endoscope image, the determination unit performs a suction necessity determination, which is a determination of whether or not suction is necessary. If the determination unit determines that the suction is necessary, the control unit performs a control for implementation of suction.

Description

[Technical field]

[0001] The present invention relates to a control device and Method of operating the control device etc. [Background technology]

[0002] Patent Document 1 discloses an intracavity pressure regulating device for an endoscope that controls the intracavity pressure to a constant level. This intracavity pressure regulating device includes a suction tube that is detachably connected to the forceps channel of the endoscope, a suction means connected to the suction tube, a pressure detection means that detects the pressure inside the suction tube, and a suction control means that operates according to an output signal from the pressure detection means to control the operation of the suction means. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-245100 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem in that it is desirable to realize appropriate suction according to the situation in suction during the insertion of an endoscope. In the suction control that only controls the pressure inside the body cavity to be constant as in the above-mentioned Patent Document 1, appropriate suction according to the situation may not be realized. For example, if the lumen is too open during insertion, it is desirable to automatically perform suction. Alternatively, if the tip of the endoscope and the inner wall of the lumen are approaching each other due to suction during insertion, it is desirable to turn off or weaken suction. Alternatively, if the distance between the tip of the endoscope and the mucosa is too close and the image turns red (red ball state), it is desirable to turn off suction. Such suction control is difficult to achieve by only controlling the internal pressure to be constant. [Means for solving the problem]

[0005] One aspect of the present disclosure relates to a control device that includes an information acquisition unit that acquires an endoscopic image, which is an image captured by an endoscope, a determination unit that performs a suction necessity determination to determine whether or not suction is necessary based on the endoscopic image, and a control unit that controls the implementation of the suction when the determination unit determines that the suction is necessary.

[0006] Another aspect of the present disclosure relates to an endoscope system including the control device described above and the endoscope.

[0007] Yet another aspect of the present disclosure relates to a control method including acquiring an endoscopic image which is an image captured by an endoscope, performing a suction necessity determination based on the endoscopic image to determine whether or not suction is necessary, and performing control to perform the suction if it is determined in the suction necessity determination that the suction is necessary. [Brief description of the drawings]

[0008] [Figure 1] 1 shows an example of the configuration of a control device. [Diagram 2] Diagram of the parts of the large intestine. [Diagram 3] 1 shows a first detailed configuration example of a control device. [Figure 4] 13 is a flowchart showing an example of automatic suction control when performing the axis-holding shortening maneuver on the sigmoid colon. [Diagram 5] The first example of judging suction strength. [Figure 6] 10 is a flowchart showing an example of automatic suction control based on an examination condition other than distance. [Figure 7] The second example of judging suction strength. [Figure 8] 3 shows a third detailed configuration example of the control device. [Figure 9] 11 is a flowchart showing an example of automatic air supply control. [Figure 10] 3 shows an example of detailed configurations of an endoscope, a suction device, and an air supply device. [Figure 11] 4 shows a fourth detailed configuration example of the control device. [Figure 12]11 is a flowchart showing an example of luminal direction presentation. [Figure 13] 13 is an example of a lumen direction display screen. [Figure 14] 1 shows an example of an endoscope system configuration. [Figure 15] An example of the configuration of an endoscope and an endoscope shape acquisition sensor. [Figure 16] 5 shows a fifth detailed configuration example of the control device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present embodiment will be described below. Note that the present embodiment described below does not unduly limit the contents described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present disclosure.

[0010] 1. Configuration example 1 shows an example of the configuration of the control device 100. The control device 100 includes an information acquisition unit 140, a determination unit 160, and a control unit 170. Here, a basic example of the configuration of the control device that performs suction control will be described. A detailed example of the configuration of the control device and the correspondence between the detailed example and the basic example will be described later.

[0011] The information acquisition unit 140 acquires an endoscopic image IMG, which is an image captured by the endoscope 10. The endoscope 10, also called a scope, is inserted into a patient's body and captures the inside of the patient's body. The endoscopic image IMG is, specifically, each frame image of a video captured by the endoscope 10. The endoscope 10 may be inserted into the body by a doctor, or may be inserted automatically or semi-automatically by an automatic insertion / removal device.

[0012] The determination unit 160 performs a suction necessity determination to determine whether or not suction is necessary based on the endoscopic image IMG. Specifically, the determination unit 160 performs an image recognition process on the endoscopic image IMG to determine whether or not suction is necessary, and outputs a determination signal DTR indicating whether or not suction is necessary based on the recognition result. For example, the determination unit 160 may determine the insertion state by the image recognition process, and then perform a suction necessity determination based on the insertion state. Alternatively, the determination unit 160 may directly obtain a determination signal DTR from the endoscopic image IMG by the image recognition process. Note that although the suction necessity determination has been described here, the determination unit 160 may further determine the strength of suction, as described later.

[0013] When the determination unit 160 determines that suction is necessary, the control unit 170 performs control to perform suction. Specifically, when the control unit 170 receives a determination signal DTR indicating that suction is necessary, the control unit 170 outputs a control signal CNT to turn on suction to the suction device 30, and when the control unit 170 receives a determination signal DTR indicating that suction is not necessary, the control unit 170 outputs a control signal CNT to turn off suction to the suction device 30. The endoscope 10 has a suction tube for sucking gas, liquid, or tissue pieces in the body cavity. The suction device 30 is connected to the suction tube, and performs suction when a control signal CNT for suction on is input, and does not perform suction when a control signal CNT for suction off is input.

[0014] According to this embodiment, since the necessity of suction is judged based on the endoscopic image, it is possible to automatically control the on / off of suction appropriately according to the situation recognized from the endoscopic image. This enables suction control that is difficult to achieve only by controlling the internal pressure to be constant. This embodiment can be applied to various situations, but as one example, suction can be performed when the endoscopic image recognizes a situation in which the lumen is too open during insertion. Alternatively, suction can be turned off when the endoscopic image recognizes a situation in which the endoscope tip and the inner wall of the lumen are approaching each other due to suction during insertion, or a red ball state in which the image turns red because the distance between the endoscope tip and the mucosa is too close.

[0015] The determination unit 160 determines whether or not suction is required depending on the distance between the mucosa of the subject and the tip of the endoscope 10 based on the endoscopic image IMG. Specifically, the determination unit 160 determines whether or not the distance between the mucosa of the subject and the tip of the endoscope 10 is greater than a predetermined distance based on the endoscopic image IMG, and determines that suction is required when the distance is greater than the predetermined distance. However, the determination unit 160 may obtain the determination result of whether or not suction is required directly from the endoscopic image IMG by performing an image recognition process on the endoscopic image IMG in which the determination result of whether or not suction is required changes depending on the distance, without calculating the distance itself. The predetermined distance is, for example, an optimal distance at which the folds of the large intestine can be turned up by angle operation and torque operation of the endoscope 10 in the axis holding shortening method.

[0016] According to this embodiment, suction can be automatically controlled appropriately according to the distance between the mucosa of the subject and the tip of the endoscope 10, as recognized from the endoscopic image IMG. When operating while maintaining an appropriate distance between the mucosa of the subject and the tip of the endoscope 10, such as in the axis-maintaining shortening method in a large intestine examination, suction can be automatically controlled to maintain that appropriate distance.

[0017] The determination unit 160 determines whether or not suction is necessary based on the change in the endoscopic image IMG when suction is performed. The "change in the endoscopic image IMG when suction is performed" refers to the change over time in the endoscopic image IMG before and after suction is performed, during suction, or both. Specifically, the determination unit 160 recognizes what kind of change has occurred in the endoscopic image IMG by image recognition processing on the endoscopic image IMG, and determines whether or not suction is necessary depending on the recognized change. However, the determination unit 160 may obtain the result of the determination of whether or not suction is necessary directly from the endoscopic image IMG by performing image recognition processing on the endoscopic image IMG in which the result of the determination of whether or not suction is necessary changes depending on what kind of change has occurred in the endoscopic image IMG, without recognizing the change itself.

[0018] According to this embodiment, suction can be automatically controlled appropriately according to changes in the endoscopic image IMG when suction is performed. As an example, in a case where an appropriate distance is maintained between the mucosa of the subject and the tip of the endoscope 10, such as in the axis-maintaining shortening method in a colon examination, suction can be automatically turned off when it is determined that an appropriate distance is not achieved even with suction or that adhesions or the like are present.

[0019] As an example, when the determination unit 160 determines that the distance between the mucosa of the subject and the tip of the endoscope 10 is not closer than a predetermined distance based on the change in the endoscopic image IMG when suction is performed, it determines not to perform suction. This makes it possible to turn off suction when the optimal distance cannot be achieved by suction for some reason.

[0020] As another example, the determination unit 160 determines whether or not there is adhesion in the subject based on the change in the endoscopic image IMG when suction is performed, and if it is determined that there is adhesion, it determines not to perform suction. This makes it possible to turn off suction when the optimal distance cannot be achieved due to adhesion.

[0021] As another example, the determination unit 160 determines an operation plan showing the operation procedure of the endoscope 10 based on the change in the endoscopic image IMG when suction is performed, and determines whether suction is necessary according to the determined operation plan. This makes it possible to switch between multiple operation plans, such as the axis holding shortening method, the axis holding shortening method considering adhesions, and the loop method, and to perform appropriate automatic suction control for each operation plan.

[0022] The determination unit 160 determines whether or not suction is required depending on the position of the endoscope 10 in the lumen based on the endoscopic image IMG. The position of the endoscope 10 here refers to the position of the tip of the endoscope 10. Specifically, when the lumen is divided into a plurality of regions, the position of the endoscope 10 may refer to the region where the tip of the endoscope 10 is located, or may be a more detailed position within the region. The determination unit 160 determines the region where the tip of the endoscope 10 is located by image recognition processing on the endoscopic image IMG, and determines whether or not suction is required depending on the region. However, the determination unit 160 may obtain the determination result of whether or not suction is required directly from the endoscopic image IMG by performing image recognition processing on the endoscopic image IMG in which the determination result of whether or not suction is required changes depending on the region where the tip of the endoscope 10 is located, without determining the region itself.

[0023] According to this embodiment, suction can be automatically controlled appropriately according to the position of the endoscope 10 in the lumen. As an example, when the axis-maintaining shortening method is used to advance through the sigmoid colon in a large intestine examination, when the endoscope 10 is positioned in the sigmoid colon, the distance from the tip of the endoscope 10 can be appropriately maintained by automatically controlling suction.

[0024] As an example, when the determination unit 160 determines that the endoscope 10 is located at a predetermined portion in the lumen based on the endoscopic image IMG, the determination unit 160 determines that suction is necessary. The predetermined portion is a portion beyond the mittrans in the sigmoid colon or the transverse colon.

[0025] There are less painful insertion methods for the free colon, such as the axis-holding shortening method. In such insertion methods, insertion is advanced while maintaining an appropriate distance between the tip of the endoscope 10 and the intestinal wall by suction. According to this embodiment, the automatic suction control maintains an appropriate distance between the tip of the endoscope 10 and the intestinal wall, so that the suction technique of an expert can be reproduced, or it becomes possible to concentrate on the insertion procedure.

[0026] When the determination unit 160 determines, based on the endoscopic image IMG, that the tip of the endoscope 10 is in a red-ball state, in which the tip is in contact with the mucous membrane of the subject, it determines not to perform suction.

[0027] If suction is performed in the red-ball state, there is a possibility that the mucosa or intestinal wall may be damaged. According to the present embodiment, since suction is turned off in the red-ball state, it is possible to prevent the mucosa or intestinal wall from being damaged by suction.

[0028] When it is determined based on the endoscopic image IMG that the lumen is in an over-air-supply state in which the lumen has expanded by air supply to a predetermined level or more, the determination unit 160 determines to perform suction.

[0029] In an over-air supply state, the patient may experience pain or discomfort. According to the present embodiment, since suction is turned off in an over-air supply state, pain or discomfort to the patient due to over-air supply can be prevented.

[0030] The determination unit 160 may acquire examination status information on the status of an endoscopic examination using the endoscope 10 based on the endoscopic image IMG, and may determine whether or not suction is necessary based on the examination status information. The examination status information is information indicating the current examination status, a time-series examination status including the current examination status, or a past examination status, and includes, for example, information indicating the insertion status of the endoscope, information indicating the operation status, or information indicating the state of the intestines. As will be described later in the first detailed configuration example, the examination status information is obtained as, for example, a combination pattern of multiple parameters indicating the examination status, and the determination unit 160 determines whether or not suction is necessary by referring to a control table that associates the pattern with a suction control determination.

[0031] According to this embodiment, the examination situation including the insertion status, operation status, and intestinal condition of the endoscope are recognized from the endoscopic image, thereby realizing appropriate automatic suction control according to the examination situation.

[0032] Although the case where the suction necessity determination is performed as the automatic suction control has been described above, the suction strength determination may further be performed. That is, the determination unit 160 performs the suction strength determination to determine the strength of the suction based on the endoscopic image IMG. The control unit 170 performs the control to perform the suction with the strength determined by the determination unit 160. Specifically, the determination unit 160 performs the image recognition process on the endoscopic image IMG to determine whether the suction should be strong or weak, and outputs the determination signal DTR indicating the strength of the suction based on the recognition result. For example, the determination unit 160 may determine the insertion status by the image recognition process, and then perform the suction strength determination based on the insertion status. However, the determination unit 160 may obtain the determination result of the suction strength directly from the endoscopic image IMG by the image recognition process without determining the insertion status.

[0033] According to this embodiment, since the suction strength is determined based on the endoscopic image, it is possible to automatically control the strength of suction appropriately according to the situation recognized from the endoscopic image. This enables suction control that is difficult to achieve by only controlling the internal pressure to be constant. This embodiment can be applied to various situations, but as one example, when the endoscopic image recognizes a situation in which the tip of the endoscope is far from the inner wall of the lumen at the time of insertion, the suction can be strengthened. Then, when the endoscopic image recognizes a situation in which the tip of the endoscope and the inner wall of the lumen are approaching each other due to suction, the suction can be weakened.

[0034] As an example, the determination unit 160 determines the strength of suction based on the endoscopic image IMG in accordance with the distance between the mucosa of the subject and the tip of the endoscope 10. Specifically, the determination unit 160 weakens the suction as the distance between the mucosa of the subject and the tip of the endoscope 10 becomes closer due to suction.

[0035] According to this embodiment, more appropriate intestinal control is possible by adjusting the strength of suction, compared to intestinal control by only turning suction on and off. This realizes automatic suction control that is closer to the suction technique of an expert.

[0036] As another example, the judgment unit 160 recognizes at least one of residue, water, resected tissue, or sprayed medicinal liquid based on the endoscopic image IMG, and judges whether suction is necessary and the strength of suction based on the recognition result.

[0037] According to this embodiment, when residue, water, excised tissue, or sprayed medicinal liquid is present, it is possible to aspirate them by turning on suction. In addition, the strength of suction can be automatically controlled according to the amount of residue, water, excised tissue, or sprayed medicinal liquid.

[0038] Although the automatic suction control based on the endoscopic image IMG has been described above, the automatic suction control may be performed using other information. As an example, the information acquisition unit 140 may acquire insertion shape information that is information representing the shape of the insertion part of the endoscope 10. The determination unit 160 may determine whether or not suction is necessary based on the endoscopic image IMG and the insertion shape information.

[0039] According to this embodiment, the position and shape of the insertion portion, or changes therein, can be directly determined in judging the examination status. For example, the location where the endoscope 10 is located, the operation performed by the doctor, the amount of operation, the procedure during the operation, and the shape of the insertion portion during the operation can be determined. This improves the accuracy of judging the examination status.

[0040] Although the automatic suction control has been described above, automatic air supply control may be further performed as described later in the third detailed configuration example. That is, the determination unit 160 may determine whether or not air supply is necessary based on the endoscopic image IMG. The control unit 170 may perform control to perform air supply when the determination unit 160 determines that air supply is necessary.

[0041] In order to perform examinations that place less of a burden on patients, intestines are controlled by suction, but in some situations, excessive suction can cause the intestines to collapse, making operation difficult. According to this embodiment, automatic air supply control can be performed to create the most suitable conditions for operation.

[0042] The processing performed by the control device 100 described above may be implemented as a control method as follows. The control method is not limited to being implemented by the control device 100, but may be implemented by various systems or devices such as an endoscope system described below. The control method includes acquiring an endoscopic image IMG which is an image captured by the endoscope 10, making a suction necessity determination based on the endoscopic image IMG to determine whether or not suction is necessary, and performing control to perform suction when it is determined in the suction necessity determination that suction is necessary.

[0043] Furthermore, a part or the whole of the above-described control device 100 may be realized by a processor. In that case, the control device 100 may be configured as follows.

[0044] The control device 100 includes a memory that stores information, and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data. The program describes some or all of the functions of the information acquisition unit 140, the determination unit 160, and the control unit 170. The processor executes the program to realize some or all of the functions of the information acquisition unit 140, the determination unit 160, and the control unit 170.

[0045] The processor includes hardware, and the hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the processor may be configured with one or more circuit devices mounted on a circuit board, or one or more circuit elements. The one or more circuit devices may be, for example, ICs. The one or more circuit elements may be, for example, resistors, capacitors, etc. The processor may be, for example, a central processing unit (CPU). However, the processor is not limited to a CPU, and various processors such as a graphics processing unit (GPU) or a digital signal processor (DSP) may be used. The processor may also be an integrated circuit device such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processor may also include an amplifier circuit, a filter circuit, etc. that process analog signals. The memory may be a semiconductor memory such as an SRAM or a DRAM, a register, a magnetic storage device such as a hard disk device, or an optical storage device such as an optical disk device. For example, the memory stores computer-readable instructions, and the functions of each part of the control device 100 are realized as processing by the processor executing the instructions. The instruction here may be an instruction from an instruction set that constitutes a program, or an instruction that instructs the hardware circuitry of the processor to operate.

[0046] The program can be stored in a non-transitory information storage medium that is a computer-readable medium. The information storage medium can be realized by, for example, an optical disk, a memory card, a HDD, or a semiconductor memory. The semiconductor memory can be, for example, a ROM or a non-volatile memory.

[0047] 2. First detailed configuration example An example in which the control device 100 is applied to an endoscopic examination of the large intestine will be described below, but the application of the control device 100 is not limited to endoscopic examination of the large intestine.

[0048] FIG. 2 is an explanatory diagram of the parts of the large intestine. The parts of the large intestine between the cecum and the anus include the ascending colon, hepatic flexure, transverse colon, splenic flexure, descending colon, SDJ, sigmoid colon, and rectum. The area near the hepatic flexure is also called the right colon, and the area near the splenic flexure is also called the left colon. SDJ stands for Sigmoid Descending colon Junction, and is the boundary between the sigmoid colon and the descending colon. The ascending colon, descending colon, and rectum are fixed colons, and the transverse colon and sigmoid colon are free colons.

[0049] In endoscopic examinations, highly advanced techniques are required to perform high-quality endoscopic examinations that are less invasive for patients. One of the important endoscopic operations for performing such examinations is the suction operation. The suction operation is not only used to suck out air and water from within the intestinal tract, but also plays a role in (1) recognizing the condition of the intestine and (2) controlling the intestine.

[0050] As an example of (1), when the lumen of the sigmoid colon appears like a clay pipe, suction is first performed to check whether the intestine is approaching due to suction. If the intestine is approaching, the axis-holding shortening method is selected, in which the intestine is shortened while being inserted, and if the intestine is not approaching, the loop method is selected, in which the scope is inserted while being pushed.

[0051] As an example of (2), when shortening the sigmoid colon and inserting the endoscope past the folds, the tip of the endoscope is advanced while applying suction, bringing the tip of the endoscope relatively close to the intestinal wall. This prevents the intestinal wall from moving away or stretching due to pushing, torque or angle operations. Furthermore, when controlling the intestine, experienced doctors finely adjust the amount of suction by not only pressing the suction button all the way, but also pressing it halfway or 1 / 4 of the way.

[0052] The suction operation for these purposes is very difficult for a junior doctor. The control device 100 of this embodiment assists the suction function, allowing even a junior doctor to perform the same operations as a veteran doctor, and providing a high-quality examination with less burden on the patient. In an endoscopic examination of the large intestine, after an endoscope is inserted from the rectum to the cecum, air is supplied to inflate the intestinal tract, and observation is performed while the endoscope is removed. The control device 100 of this embodiment may control the suction both during insertion and observation, or may control the suction only during insertion.

[0053] A detailed configuration example of the control device 100 of this embodiment will be described below. Fig. 3 shows a first detailed configuration example of the control device 100. The control device 100 includes an image acquisition unit 141, an examination status determination unit 161, a control determination unit 162, and a control unit 170. In this configuration example, the image acquisition unit 141 corresponds to the information acquisition unit 140 in Fig. 1, and the examination status determination unit 161 and the control determination unit 162 correspond to the determination unit 160 in Fig. 1. Note that descriptions of components that are the same as those already described will be omitted as appropriate.

[0054] The image acquisition unit 141 acquires the endoscopic image IMG by receiving the endoscopic image IMG transmitted by the endoscope 10. The endoscopic image IMG is input as information IFIN to the examination status determination unit 161. Note that suction status information acquired from the suction device may be further input to the examination status determination unit 161 as information IFIN.

[0055] The inspection status determination unit 161 determines the latest status and the time-series status from the information IFIN, and outputs latest status information TYK and time-series status information JIK. The inspection status determination unit 161 includes a latest status determination unit 121 and a time-series status determination unit 122.

[0056] The immediate situation determination unit 121 recognizes the immediate situation of the endoscope insertion and the intestine from the information IFIN, and outputs the result as immediate situation information TYK. The immediate situation is the situation recognized from one endoscopic image IMG among the moving images captured by the endoscope 10.

[0057] The time-series situation determination unit 122 recognizes the time-series insertion situation of the endoscope and the intestine situation from the information IFIN, and outputs the result as time-series situation information JIK. The time-series situation is a situation recognized from a plurality of endoscopic images IMG in a time series among the moving images captured by the endoscope 10.

[0058] The control determination unit 162 determines whether to control the suction operation based on the most recent situation information TYK, the time-series situation information JIK, or both. The control of the suction operation refers to at least one of the following controls: turning suction on and off, changing the strength of suction, or switching the suction path. Switching the suction path refers to switching between an automatic suction mode in which suction is automatically controlled based on information from the time-series situation determination unit 122, and a manual suction mode in which the doctor manually operates the suction. The endoscope 10 is provided with a suction button, and the control determination unit 162 switches from the automatic suction mode to the manual suction mode when it detects that the doctor has intentionally pressed the suction button. In the manual suction mode, the automatic suction control by the control device 100 is disabled, and the suction device 30 operates under manual control using the suction button.

[0059] As described above, the control device 100 recognizes the current examination situation and the state of the intestine from the endoscopic image IMG, and assists insertion by adjusting whether or not to perform suction, or whether or not to perform suction and the amount of suction according to the situation. In endoscopic examination, suction is related to intestine control, determination of operation plan, removal of residue, or prevention of excessive air supply during insertion, and is related to removal of water, removal of medicinal liquid, or removal of tumor during observation or treatment. Intestine control includes prevention of extension of the free colon, prevention of pain, etc. By assisting with these suctions, the control device 100 can provide more appropriate examinations that are less painful for patients, and an examination environment that reduces the burden on doctors.

[0060] As an example, during the axis-holding shortening method, the control device 100 determines the distance between the endoscope tip and the intestinal wall from the endoscopic image, and automatically performs suction so that the distance is optimal. This maintains the distance between the endoscope tip and the intestinal wall optimally. The control device 100 not only performs suction at an appropriate timing, but also automatically adjusts the amount of suction. This realizes automatic suction control that is closer to the suction operation of an expert than control by only turning suction on and off. The control device 100 also adjusts whether or not to perform suction, or whether or not to perform suction and the amount of suction, not only during the axis-holding shortening method, but also in various situations where suction is required. Examples of situations where various suction is required include situations where intestinal fluid has accumulated or where excessive air is being blown. This not only maintains the distance between the endoscope tip and the intestinal wall optimally during the axis-holding shortening method, but also realizes automatic suction control in various situations where suction is required.

[0061] In the above-mentioned Patent Document 1, a pressure sensor is installed in the forceps channel, and the intracavity pressure regulating device of the endoscope automatically performs suction according to the output value of the pressure sensor to keep the intracavity pressure constant. According to this embodiment, the necessity of suction, or the necessity and strength of suction, is determined from the endoscopic image, not the output value of the pressure sensor. This allows automatic suction control to be realized even in situations where suction is required that cannot be determined from the output value of the pressure sensor, such as the optimal distance between the endoscope tip and the intestinal wall in the shaft holding shortening method, or when intestinal fluid has accumulated.

[0062] Suction is an essential operation in colonoscopic examination, and by automatically adjusting the timing and amount of suction, the state of the intestine can be kept appropriate, and it is expected that the degree of pain for the patient will be reduced or the quality of the examination will be improved. For example, when inserting into the free colon, the intestine is pulled in by suction, and an appropriate distance is maintained between the tip of the endoscope and the intestinal wall, thereby reducing the operation that leads to the patient's pain. Furthermore, by automating suction, the doctor can concentrate on other operations and the patient's pain caused by air supply can be prevented. Alternatively, when the endoscope is removed, the intestine is inflated by air supply, but the automatic suction control maintains the state of the intestine suitable for observation and does not cause pain to the patient, allowing high-quality observation. Furthermore, when the observation is completed, it is also possible to assist the observation and reduce the patient's discomfort by suctioning the air in the intestine.

[0063] The inspection status determination unit 161 is realized, for example, by machine learning using a neural network or the like. Specifically, a memory (not shown) stores a program in which an inference algorithm is described and parameters used in the inference algorithm as information of a trained model. Then, the processor performs processing based on the information of the trained model. That is, the processor executes the processing of the inspection status determination unit 161 by executing the program using the parameters stored in the memory. Note that the entire inspection status determination unit 161 may be realized by one trained model, or each of the latest status determination unit 121 and the time-series status determination unit 122 may be realized by an individual trained model.

[0064] As the inference algorithm, for example, a neural network can be adopted. The weight coefficient of the connection between nodes in the neural network is a parameter. The neural network includes an input layer to which input data is input, an intermediate layer that performs arithmetic processing on the data input through the input layer, and an output layer that outputs a recognition result based on the arithmetic result output from the intermediate layer. The inference algorithm is not limited to a neural network, and various machine learning techniques used in recognition processing can be adopted. The input data is information IFIN, and the recognition result is the latest situation information TYK and the time-series situation information JIK. The learning device that executes the learning process is, for example, an information processing device such as a PC. The learning device inputs the teacher data into the learning model, and generates a learned model by feeding back the learning model based on the recognition result. The teacher data includes multiple sets of data, and each set includes input data and correct answer data. The correct answer data is the recognition result to be obtained for the input data, and is prepared in advance by a medical professional or the like.

[0065] In addition, not only the inspection status determination unit 161 but also the control determination unit 162 may be realized by machine learning. In this case, the recognition result output from the output layer of the neural network is the determination signal DTR. Alternatively, the recognition result may be the determination signal DTR, the latest status information TYK, and the time-series status information JIK.

[0066] One example of the case where the control determination unit 162 performs control determination using an algorithm other than machine learning is a method of using a control table. Specifically, the control determination unit 162 stores a control table in which combination patterns of multiple parameters indicating the test status are associated with suction control determinations. The test status determination unit 161 outputs a determination result for each parameter from the information IFIN, and the control determination unit 162 performs suction control determination by referring to the control determination result corresponding to the determination result of the multiple parameters output by the test status determination unit 161 from the table.

[0067] The multiple parameters indicating the examination status correspond to the most recent status information TYK and the time-series status information JIK, and examples include the location where the endoscope tip is located, the distance between the endoscope tip and the intestinal wall, changes in that distance, the currently performed endoscope insertion method, the currently performed endoscopic operation, the presence or absence of adhesions, the presence or absence of water, the presence or absence of residue, and whether or not the intestine is in a red ball state. For example, in the control table, a suction-on control decision is associated with a parameter pattern in which the location is the sigmoid colon, the insertion method is the axis-holding shortening method, and the distance is greater than a predetermined value. The control table is configured to realize a desired suction control, without being limited to this.

[0068] 3. Second detailed configuration example As a second detailed configuration example, a specific example of automatic suction control will be shown. The configuration of the control device 100 is the same as that shown in FIG.

[0069] FIG. 4 is a flow chart showing an example of automatic suction control when performing the axis-holding shortening maneuver on the sigmoid colon.

[0070] In step S1, the image acquisition unit 141 acquires an endoscopic image IMG from the endoscope 10. In step S2, the examination status determination unit 161 determines a part from the endoscopic image IMG. The examination status determination unit 161 recognizes a part from the endoscopic image IMG by utilizing the fact that image features differ depending on the part. When using insertion shape information described later, the examination status determination unit 161 may acquire the length of the part of the endoscope insertion unit that is inserted into the patient from the insertion shape information, and estimate the part from the insertion length.

[0071] If the examination status determination unit 161 determines in step S2 that the endoscope 10 is located in a region other than the sigmoid colon, then in step S9, the control determination unit 162 determines to turn off suction, and the control unit 170 outputs a control signal CNT for turning off suction.

[0072] If the examination status determination unit 161 determines in step S2 that the endoscope 10 is located outside the sigmoid colon, then in step S3 the examination status determination unit 161 determines from the endoscopic image IMG whether the distance between the tip of the endoscope 10 and the intestinal wall is an optimal distance. The optimal distance is the optimal distance between the insertion part and the intestinal wall that allows the folds to be turned up by angle manipulation, torque manipulation, or both.

[0073] When the examination status determination unit 161 determines in step S3 that the distance between the tip of the endoscope 10 and the intestinal wall is optimal, the control determination unit 162 determines to turn off the suction in step S4, and the control unit 170 outputs a control signal CNT for turning off the suction. Then, the process returns to step S1.

[0074] In step S3, when the examination status determination unit 161 determines that the distance between the tip of the endoscope 10 and the intestinal wall is not optimal, in step S5, the control determination unit 162 determines that suction is on, and the control unit 170 outputs a control signal CNT for suction on.

[0075] In step S6, the control determination unit 162 performs a suction strength determination. FIG. 5 shows a first example of the suction strength determination. The control determination unit 162 determines that the suction should be gradually weakened as the distance between the tip of the insertion unit 14 and the intestinal wall approaches the optimal distance, and the control unit 170 outputs a control signal CNT that indicates the amount of suction according to the determination result. FIG. 5 shows an example of controlling the strength of suction in two stages, where the suction is set to strong when the distance is A in the clay pipe shape, and the suction is set to weak when the distance is B smaller than the distance A in the clay pipe shape. As shown in FIG. 5, the intestinal tract is folded as if it is deflated by suction, and the way the intestinal tract is imaged changes depending on the degree of deflation. When the intestinal tract is in the clay pipe shape, it is imaged as a lumen, and when it is at the optimal distance, it is imaged as a folded intestinal wall. From the way the intestinal tract is imaged, the distance between the tip of the insertion unit 14 and the intestinal wall can be estimated. For example, the distance is the distance between the tip of the insertion unit and the intestinal wall in the optical axis direction of the imaging unit. Alternatively, the distance may be determined comprehensively from the distances between the tip of the insertion portion and the intestinal wall in various directions. When the optimal distance is achieved by suction, suction is turned off in step S4.

[0076] In step S7, the examination status determination unit 161 determines the presence or absence of adhesions from changes in the endoscopic image IMG during suction. Since adhesions in the intestinal tract are unlikely to change even when suctioned, the presence or absence of adhesions can be recognized from the image.

[0077] If the examination status determining unit 161 determines in step S7 that no adhesion is present, the process returns to step S1.

[0078] If the examination status determination unit 161 determines in step S7 that adhesion exists, the examination status determination unit 161 changes the operation plan from the axis holding shortening method to the loop method in step S8. In step S9, in response to the operation plan being changed to the loop method, the control determination unit 162 determines to turn off suction.

[0079] FIG. 6 is a flowchart showing an example of automatic suction control based on an examination condition other than the distance.

[0080] In step S20, the image acquisition unit 141 acquires an endoscopic image IMG from the endoscope 10. In step S21, the examination status determination unit 161 determines whether or not the endoscopic image IMG is in a red-ball state. The red-ball state is a state in which a part or all of an image turns red due to the objective lens provided at the tip of the endoscope 10 coming into contact with a mucous membrane, and the examination status determination unit 161 can recognize the red-ball state from the image.

[0081] If the inspection status determination unit 161 determines that the state is a red ball in step S20, the control determination unit 162 determines that the suction is off in step S22, and the control unit 170 outputs a suction-off control signal CNT. Then, the process returns to step S20.

[0082] If the examination status determination unit 161 determines in step S20 that the state is not a red ball state, the examination status determination unit 161 determines in step S23 whether or not there is excessive gas supply. Excessive gas supply can be recognized from the way the image is captured depending on the thickness of the lumen or the stretched state of the folds captured in the image.

[0083] If the examination status determination section 161 determines in step S23 that excessive air is being supplied, then in step S25 the control determination section 162 determines that suction is on, and the control section 170 outputs a suction-on control signal CNT.

[0084] If the examination status determination unit 161 determines in step S23 that there is no excessive air supply, then in step S24 the examination status determination unit 161 determines whether water or the like is present. The water or the like is water, residue, a medicinal solution, or tissue fragments. A tissue fragment is a polyp or a tumor that has been removed during observation or treatment. The water or medicinal solution is injected into the lumen by a syringe from a port provided on the endoscope, for example. Alternatively, the water is injected into the lumen by a water jet controlled by a foot switch.

[0085] If the examination status determination section 161 determines in step S24 that water, residue, or tissue particles are present, then in step S25 the control determination section 162 determines that suction is on, and the control section 170 outputs a suction-on control signal CNT.

[0086] In step S26, the control determination unit 162 performs a suction strength determination. FIG. 7 shows a second example of suction strength determination. The examination status determination unit 161 recognizes the presence of a tissue piece and the size of the tissue piece from the endoscopic image. The control determination unit 162 determines that the suction should be stronger as the size of the tissue piece recognized by the examination status determination unit 161 becomes larger. The control unit 170 outputs a control signal CNT that indicates the amount of suction according to the determination result. Note that if the size of the tissue piece is equal to or larger than a predetermined value, the tissue piece cannot pass through the suction tube, and the control determination unit 162 determines that the suction is turned off. As another example of suction strength determination, the control determination unit 162 determines that the suction should be stronger as the amount of water or residue shown in the endoscopic image becomes greater.

[0087] After step S26, or when the control determination unit 162 determines in step S24 that no water, residue, or tissue particles are present, in step S27, the examination status determination unit 161 determines whether the examination has ended. The examination status determination unit 161 determines that the examination has ended based on input information from the doctor or when a predetermined state is recognized from the endoscopic image. The predetermined state is when the endoscope 10 has been removed to the rectum or when the endoscope 10 has come out of the body.

[0088] If the examination status determination unit 161 determines in step S27 that the examination has not ended, the process returns to step S20.

[0089] If the examination status determination unit 161 determines in step S27 that the examination has ended, then in step S28 the control determination unit 162 determines that suction is on, and the control unit 170 outputs a suction-on control signal CNT. In step S29, after the swelling of the intestinal tract caused by the gas supply is released by suction, the control determination unit 162 determines that suction is off, and the control unit 170 outputs a suction-off control signal CNT.

[0090] As described above, the control device 100 of this embodiment can prevent intestinal stretching or pain by attracting the intestinal wall by suction when the endoscope 10 is inserted into the intestine in the non-fixed part. The control device 100 can also prevent red balls by controlling suction, or improve insertability by adjusting the amount of air. The control device 100 can also improve insertability by automatically controlling suction when it is determined that water or residue, etc., will impair visibility and interfere with insertion when the endoscope is inserted into the intestine in the non-fixed part or fixed part.

[0091] The above-described suction control is an example. As another example, when the examination status determination unit 161 determines that the tip of the endoscope 10 has passed the mitt transformer during the mitt transformer operation of the transverse colon, the control determination unit 162 determines that suction is on. The intestinal wall can be brought closer to the tip of the endoscope 10 by suction, making it possible to set the distance suitable for mitt transformer operation.

[0092] 4.Third detailed configuration example 8 shows a third detailed configuration example of the control device 100. In this configuration example, the control device 100 performs not only the automatic suction control but also the automatic air supply control. The automatic suction control has been described above, and the automatic air supply control will be mainly described below.

[0093] FIG. 9 is a flowchart showing an example of the automatic air supply control.

[0094] In step S40, the image acquisition unit 141 acquires an endoscopic image IMG from the endoscope 10. In step S41, the examination status determination unit 161 determines a region from the endoscopic image IMG.

[0095] If the examination status determination unit 161 determines in step S41 that the endoscope 10 is located in a region other than the sigmoid colon, then in step S42 the control determination unit 162 determines that gas supply is off, and the control unit 170 outputs a control signal CNT for turning off gas supply. The gas supply device 40 turns off gas supply upon receiving the control signal CNT. Note that although the control signal for gas intake and the control signal for gas supply are collectively referred to as CNT, gas intake and gas supply are controlled independently.

[0096] In step S41, if the examination status determination unit 161 determines that the endoscope 10 is located outside the sigmoid colon, in step S44, the examination status determination unit 161 determines from the endoscopic image IMG whether the distance between the tip of the endoscope 10 and the intestinal wall is an optimal distance.

[0097] When the examination status determination unit 161 determines in step S44 that the distance between the tip of the endoscope 10 and the intestinal wall is optimal, the control determination unit 162 determines to turn off the gas supply in step S46, and the control unit 170 outputs a control signal CNT to turn off the gas supply. The gas supply device 40 turns off the gas supply upon receiving the control signal CNT. Then, the process returns to step S1.

[0098] When the examination status determination unit 161 determines in step S44 that the distance between the tip of the endoscope 10 and the intestinal wall is not optimal, the control determination unit 162 determines in step S45 that gas supply is on, and the control unit 170 outputs a control signal CNT for gas supply on. The gas supply device 40 turns on gas supply upon receiving the control signal CNT. Then, the process returns to step S1.

[0099] In order to perform examinations that place less of a burden on the patient, the intestines are controlled by suction, but in some situations, excessive suction can cause the intestines to collapse, making operation difficult. In this embodiment, automatic air supply control is performed, making it possible to create the most suitable conditions for operation.

[0100] 5. Endoscopes, suction devices, and air supply devices Fig. 10 shows an example of the detailed configuration of the endoscope 10, the suction device 30, and the gas supply device 40. In Fig. 10, configurations unrelated to suction and gas supply are omitted as appropriate. For example, the illumination device, imaging device, angle operation dial, and the like of the endoscope 10 are omitted. Note that the configuration shown in Fig. 10 is only an example, and the configurations of the endoscope 10, the suction device 30, and the gas supply device 40, or the connection relationships between the components, are not limited to those shown in Fig. 10.

[0101] The endoscope 10 includes a suction operation button 54, a suction operation detection sensor 55, an air supply operation button 64, an air supply operation detection sensor 65, a port 53, an insertion section 14, a suction port 51, a suction tube 52, an air supply port 61, and an air supply tube 62.

[0102] The suction tube 52 is provided in the insertion section 14, one end of which is connected to a suction port 51 provided at the tip of the insertion section 14, and the other end of which is connected to a port 53. The port 53 is provided in order to insert a treatment tool such as forceps into the body cavity from the suction port 51 via the suction tube 52, or to spray a medicinal liquid or the like into the body cavity from the suction port 51 via the suction tube 52 using a syringe. The suction tube 52 branches near the port 53, and the branched suction tube 52 is connected to the suction device 30.

[0103] The air supply tube 62 is provided within the insertion portion 14, the endoscope gripping portion, and the universal cord, and one end of the air supply tube 62 is connected to an air supply port 61 provided at the tip of the insertion portion 14, and the other end is connected to the air supply device 40.

[0104] The suction operation button 54, the air supply operation button 64, the suction operation detection sensor 55, and the air supply operation detection sensor 65 are provided on the endoscope gripping portion. The suction operation detection sensor 55 detects pressing of the suction operation button 54, and outputs a detection signal to the control determination unit 162. The air supply operation detection sensor 65 detects pressing of the air supply operation button 64, and outputs a detection signal to the control determination unit 162.

[0105] The suction device 30 includes a suction control circuit 31, a suction pump 32, a suction tank 34, a solenoid valve 35, a suction amount measuring unit 37, and a pressure sensor 36.

[0106] The suction pipe 52 branching off from near the port 53 is connected to the suction pump 32 via the suction amount measuring unit 37, the solenoid valve 35, and the suction tank 34. The suction control circuit 31 includes a drive circuit for the suction pump 32 and a drive circuit for the solenoid valve 35, and controls the suction on / off and the strength of the suction by driving the suction pump 32 and the solenoid valve 35 in response to a control signal CNT output by the control unit 170. The suction amount measuring unit 37 measures the amount of gas or liquid sucked by the suction pump 32, and outputs the measurement value to the examination status determination unit 161. The suction amount measuring unit 37 is, for example, a flow rate sensor.

[0107] The pressure sensor 36 is connected to the suction tube 52 branching off from the vicinity of the port 53. The pressure sensor 36 measures the pressure in the suction tube 52 and outputs the measurement value to the examination status determination unit 161.

[0108] The air supply device 40 includes an air supply control circuit 41 , an air supply pump 42 , and an electromagnetic valve 45 .

[0109] The air supply pipe 62 is connected to the air supply pump 42 via the solenoid valve 45. The air supply control circuit 41 includes a drive circuit for the air supply pump 42 and a drive circuit for the solenoid valve 45, and controls the air supply on and off by driving the air supply pump 42 and the solenoid valve 45 in response to a control signal CNT output by the control unit 170.

[0110] In path switching, the control determination unit 162 switches from the automatic suction mode to the manual suction mode when it recognizes that the suction operation button 54 has been operated based on a detection signal from the suction operation detection sensor 55. In the manual suction mode, the control unit 170 outputs the detection signal from the suction operation detection sensor 55 to the suction control circuit 31 as a control signal CNT.

[0111] Furthermore, when the control determination unit 162 recognizes that the air supply operation button 64 has been operated based on a detection signal from the air supply operation detection sensor 65, the automatic air supply mode may be switched to the manual air supply mode. In the manual air supply mode, the control unit 170 outputs the detection signal from the air supply operation detection sensor 65 to the air supply control circuit 41 as a control signal CNT. Note that in the first and second detailed configuration examples, the switching of the air supply path may be omitted, and the configuration may be such that only the manual air supply mode is performed. That is, the detection signal from the air supply operation detection sensor 65 may be input directly to the air supply control circuit 41.

[0112] 6.Fourth detailed configuration example 11 is a fourth detailed configuration example of the control device 100. In this configuration example, the control device 100 further includes a support information generating unit 150. The automatic suction control is as described above, and the support information generation will be mainly described below. Note that automatic air supply control may be further combined as in the third detailed configuration example.

[0113] The support information generating unit 150 generates the support information AST according to the information IFIN. The support information AST is information that supports the insertion of the endoscope into the patient. Specifically, the support information AST is information presented or notified to a medical professional or an insertion / removal device that performs an operation to insert an endoscope into a patient, and is information indicating what operation should be performed next for the current insertion status, or information for a doctor to understand the current insertion status, etc. The support information generating unit 150 grasps the insertion status of the endoscope from the latest status information TYK and the time-series status information JIK, and generates the support information AST according to the insertion status. The insertion status is a predetermined insertion status that appears in the procedure of proceeding with the endoscopic examination, and is specified by, for example, a predetermined insertion part position, a predetermined change in the insertion part position, a predetermined insertion part shape, a predetermined change in the insertion part shape, a predetermined endoscope operation, or a combination of any two or more of them. Note that the information on the insertion part shape, the change in the insertion part shape, and the endoscope operation may be acquired by the insertion shape information described later, etc.

[0114] 11 shows an example in which the support information generating unit 150 displays characters, symbols, images, etc. as the support information AST on the display device 220. However, the method of presenting the support information AST is not limited to images, and may be sound, voice, light, vibration, etc. Alternatively, when an automatic insertion / removal device is used, a control signal for causing the automatic insertion / removal device to operate or process may be input to the automatic insertion / removal device as the support information AST. Note that the presentation of the support information AST and the automatic suction control may be performed simultaneously, or only one of the presentation of the support information AST and the automatic suction control may be operated depending on the situation.

[0115] A specific example of support information presentation will be described. There are various insertion methods for inserting into a non-fixed part of the intestine, such as the sigmoid colon or the transverse colon, and each insertion method has its own insertion status, such as the shape of the insertion part or operation, that appears when proceeding with the insertion procedure. The operation to be performed next is determined for each insertion status. The support information generating unit 150 generates support information AST according to the insertion procedure.

[0116] In this embodiment, when the insertion progresses to a non-fixed portion of the intestine such as the sigmoid colon or the transverse colon, not only is the intestine controlled by automatic suction, but operation support information can also be presented. This allows further improvement of insertability and insertion support. The support information may be generated and presented using information on the situation that becomes clear only after suction. The situation that becomes clear only after suction is, for example, the presence or absence of adhesions or an available insertion method, and an insertion plan or the like according to that situation may be presented as support information. The available insertion method is, for example, the loop method or the axis-holding shortening method.

[0117] As a first example of the presentation of support information, in an endoscopic operation in the sigmoid colon, the control device 100 presents support information AST or performs automatic suction control as follows. When the examination status determination unit 161 determines that an operation that requires a suction operation, such as the axis holding shortening method, the control determination unit 162 and the control unit 170 perform automatic suction, and the support information generation unit 150 presents operation support information. When the examination status determination unit 161 recognizes that the endoscope 10 has entered the sigmoid colon, the support information generation unit 150 performs the suction operation. of Alternatively, the control determination unit 162 and the control unit 170 automatically perform suction.

[0118] In addition, the support information generating unit 150 checks the status of the suction and changes the content of the support information AST to be presented next. When the intestinal wall approaches by suction and the optimal distance is maintained, the support information generating unit 150 presents a guide for the axis holding shortening method. Until the suction changes from the clay pipe shape to the optimal distance, the support information generating unit 150 presents a guide for the suction operation in addition to the axis holding operation. At this time, automatic suction may be performed. When the display becomes a red ball, the support information generating unit 150 presents a suggestion to stop suction, or the control determining unit 162 and the control unit 170 automatically stop suction. When the intestinal wall does not approach even with suction, the support information generating unit 150 presents a guide for the Push method such as the N-loop method. The support information generating unit 150 successively changes the guide content according to the situation at that time.

[0119] As a second example of the support information presentation, when the examination status determination unit 161 determines that the operation requires intestinal control by suction, such as a transverse colon shortening operation, the control determination unit 162 and the control unit 170 perform automatic suction, and the support information generation unit 150 presents operation support information. When the examination status determination unit 161 determines that the mittrans has been passed, the support information generation unit 150 presents a suction operation, or the control determination unit 162 and the control unit 170 automatically perform suction.

[0120] Furthermore, the support information generating unit 150 checks the suction status and changes the content of the support information AST to be presented next. If the intestinal wall is brought closer by suction and an optimal distance is maintained, the support information generating unit 150 presents a shortening operation guide using suction. If the intestinal wall does not come closer even with suction, the support information generating unit 150 presents a shortening operation guide that does not use suction. The support information generating unit 150 successively changes the guide content according to the situation at that time.

[0121] A third example of the support information presentation is the presentation of the lumen direction. The support information generating unit 150 presents a dedicated lumen direction presentation depending on the presence or absence of suction. Fig. 12 is a flowchart showing an example of the lumen direction presentation.

[0122] In step S60, suction is performed. This suction may be performed by automatic suction control or may be performed by manual operation. In step S61, the image acquisition unit 141 acquires an endoscopic image IMG from the endoscope 10.

[0123] In step S62, the examination status determination unit 161 determines the lumen direction from the endoscopic image IMG. In step S63, it is determined whether the lumen direction has been determined. If the lumen direction has not been determined, the process returns to step S60. If the lumen direction has been determined, in step S64, the support information generation unit 150 displays the determined lumen direction on the display device 220.

[0124] During operation while suctioning, the endoscopic image IMG tends to become a luminal image or a red ball state specific to suctioning, making it difficult to understand the direction of the lumen. Therefore, a trained model is generated by training an endoscopic image of the lumen during suctioning in advance, and the examination status determination unit 161 performs lumen recognition dedicated to suctioning by inference processing using the trained model. Then, the examination status determination unit 161 estimates and tracks the luminal direction using the recognition result and optical flow technology, etc., and the support information generation unit 150 sequentially presents the resulting luminal direction on the screen. In tracking the lumen, the examination status determination unit 161 may estimate the luminal direction using an image such as optical flow, may estimate the luminal direction using the movement of the endoscope grasped from insertion shape information such as UPD, or may estimate the luminal direction based on information obtained from a sensor attached to the doctor. The UPD and the sensor attached to the doctor will be described later. In addition, water in the intestine is also sucked in during suctioning. Therefore, the examination status determination unit 161 may estimate the water flow caused by suction from the endoscopic image, predict the lumen direction from the estimation result, and the support information generation unit 150 may present the predicted lumen direction.

[0125] An example of a lumen direction presentation screen is shown in Fig. 13. The support information generating unit 150 causes the display device 220 to display an arrow indicating the lumen direction together with the endoscopic image as a display image. The arrow indicating the lumen direction corresponds to the support information AST. Although the endoscopic image and the arrow are shown separately in Fig. 13, the arrow may be superimposed on the endoscopic image.

[0126] As another example of presenting support information, the examination status determination unit 161 determines that if the lumen is open, there is a possibility of over-inflation, and the support information generation unit 150 suggests suction, or the control determination unit 162 and the control unit 170 automatically perform suction.

[0127] Furthermore, the examination status determination unit 161 determines adhesions from the movement of the intestinal wall during suction, and changes the operation plan based on the result. For example, when the examination status determination unit 161 determines that adhesions are present in the sigmoid colon, it determines that insertion using the general axis-holding shortening method is difficult. This is because pain occurs when the adhesions are stimulated by the pulling operation in the axis-holding shortening method. At this time, the support information generation unit 150 presents an axis-holding shortening method guide or a push method guide that takes adhesions into consideration.

[0128] Furthermore, when a polyp is removed during treatment, the examination status determination unit 161 determines the size and location of the polyp and determines whether the polyp should be removed by suction or by using a treatment tool. The support information generation unit 150 presents the determination result as support information AST. When the examination status determination unit 161 determines that the polyp should be removed by suction, the control determination unit 162 and the control unit 170 adjust the amount of suction depending on the size or condition of the polyp. Furthermore, when a polyp is suctioned, water is automatically made to flow through the suction tube of the endoscope, making it easier for the polyp to reach the location where it is to be caught.

[0129] 7. Endoscope system and fifth detailed configuration example Hereinafter, a configuration example in which the control device 100 performs automatic suction control using information other than the endoscopic image IMG will be described.

[0130] 14 is a configuration example of an endoscope system 400 including the control device 100. The endoscope system 400 includes an endoscope device 300 and an insertion shape observation device 200. Note that, when the control device 100 does not use the insertion shape information, the insertion shape observation device 200 does not need to be provided.

[0131] The endoscopic device 300 includes an endoscope 10, a suction device 30, an air supply device 40, a light source device 330, a signal processing device 310, and a display device 320. The light source device 330 generates and controls illumination light, which is guided to the tip of the endoscope 10 by a light guide and emitted from the tip of the endoscope 10. The signal processing device 310 processes an image signal output by the endoscope 10 to generate an endoscopic image. The signal processing device 310 also acquires the ID and the like of the endoscope 10 as endoscope type information. The display device 320 displays the endoscopic image generated by the signal processing device 310.

[0132] The insertion shape observation device 200 includes an endoscope shape acquisition sensor 20, a main device 210, and a display device 220. The endoscope shape acquisition sensor 20 detects a magnetic field of a source coil provided in the endoscope insertion part. The main device 210 acquires the position and shape of the endoscope insertion part based on a detection signal from the endoscope shape acquisition sensor 20, and outputs an image showing the position and shape of the endoscope insertion part to the display device 220. The display device 220 displays the image output by the main device 210. The display device 220 and the display device 320 are also called monitors, and are liquid crystal display devices or the like. Note that one display device may be provided in the endoscope system 400, and the endoscope device 300 and the insertion shape observation device 200 may share the one display device.

[0133] The control device 100 is provided in the signal processing device 310. The control device 100 receives, as inputs, an endoscopic image and endoscope type information from the signal processing device 310, and information on the position and shape of the endoscope insertion portion acquired by the main device 210. The control device 100 may be provided anywhere in the endoscope system 400.

[0134] 15 shows an example of the configuration of an endoscope 10 and an endoscope shape acquisition sensor 20. As shown in FIG. 15, the endoscope 10 includes an operation section 12, an insertion section 14, and a source coil 18.

[0135] The insertion section 14 has a flexible, elongated shape and includes a rigid section 16 provided at its tip and a bending section 15 that can be angled. The rigid section 16 is provided with an imaging device, an illumination lens, a water supply port, an air supply port, a forceps port, and the like.

[0136] The operation unit 12 is a device for the user to operate the endoscope 10, and includes, for example, a grip, an angle operation dial, and an air / water supply button. The operation of the endoscope will now be described. When the user pushes the grip in the longitudinal direction of the insertion unit 14, the insertion unit 14 is inserted. This is called a push operation. When the user pulls the grip in the longitudinal direction of the insertion unit 14, the insertion unit 14 is pulled out. This is called a pull operation. When the user turns the grip in the circumferential direction of the insertion unit 14, the insertion unit 14 rotates in the circumferential direction. This is called a torque operation. When the user operates the angle operation dial, the bending portion 15 of the insertion unit 14 bends up, down, left, and right. This is called an angle operation.

[0137] The source coil 18 generates a magnetic field. For example, a plurality of source coils 18 are provided at a predetermined interval in the insertion section 14. The endoscope shape acquisition sensor 20 detects the magnetic field from each source coil 18, and the main unit 210 of the insertion shape observation device 200 detects the position of each source coil 18 based on the detection signal, thereby detecting the position of each part of the insertion section 14. The main unit 210 also detects the shape of the insertion section 14 based on the detected positions of the plurality of source coils 18. Note that the sensing method for observing the insertion shape is not limited to a method using a magnetic field, and may be a method using, for example, electromagnetic waves, ultrasound, light, or the like.

[0138] Fig. 16 is a fifth detailed configuration example of the control device 100. In this configuration example, the control device 100 further includes an endoscope shape acquisition unit 142, a patient information acquisition unit 143, an operation observation information acquisition unit 144, a suction volume acquisition unit 145, and a pressure acquisition unit 146. In this configuration example, the image acquisition unit 141, the endoscope shape acquisition unit 142, the patient information acquisition unit 143, the operation observation information acquisition unit 144, the suction volume acquisition unit 145, and the pressure acquisition unit 146 correspond to the information acquisition unit 140 in Fig. 1. Note that descriptions of components that are the same as those already described will be omitted as appropriate.

[0139] It is not necessary to provide all of the endoscope shape acquisition unit 142, the patient information acquisition unit 143, the operation observation information acquisition unit 144, the suction volume acquisition unit 145, and the pressure acquisition unit 146, and any one to five of these units may be omitted. Also, the support information generation unit 150 may be omitted, or automatic air supply control may be further combined as in the third detailed configuration example.

[0140] The endoscope shape acquisition unit 142 acquires insertion shape information based on a detection signal from the endoscope shape acquisition sensor 20. The patient information acquisition unit 143 acquires patient information from the electronic medical record 600. The operation observation information acquisition unit 144 acquires operation observation information from the operation observation sensor 700. The image acquisition unit 141 acquires an endoscopic image from the endoscope 10. The suction volume acquisition unit 145 acquires a measurement value of the suction volume from the suction volume measurement unit 37. The pressure acquisition unit 146 acquires a measurement value of the pressure from the pressure sensor 36.

[0141] The insertion shape information, patient information, operation observation information, endoscopic image, suction volume measurement value, and pressure measurement value are input as information IFIN to the examination status determination unit 161. The examination status determination unit 161 determines which of a plurality of examination statuses the examination status corresponds to based on the information IFIN. The examination status includes, for example, the endoscope insertion status, the operation status, or the state of the intestines. Specifically, the examination status determination unit 161 classifies the examination status based on the information IFIN, and outputs most recent status information TYK and time-series status information JIK based on the classification result.

[0142] The insertion shape information is information on the position and shape of the endoscope insertion part, and is acquired by the above-mentioned insertion part shape observation device. The examination status determination unit 161 may recognize changes in the position and shape of the endoscope insertion part, and use the recognition result in the above classification.

[0143] By using the insertion shape information in judging the examination status, the position and shape of the insertion part, or changes therein, can be directly judged. For example, the location where the endoscope 10 is located, the operation performed by the doctor, the amount of operation, the procedure during the operation, and the shape of the insertion part during the operation can be judged. This improves the accuracy of judging the examination status.

[0144] The operation observation sensor 700 is a sensor that can measure the movement of a doctor or the like operating the endoscope 10, and is, for example, a motion capture sensor. The operation observation sensor 700 may be a sensor that is worn by a doctor or the like, or may be a sensor that measures the movement of a doctor or the like from a location remote from the doctor or the like. The operation observation information acquisition unit 144 acquires the measured motion information of the doctor or the like as operation observation information.

[0145] By using the operation observation information in judging the examination status, the operation performed by the doctor, etc. can be directly judged. For example, it is possible to judge the operation performed by the doctor, etc., the amount of operation, the procedure during the operation, and whether or not the procedure is correct for the currently adopted operation method. This improves the accuracy of judging the examination status.

[0146] The electronic medical record 600 accumulates patient information, which is information on the attributes of a patient. The electronic medical record 600 is stored in a storage device provided outside the endoscope system 400, for example, and the patient information acquisition unit 143 acquires the patient information from the storage device. The patient information is, for example, the patient's physique, sex, age, medical history, body fat percentage, CT image, or any combination of two or more of them. The physique is BMI, height, weight, or any combination of two or more of them. The patient information may also include past examination information. The past examination information is, for example, which insertion method was used, which scope was used, the history of suction during insertion, whether pain occurred, or any combination of two or more of them.

[0147] By using patient information to determine the examination status, it is possible to present support information or control automatic suction based on the information of the patient undergoing the examination. For example, when CT images are used, the shape of the patient's intestines can be known, and an ideal insertion plan can be created based on this information. At this time, optimal automatic suction control may be performed for that insertion plan. In addition, the created intestinal MAP can be compared with each piece of acquired information to sequentially present optimal guides. During observation or treatment, the location of polyps can be superimposed on the CT image in advance, shortening the time required to search for polyps one by one and allowing for a smooth transition to treatment. It is also possible to determine whether a discovered polyp has been there before or is new.

[0148] The pressure sensor 36 is as described above in FIG. 10. By using the pressure measurement value in judging the examination status, the luminal status based on the intraluminal pressure can be judged. This improves the accuracy of judging the examination status. For example, excessive air supply, etc. can be judged from the intraluminal pressure. Alternatively, if suction is not possible due to the suction tube being clogged, etc., this can be estimated from the pressure sensor measurement value, so that if the intestinal wall does not come closer even when suction is performed, it can be determined whether adhesions, etc., or a malfunction are the cause.

[0149] The suction amount measuring unit 37 is as described above with reference to FIG. 10. By using the measured value of the suction amount in determining the test status, the test status can be determined based on the suction amount. This improves the accuracy of determining the test status. For example, it can determine whether suction is turned on or off at the appropriate timing, or whether suction is being performed with appropriate suction strength.

[0150] Although the present embodiment and its modified examples have been described above, the present disclosure is not limited to each embodiment or its modified example as it is, and in the implementation stage, the components can be modified and embodied within the scope of the gist. In addition, a plurality of components disclosed in each of the above-mentioned embodiments and modified examples can be appropriately combined. For example, some components may be deleted from all the components described in each embodiment or modified example. Furthermore, the components described in different embodiments or modified examples may be appropriately combined. In this way, various modifications and applications are possible within the scope of the gist of the present disclosure. In addition, a term described at least once in the specification or drawings together with a different term having a broader meaning or the same meaning can be replaced with the different term in any part of the specification or drawings. [Explanation of symbols]

[0151] 10 endoscope, 12 operation section, 14 insertion section, 15 bending section, 16 rigid section, 18 source coil, 20 endoscope shape acquisition sensor, 30 suction device, 31 suction control circuit, 32 suction pump, 34 suction tank, 35 solenoid valve, 36 pressure sensor, 37 suction amount measurement section, 40 air supply device, 41 air supply control circuit, 42 air supply pump, 45 solenoid valve, 51 suction port, 52 suction tube, 53 port, 54 suction operation button, 55 suction operation detection sensor, 61 air supply port, 62 air supply tube, 64 air supply operation button, 65 air supply operation detection sensor, 100 control device, 121 immediate situation determination section, 122 time-series situation determination section, 140 information acquisition section, 141 image acquisition section, 142 endoscope shape acquisition section, 143 patient information acquisition section, 144 Operation observation information acquisition unit, 145 suction volume acquisition unit, 146 pressure acquisition unit, 150 support information generation unit, 160 judgment unit, 161 examination status judgment unit, 162 control judgment unit, 170 control unit, 200 insertion shape observation device, 210 main body device, 220 display device, 300 endoscope device, 310 signal processing device, 320 display device, 330 light source device, 400 endoscope system, 600 electronic medical record, 700 operation observation sensor, AST support information, CNT control signal, IMG endoscope image

Claims

1. A processor is included. The processor, Acquire an endoscopic image which is an image captured by an endoscope; determining whether or not suction is necessary based on any one of a distance between a tip of the endoscope and a mucosa of a subject shown in the endoscopic image, a position of the tip of the endoscope, and a state inside a lumen shown in the endoscopic image; When it is determined that the suction is necessary, control is performed to perform the suction. determining a state of the subject based on a change in the endoscopic image when the suction is performed; A control device that controls aspiration based on the state of the subject.

2. In claim 1, The processor, a control device that determines whether or not the suction is necessary based on the endoscopic image and depending on the distance between the mucous membrane of the subject and the tip of the endoscope.

3. In claim 2, The processor, A control device characterized by determining whether the distance between the mucosa of the subject and the tip of the endoscope is greater than a predetermined distance based on the endoscopic image, and determining that the suction is necessary if it is determined that the distance is greater than the predetermined distance.

4. In claim 3, A control device characterized in that the predetermined distance is an optimal distance at which the folds of the large intestine can be everted by angle and torque operations of the endoscope in the axis-holding shortening method.

5. In claim 1, The processor, A control device which determines whether or not the suction is necessary based on a change in the endoscopic image when the suction is performed.

6. In claim 5, The processor, A control device characterized by determining not to perform suction when it is determined, based on a change in the endoscopic image when the suction is performed, that the distance between the mucosa of the subject and the tip of the endoscope is not closer than a predetermined distance.

7. In claim 5, The processor, A control device characterized by determining the presence or absence of adhesions in the subject based on changes in the endoscopic image when the suction is performed, and determining not to perform the suction if it is determined that adhesions are present.

8. In claim 5, The processor, A control device characterized by determining an operation plan indicating an operation procedure of the endoscope based on a change in the endoscopic image when the suction is performed, and determining whether the suction is necessary based on the determined operation plan.

9. In claim 1, The processor, A control device which determines whether or not the suction is necessary based on the endoscopic image and in accordance with the position of the endoscope within the lumen.

10. In claim 9, The processor, a control device that determines that the suction is necessary when it is determined based on the endoscopic image that the endoscope is located at a predetermined position within the lumen.

11. In claim 10, A control device characterized in that the specified portion is a portion beyond the mittrans in the sigmoid colon or the transverse colon.

12. In claim 1, The processor, A control device characterized by determining not to perform the suction when it determines, based on the endoscopic image, that the tip of the endoscope is in a red ball state, meaning that it is in contact with the mucous membrane of the subject.

13. In claim 1, The processor, a control device that determines, based on the endoscopic image, that the lumen is in an over-inflation state in which it has been expanded by air supply to a predetermined extent or more, and determines to perform the suction.

14. In claim 1, The processor, determining the strength of the suction based on the endoscopic image; A control device that controls the suction to be performed at the determined strength.

15. In claim 14, The processor, a control device that determines the strength of the suction according to the distance between the mucous membrane of the subject and the tip of the endoscope based on the endoscopic image.

16. In claim 15, The processor, a control device which weakens the suction as the distance between the mucous membrane of the subject and the tip of the endoscope becomes closer due to the suction;

17. In claim 14, The processor, A control device that recognizes at least one of residue, water, excised tissue, or sprayed medicinal solution based on the endoscopic image, and determines whether or not the suction is necessary and the strength of the suction depending on the recognition result.

18. In claim 1, The processor, Acquire insertion shape information which is information representing the shape of an insertion portion of the endoscope; A control device which determines whether or not the suction is necessary based on the endoscopic image and the insertion shape information.

19. In claim 1, The processor, A control device characterized by acquiring examination status information regarding the status of an endoscopic examination using the endoscope based on the endoscopic image, and determining whether or not the suction is necessary based on the examination status information.

20. A control device acquires an endoscopic image which is an image captured by an endoscope; the control device determines whether or not suction is necessary based on any one of a distance between a tip of the endoscope and a mucous membrane of a subject shown in the endoscopic image, a position of the tip of the endoscope, and a state inside a lumen shown in the endoscopic image; When it is determined that the suction is necessary, the control device performs control to perform the suction. the control device determines a state of the subject based on a change in the endoscopic image when the suction is performed; and the control device controls aspiration based on a state of the subject; A method for operating a control device comprising:

Images