Insertion assistance system, endoscope system, and method of operating the insertion assistance system

Abstract

An insertion assistance system (100) comprises an endoscope state information acquisition unit (150), a pain state recognition unit (130), and an insertion assistance information generation unit (150). The endoscope state information acquisition unit acquires endoscope state information including at least one of an endoscope image, insertion part shape information, and operation recognition information. The pain state recognition unit receives at least one of input pain information which is information relating to pain of a patient acquired from the patient or a medical professional in real time and the endoscope state information, and recognizes a pain state in which the patient is feeling pain in an endoscope examination and acquires pain state information. The insertion assistance information generation unit generates insertion assistance information according to the pain state information and the endoscope state information.

Description

[Technical field]

[0001] The present invention relates to an insertion assist system, an endoscope system, and Method of Operation of Insertion Assist System etc. [Background technology]

[0002] Patent Document 1 discloses a technique for providing operational guidance in colon examination using an endoscope. In this technique, an insertion shape observation device observes the shape of the endoscope insertion part inserted into the large intestine, the endoscope takes an endoscopic image of the large intestine, and an insertion assistance device judges the insertion status of the endoscope insertion part based on the shape of the endoscope insertion part and the endoscopic image, and provides operational guidance for the endoscope based on the judgment result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 235185 Summary of the Invention [Problem to be solved by the invention]

[0004] In an endoscopic examination, a patient may experience pain depending on the operation of the endoscope insertion part. In Patent Document 1, operation guidance is not provided that takes into consideration the pain of the patient undergoing the examination, so operation guidance is provided regardless of whether the operation will cause pain to the patient, and there is a possibility that the patient will experience pain due to operations following the operation guide. [Means for solving the problem]

[0005] One aspect of the present disclosure relates to an insertion assistance system including an endoscope status information acquisition unit that acquires endoscope status information including at least one of an endoscopic image, insertion portion shape information which is information about the shape of an endoscope insertion portion, or operation recognition information which is information about a change in at least one of the shape or position of the endoscope insertion portion during an endoscopic examination using an endoscope; a pain situation recognition unit that receives input pain information which is information about the pain of the patient acquired from a patient or medical professional in real time, or at least one of the endoscope status information, and recognizes a pain situation in which the patient is experiencing pain during the endoscopic examination, to acquire pain situation information; and an insertion assistance information generation unit that generates insertion assistance information according to the pain situation information and the endoscope status information.

[0006] Another aspect of the present disclosure relates to an endoscopic system including an endoscope used for endoscopic examination, an endoscope status information acquisition unit that acquires endoscopic status information including at least one of an endoscopic image, insertion portion shape information which is the shape of the endoscope insertion portion, or operation recognition information which is information on at least one change in the shape or position of the endoscope insertion portion during the endoscopic examination using the endoscope, a pain situation recognition unit that receives input pain information which is information on the pain of the patient acquired from a patient or medical professional in real time, or at least one of the endoscope status information, and recognizes a pain situation which is a situation in which the patient is experiencing pain during the endoscopic examination, to acquire pain situation information, and an insertion assistance information generation unit that generates insertion assistance information according to the pain situation information and the endoscope status information.

[0007] Yet another aspect of the present disclosure relates to an insertion assistance method including: acquiring endoscope status information including at least one of an endoscopic image, insertion portion shape information which is the shape of the endoscope insertion portion, or operation recognition information which is information regarding a change in at least one of the shape or position of the endoscope insertion portion during an endoscopic examination using an endoscope; input pain information which is information regarding the pain of the patient acquired from a patient or a medical professional in real time, or inputting at least one of the endoscope status information, recognizing a pain situation which is a situation in which the patient is experiencing pain during the endoscopic examination, and acquiring pain situation information; and generating insertion assistance information according to the pain situation information and the endoscope status information. [Brief description of the drawings]

[0008] [Figure 1] 1 shows a first configuration example of an insertion assistance system. [Diagram 2] 2 shows a second configuration example of an insertion assistance system. [Diagram 3] 1 shows an example of an endoscope system configuration. [Figure 4] An example of the configuration of an endoscope and an endoscope shape acquisition sensor. [Diagram 5] FIG. [Figure 6] 1 shows a first detailed configuration example of an insertion support system. [Figure 7] Diagram of the parts of the large intestine. [Figure 8] 13 is an example of the shape transition of the endoscope insertion part. [Figure 9] An illustration of the pain condition. [Figure 10] An illustration of the pain condition. [Figure 11] An illustration of the pain condition. [Figure 12] An illustration of the pain condition. [Figure 13] An illustration of the pain condition. [Figure 14] FIG. 13 is an explanatory diagram of pain situation recognition according to the amount of displacement. [Figure 15] 6 is a flowchart of a process performed by the insertion support system in the first detailed configuration example. [Figure 16]11 is a flowchart showing a specific example of recognition of a painful state and generation of insertion support information according to the recognition result. [Figure 17] 11 is a flowchart showing a specific example of recognizing a pain condition. [Figure 18] 2 shows a second detailed configuration example of the insertion support system. [Figure 19] 13 is a flowchart of a process performed by the insertion support system in the second detailed configuration example. [Figure 20] 3 shows a third detailed configuration example of the insertion assistance system. [Figure 21] 4 shows a fourth detailed configuration example of the insertion assistance system. [Figure 22] 5 shows a fifth detailed configuration example of an insertion assistance system. [Diagram 23] FIG. 13 is a diagram for explaining a specific example of recognition of a pain state. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present embodiment will be described below. Note that the present embodiment described below does not unduly limit the contents described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present disclosure.

[0010] 1. Configuration example Here, a basic configuration example of the insertion support system in this embodiment will be described. A detailed configuration example of the insertion support system and the correspondence between the detailed configuration example and the basic configuration example will be described later.

[0011] 1 shows a first configuration example of the insertion support system 100. The insertion support system 100 includes an endoscope status information acquisition unit 110, a pain status recognition unit 130, and an insertion support information generation unit 150.

[0012] The endoscope status information acquisition unit 110 acquires endoscope status information during an endoscopic examination using an endoscope. The endoscope status information includes at least one of an endoscope image, insertion part shape information, and endoscope status information. Note that "at least one of an endoscope image, insertion part shape information, and endoscope status information" refers to an endoscope image, insertion part shape information, endoscope status information, or a combination of any two or more of these pieces of information.

[0013] The endoscopic image is an image captured by an endoscope. Specifically, the endoscopic image is each frame image of a video captured by an endoscope. The insertion part shape information is information on the shape of the endoscope insertion part, and is acquired, for example, by an insertion part shape observation device described later. The endoscope insertion part is a part of the endoscope that is inserted into the body, and in this embodiment, it is an insertion part of a flexible scope used in the digestive tract or the like. The operation recognition information is information on at least one change in the shape or position of the endoscope insertion part. That is, the operation recognition information is a temporal change in at least one of the shape or position of the endoscope insertion part caused by the operation of the endoscope. For example, the insertion part shape observation device observes the insertion part shape from moment to moment, thereby outputting the insertion part shape information in a time series, and the operation recognition information is acquired from the temporal change in the insertion part shape information output in a time series.

[0014] The endoscope status information acquisition unit 110 outputs the endoscope status information ESI. This endoscope status information ESI may be the endoscope status information itself acquired by the endoscope status information acquisition unit 110 as described above, or may be information obtained by processing the acquired endoscope status information, or may be a combination of these.

[0015] At least one of the input pain information INPN and the endoscope status information ESI is input to the pain situation recognition unit 130. Although Fig. 1 illustrates an example in which both the input pain information INPN and the endoscope status information ESI are input to the pain situation recognition unit 130, it is not necessary for either the input pain information INPN or the endoscope status information ESI to be input to the pain situation recognition unit 130.

[0016] The input pain information INPN is information about the patient's pain that is acquired in real time from the patient or a medical professional. The patient is a patient undergoing an endoscopic examination using an endoscope. The medical professional is a doctor performing the endoscopic examination or his / her assistant, who is able to recognize the patient's voice or facial expression, etc. As described below, the input pain information INPN is acquired, for example, by a switch, the result of image recognition of facial expression, or the result of voice recognition, etc.

[0017] The pain situation recognition unit 130 recognizes a pain situation in which the patient is in pain during an endoscopic examination, and acquires pain situation information PSI. The pain situation information PSI is a recognition result of the pain situation, for example, the presence or absence of a pain situation, an estimated level of pain, or a combination thereof. The "situation in which the patient is in pain" is not limited to a situation in which pain actually occurs, but may be any situation in which the occurrence of pain is estimated. More specifically, a situation in which pain is likely to occur is determined in advance, and when a situation recognized based on the endoscope situation information ESI matches the predetermined situation, it is recognized as a "situation in which the patient is in pain". Alternatively, when a pain situation is recognized based on the input pain information INPN, when input pain information INPN indicating the occurrence of pain is input, it is recognized as a "situation in which the patient is in pain". Even in this case, the input pain information INPN may be information from which the occurrence of pain is estimated. For example, when the input pain information INPN is input based on the facial expression of the patient, it is sufficient that the occurrence of pain is estimated from the facial expression.

[0018] The insertion support information generating unit 150 generates insertion support information according to the pain state information PSI and the endoscope state information ESI. The insertion support information is information that supports the insertion of an endoscope into a patient. Specifically, the insertion support information is information that is presented or notified to a medical professional or an insertion / removal device that performs an operation to insert an endoscope into a patient, and indicates what operation should be performed next for the current insertion state. The insertion support system 100 may, for example, display characters, symbols, images, etc. corresponding to the insertion support information on a display device of the endoscope system, or may output a control signal to the insertion / removal device to perform an operation or process indicated by the insertion support information.

[0019] According to this embodiment, it is possible to generate insertion support information that takes into account the situation in which a patient is in pain during an endoscopic examination. That is, by having a medical professional or an insertion / removal device perform an operation based on the insertion support information of this embodiment, it is possible to perform an operation that relieves the patient's pain or an operation that prevents the occurrence of pain when or before the patient experiences pain.

[0020] When both the input pain information INPN and the endoscope condition information ESI are input, the pain condition recognition unit 130 recognizes the pain condition based on the input pain information INPN. Here, "when the input pain information INPN is input" means when the input pain information INPN indicating that the patient is in pain is input, that is, when a condition in which the patient is in pain is recognized based on the input pain information INPN.

[0021] Specifically, when both the input pain information INPN and the endoscope status information ESI are input, the pain situation recognition unit 130 prioritizes the recognition result of the pain situation based on the input pain information INPN over the recognition result of the pain situation based on the endoscope status information ESI. That is, even if the pain situation recognition unit 130 recognizes that the situation is not one in which pain is occurring based on the endoscope status information ESI, if it recognizes that the situation is one in which pain is occurring based on the input pain information INPN, it outputs pain situation information PSI indicating that the situation is one in which pain is occurring.

[0022] More specifically, when endoscope status information ESI is input to the pain situation recognition unit 130 but input pain information INPN is not input, the pain situation recognition unit 130 recognizes the pain situation based on the endoscope status information ESI, and the insertion support information generation unit 150 generates insertion support information according to the pain situation recognized based on the endoscope status information ESI. When input pain information INPN is input to the pain situation recognition unit 130 in addition to the endoscope status information ESI, the pain situation recognition unit 130 recognizes the pain situation based on the input pain information INPN, and the insertion support information generation unit 150 generates insertion support information according to the pain situation recognized based on the input pain information INPN.

[0023] According to this embodiment, the pain state can be recognized based on the input pain information INPN issued by the patient or medical staff. As a result, even if the pain state is not recognized based on the endoscope status information ESI, the pain state can be recognized by prioritizing the input pain information INPN issued by the patient or medical staff, and the pain state information PSI can be output. For example, if there is a pain state that varies depending on the individual patient, or a pain state that is unique to the patient, there is a possibility that the pain state cannot be recognized based on the endoscope status information ESI, but according to this embodiment, the input pain information INPN is prioritized, and such pain states can be recognized.

[0024] The endoscope status information acquisition unit 110 classifies the endoscope insertion status based on at least one of the endoscope image, the insertion part shape information, and the operation recognition information, and outputs the endoscope status information ESI including the classification result. The pain status recognition unit 130 recognizes the pain status by recognizing whether the insertion status indicated by the classification result corresponds to a status in which pain occurs.

[0025] The insertion status of the endoscope is a predetermined insertion status that appears in the procedure of proceeding with the endoscopic examination, and is specified by, for example, a predetermined insertion part position, a predetermined change in the insertion part position, a predetermined insertion part shape, a predetermined change in the insertion part shape, a predetermined endoscope operation, or a combination of any two or more of them. For example, there are insertion methods such as the loop method and the axis holding shortening method in a colonoscopic examination, and there are insertion statuses such as the insertion part shape or operation that appear when proceeding with the insertion procedure in each insertion method. In each of these insertion statuses, the operation to be performed next is determined, and pain is likely to occur in certain insertion statuses among these insertion statuses.

[0026] The endoscope status information acquisition unit 110 determines which of a plurality of insertion statuses corresponds to the insertion status based on at least one of the endoscope image, the insertion part shape information, and the operation recognition information. The plurality of insertion statuses includes, for example, a plurality of insertion statuses for which insertion support information should be presented, and a plurality of insertion statuses in which pain occurs. The pain status recognition unit 130 recognizes that the determined insertion status corresponds to an insertion status in which pain occurs, as a status in which pain occurs. The insertion support information generation unit 150 generates insertion support information indicating the next operation to be performed in the determined insertion status, and at that time, changes the insertion support information depending on whether the pain status has been recognized or not.

[0027] According to this embodiment, the insertion state of the endoscope is classified, so that the pain state recognition unit 130 can recognize the pain state by recognizing whether or not the insertion state is an insertion state that causes pain. Also, since there are insertion states that are likely to cause pain in endoscopic examination as described above, by setting the insertion states as candidates for classification, it becomes possible to determine whether or not the insertion state is an insertion state that causes pain.

[0028] The insertion support information generating unit 150 generates insertion support information corresponding to the insertion status of the endoscope based on the endoscope status information ESI. At this time, when pain status information PSI indicating that the status is not painful is input, the insertion support information generating unit 150 generates first insertion support information as the insertion support information, and when pain status information PSI indicating that the status is painful is input, the insertion support information generating unit 150 generates second insertion support information as the insertion support information. The second insertion support information is different from the first insertion support information.

[0029] Specifically, the operation presented by the second insertion support information is different from the operation presented by the first insertion support information. The first insertion support information indicates an operation when there is no pain state, that is, a normal operation in the insertion procedure. The second insertion support information indicates an operation when there is a pain state, that is, a pain relieving operation or a pain avoiding operation different from the above-mentioned normal procedure. Different insertion support information may be generated depending not only on the presence or absence of a pain state but also on the pain level. That is, the insertion support information generating unit 150 may generate the second insertion support information when the pain state recognizing unit 130 recognizes a pain state of the first pain level, and may generate the third insertion support information when the pain state recognizing unit 130 recognizes a pain state of the second pain level. The second pain level is higher or lower than the first pain level, and the third insertion support information is different from the first insertion support information and the second insertion support information.

[0030] According to this embodiment, different insertion support information can be generated depending on whether or not the situation is painful. That is, it is possible to present appropriate insertion support information for both a situation where no pain occurs and a situation where pain occurs.

[0031] The above-mentioned operation recognition information may include insertion portion shape change information, which is information on the shape change of the endoscope insertion portion. In this case, the endoscope status information acquisition unit 110 performs the above-mentioned classification based on the insertion portion shape information and the insertion portion shape change information. The shape change is a change in the shape of the endoscope insertion portion between a shape before the change and a shape after the change. The shape change information may include information on the direction of the change, information on the magnitude of the change, or both.

[0032] When the endoscope is operated, the shape of the endoscope insertion section changes, and pain may occur depending on the shape displacement at this time. According to this embodiment, classification is performed based on the insertion section shape displacement information, so that the pain state caused depending on the shape displacement of the endoscope insertion section can be recognized.

[0033] As described above, the insertion portion shape displacement information may include shape displacement amount information, which is information on the magnitude of the shape displacement. The endoscope status information acquisition unit 110 performs the above classification based on the insertion portion shape information and the shape displacement amount information.

[0034] In some insertion situations, pain may occur when the magnitude of the shape displacement exceeds a predetermined value. According to the present embodiment, classification is performed based on the shape displacement amount information, so that a pain state that occurs in response to the magnitude of the shape displacement exceeding a predetermined value can be recognized.

[0035] The input pain information INPN described above is at least one of patient facial expression information and pain transmission information. The patient facial expression information is information related to the facial expression of the patient in an endoscopic examination. Specifically, a camera captures the facial expression of the patient, and a facial expression recognition unit recognizes whether the facial expression of the patient shown in the image is an expression of pain or not, and outputs the result as pain transmission information. The pain transmission information is information from a transmission device operated by a patient or a medical professional. Specifically, the transmission device is a device that can be operated by a patient or a medical professional depending on the patient's pain condition, such as a switch or a touch panel. Each of the camera, the facial expression recognition unit, and the transmission device may be included in the insertion support system 100, or may be provided outside the insertion support system 100.

[0036] According to this embodiment, input pain information INPN can be obtained based on the patient's facial expression, the patient's message, the medical professional's message, or a combination of any two or more of these, and as described above, it is possible to recognize pain situation information PSI and generate insertion support information based on the input pain information INPN.

[0037] The above-mentioned pain state is not limited to the presence or absence of pain. Specifically, the pain state recognition unit 130 may recognize the pain state by distinguishing and recognizing states with different pain levels, states with different pain frequencies, or states with different ease of avoiding pain.

[0038] The pain level is the level of pain felt by the patient when a situation in which pain is occurring to the patient is recognized. For example, the pain level changes according to the amount of displacement of the endoscope insertion portion in a certain insertion situation. In this case, the pain level is determined based on the amount of displacement. Alternatively, when the pain level changes according to the insertion situation, a pain level is associated with each insertion situation, and when a certain insertion situation is detected, the pain level associated with that insertion situation is output.

[0039] The frequency of pain is the frequency with which a situation in which the patient feels pain is recognized, for example, the number of times the pain situation is recognized per unit time. This frequency may be either the frequency when the same insertion situation is repeated, or the frequency when a plurality of insertion situations are mixed. For example, the insertion assistance system 100 includes a memory (not shown), and the pain situation recognition unit 130 stores the pain situation information PSI in the memory, and detects the frequency of pain by referring to the pain situation information PSI stored in the memory.

[0040] The ease of avoiding pain is determined depending on whether or not there is an operation to relieve pain or an operation to avoid pain in each pain situation. For example, in a certain insertion situation, if there is no operation to relieve pain or an operation to avoid pain, the degree of ease of avoiding pain is low. In addition, in a certain insertion situation, if there is an operation to avoid pain, the degree of ease of avoiding pain is high. In addition, in a certain insertion situation, if there is an operation to relieve pain but no operation to avoid pain, the degree of ease of avoiding pain is medium. For example, when the pain situation recognition unit 130 recognizes a certain pain situation, it can know the operation that can be selected in that pain situation by referring to the insertion support information associated with that pain situation.

[0041] Fig. 2 shows a second configuration example of the insertion support system 100. In Fig. 2, the insertion support system 100 further includes an inspection condition information acquisition unit 160. Note that the description of the same components as those already described will be omitted as appropriate.

[0042] The examination condition information acquisition unit 160 acquires examination condition information including at least one of endoscope type information, patient information, and past examination information. The endoscope type information is information on the type of endoscope insertion section used in endoscopic examination. The patient information is information on the attributes of the patient. The past examination information is information on past endoscopic examinations. The insertion support information generation unit 150 generates insertion support information based on the examination condition information. In addition, the pain situation recognition unit 130 recognizes the pain situation based on the examination condition information.

[0043] The endoscope insertion portion has different physical characteristics such as thickness or hardness depending on the type of endoscope. For example, an endoscope stores an ID indicating its model, and endoscope type information can be obtained by acquiring the ID. The likelihood of a pain condition occurring or the pain level changes depending on the physical characteristics such as thickness or hardness of the endoscope insertion portion. By acquiring the endoscope type information, the pain condition recognition unit 130 can appropriately recognize a pain condition depending on the physical characteristics such as thickness or hardness of the endoscope insertion portion.

[0044] The patient information is, for example, information such as the patient's gender, physique, body type, or medical history. For example, as described below, the patient information can be acquired from information stored in an electronic medical record. Alternatively, a medical professional may input the patient information to the insertion assistance system 100. The likelihood of a pain condition occurring or the pain level, etc., changes depending on the patient's gender, physique, body type, or medical history. By acquiring the patient information, the pain condition recognition unit 130 can appropriately recognize the pain condition depending on the patient's gender, physique, body type, or medical history, etc.

[0045] The past examination information is past examination information related to the same part as the part to be examined this time, and is, for example, the pain state that occurred, the insertion support information presented at that time, or both of these pieces of information. The past examination information may be examination information previously acquired from the same patient as the patient currently being examined, or may be examination information previously acquired from multiple patients. For example, the pain state information PSI output by the pain state recognition unit 130 and the insertion support information generated by the insertion support information generation unit 150 are recorded as logs in a memory (not shown), and the past examination information can be acquired by referring to the previously recorded logs.

[0046] The processing of the insertion support system 100 described above may be implemented as an insertion support method as follows. The insertion support method is not limited to being implemented by the insertion support system 100, but may be implemented by various systems or devices such as an endoscope system described below. The insertion support method includes acquiring endoscope status information ESI, receiving at least one of input pain information INPN and endoscope status information ESI, recognizing a pain status and acquiring pain status information PSI, and generating insertion support information according to the pain status information PSI and the endoscope status information ESI.

[0047] Further, a part or all of the processing of the above-described insertion assist system 100 may be realized by a program. In that case, the insertion assist system 100 may be configured as follows.

[0048] The insertion support system 100 includes a memory that stores information, and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data. The program describes some or all of the functions of the endoscope status information acquisition unit 110, the pain status recognition unit 130, the insertion support information generation unit 150, and the examination condition information acquisition unit 160. The processor executes the program to realize some or all of the functions of the endoscope status information acquisition unit 110, the pain status recognition unit 130, the insertion support information generation unit 150, and the examination condition information acquisition unit 160.

[0049] The processor includes hardware, and the hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the processor may be configured with one or more circuit devices mounted on a circuit board, or one or more circuit elements. The one or more circuit devices may be, for example, ICs. The one or more circuit elements may be, for example, resistors, capacitors, etc. The processor may be, for example, a CPU (Central Processing Unit). However, the processor is not limited to a CPU, and various processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) may be used. The processor may also be an integrated circuit device such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The processor may also include an amplifier circuit, a filter circuit, etc. for processing analog signals. The memory may be a semiconductor memory such as an SRAM or a DRAM, a register, a magnetic storage device such as a hard disk device, or an optical storage device such as an optical disk device. For example, the memory stores computer-readable instructions, and the instructions are executed by the processor to realize the functions of each part of the insertion assistance system 100 as processes. The instructions here may be instructions in an instruction set constituting a program, or may be instructions that instruct the hardware circuitry of the processor to operate.

[0050] The program can be stored in a non-transitory information storage medium that is a computer-readable medium. The information storage medium can be realized by, for example, an optical disk, a memory card, a HDD, or a semiconductor memory. The semiconductor memory can be, for example, a ROM or a non-volatile memory.

[0051] 2. Endoscope System An example of the detailed configuration will be described below. Fig. 3 shows an example of the configuration of an endoscope system 400 including the insertion support system 100. The endoscope system 400 includes an endoscope device 300 and an insertion shape observation device 200.

[0052] The endoscope device 300 includes an endoscope 10, a light source device 330, a signal processing device 310, and a display device 320. The endoscope 10, also called a scope, is inserted into a patient's body and captures images of the inside of the patient's body. The light source device 330 generates and controls illumination light, which is guided to the tip of the endoscope 10 by a light guide and emitted from the tip of the endoscope 10. The signal processing device 310 processes an image signal output by the endoscope 10 to generate an endoscopic image. The signal processing device 310 also acquires the ID and the like of the endoscope 10 as endoscope type information. The display device 320 displays the endoscopic image generated by the signal processing device 310.

[0053] The insertion shape observation device 200 includes an endoscope shape acquisition sensor 20, a main device 210, and a display device 220. The endoscope shape acquisition sensor 20 detects a magnetic field of a source coil provided in the endoscope insertion part. The main device 210 acquires the position and shape of the endoscope insertion part based on a detection signal from the endoscope shape acquisition sensor 20, and outputs an image showing the position and shape of the endoscope insertion part to the display device 220. The display device 220 displays the image output by the main device 210. The display device 220 and the display device 320 are also called monitors, and are liquid crystal display devices or the like. Note that one display device may be provided in the endoscope system 400, and the endoscope device 300 and the insertion shape observation device 200 may share the one display device.

[0054] The insertion support system 100 is provided in a main body device 210. An endoscopic image and endoscope type information from a signal processing device 310, and information on the position and shape of an endoscope insertion portion acquired by the main body device 210 are input to the insertion support system 100. The insertion support system 100 may be provided anywhere in the endoscopic system 400.

[0055] 4 shows an example of the configuration of the endoscope 10 and the endoscope shape acquisition sensor 20. As shown in FIG. 4, the endoscope 10 includes an operation section 12, an endoscope insertion section 14, and a source coil 18.

[0056] The endoscope insertion section 14 has a flexible, elongated shape and includes a rigid section 16 provided at its tip and a bending section 15 that can be angled. The rigid section 16 is provided with an imaging device, an illumination lens, a water supply port, an air supply port, a forceps port, and the like.

[0057] The operation unit 12 is a device for the user to operate the endoscope 10, and includes, for example, a grip, an angle operation dial, and an air / water supply button. FIG. 5 shows an explanatory diagram of the endoscope operation. As shown in A1, the user pushes the grip in the longitudinal direction of the endoscope insertion portion 14, thereby inserting the endoscope insertion portion 14. This is called a push operation. Also, the user pulls the grip in the longitudinal direction of the endoscope insertion portion 14, thereby pulling out the endoscope insertion portion 14. This is called a pull operation. As shown in A2, the user turns the grip in the circumferential direction of the endoscope insertion portion 14, thereby rotating the endoscope insertion portion 14 in the circumferential direction. This is called a torque operation. When viewed from the grip toward the endoscope insertion portion 14, a clockwise torque operation is called a right torque operation, and a counterclockwise torque operation is called a left torque operation. As shown in A3 and A4, the user operates the angle operation dial, thereby bending the bending portion 15 of the endoscope insertion portion 14 up, down, left, and right. This is called an angle operation. The vertical angle adjustment and the horizontal angle adjustment can be performed independently of each other.

[0058] The source coil 18 generates a magnetic field. For example, a plurality of source coils 18 are provided at a predetermined interval in the endoscope insertion section 14. The endoscope shape acquisition sensor 20 detects the magnetic field from each source coil 18, and the main body device 210 of the insertion shape observation device 200 detects the position of each source coil 18 based on the detection signal, thereby detecting the position of each part of the endoscope insertion section 14. The main body device 210 also detects the shape of the endoscope insertion section 14 based on the detected positions of the plurality of source coils 18. Note that the sensing method for observing the insertion shape is not limited to a method using a magnetic field, and may be a method using, for example, electromagnetic waves, ultrasound, light, or the like.

[0059] 3. First detailed configuration example of the insertion support system 6 is a first detailed configuration example of the insertion support system 100. The insertion support system 100 includes an information acquisition unit 140 and a situation recognition unit 140. 120 and an insertion support information generating unit 150. In this configuration example, for example, a part of the information acquiring unit 140 and the latest situation determining unit 121 correspond to the endoscope situation information acquiring unit 110 in Fig. 1 and Fig. 2, and a part of the information acquiring unit 140 and the time-series situation determining unit 122 correspond to the examination condition information acquiring unit 160 in Fig. 2. Note that descriptions of components that are the same as those already described will be omitted as appropriate.

[0060] The information acquisition unit 140 acquires various pieces of information IFIN used to recognize a pain state. The information IFIN is, for example, an endoscopic image, insertion part shape information, input pain information, endoscope type information, patient information, past examination information, or any combination of two or more of them.

[0061] The situation recognition unit 120 recognizes the latest situation, time-series situation, and pain situation from the information IFIN, and outputs latest situation information TYK, time-series situation information JIK, and pain situation information PSI. The situation recognition unit 120 includes a latest situation determination unit 121, a time-series situation determination unit 122, and a pain situation recognition unit .

[0062] The most recent situation determination unit 121 recognizes the current insertion situation of the endoscope from information such as an endoscope image or insertion part shape information, and outputs the result as most recent situation information TYK. "Currently" includes not only the current moment, but also the most recent time including the present. For example, when an operation or operation procedure currently being performed is specified by a series of movements of the insertion part, the most recent time may be long enough to recognize the series of movements of the insertion part.

[0063] The time-series situation determination unit 122 recognizes the insertion situation in a time-series manner, including past situation recognition results, from information such as past examination information, and outputs the result as time-series situation information JIK. "Past" refers to a time before the above-mentioned "most recent time," and may be either within the current endoscopic examination or an endoscopic examination prior to the previous one.

[0064] The pain situation recognition unit 130 recognizes situations in which pain is likely to occur or situations in which pain has occurred from the most recent situation information TYK and the time-series situation information JIK, and outputs the result as pain situation information PSI. The pain situation recognition unit 130 also determines the current pain situation based on past recognition results by using not only the most recent situation information TYK but also the time-series situation information JIK.

[0065] The insertion support information generator 150 generates insertion support information appropriate for the situation indicated by the latest situation information TYK, the time-series situation information JIK, and the pain situation information PSI, based on the information. By presenting a guide appropriate for the situation, support for endoscope insertion with less pain is realized.

[0066] Situational Awareness Department 120 As an example, the inference algorithm is realized by machine learning using a neural network or the like. Specifically, a memory (not shown) stores a program in which an inference algorithm is described and parameters used in the inference algorithm as information of the trained model. Then, the processor performs processing based on the information of the trained model. In other words, the processor executes the program using the parameters stored in the memory, thereby realizing a situation recognition unit. 120 The situation recognition section executes the process. 120may be realized as a whole by one trained model, or each of the latest situation determination unit 121, the time-series situation determination unit 122, and the pain situation recognition unit 130 may be realized by an individual trained model. Also, only some of the latest situation determination unit 121, the time-series situation determination unit 122, and the pain situation recognition unit 130 may be realized by a trained model.

[0067] As the inference algorithm, for example, a neural network can be adopted. The weight coefficients of the connections between nodes in the neural network are parameters. The neural network includes an input layer to which input data is input, an intermediate layer that performs calculation processing on the data input through the input layer, and an output layer that outputs a recognition result based on the calculation result output from the intermediate layer. The inference algorithm is not limited to a neural network, and various machine learning techniques used in recognition processing can be adopted. Situation Recognition Unit 120 The learning process will be described below by taking as an example a case where the entirety of the above is realized by one trained model. In this case, the input data is information IFIN, and the recognition results are latest situation information TYK, time-series situation information JIK, and pain situation information PSI. A learning device that executes the learning process is, for example, an information processing device such as a PC. The learning device inputs training data to the learning model, and generates a trained model by providing feedback to the learning model based on the recognition result. The training data includes multiple sets of data, and each set includes input data and correct answer data. The correct answer data is the recognition result to be obtained for the input data, and is prepared in advance by a medical professional or the like.

[0068] Hereinafter, an example in which the insertion assist system 100 is applied to a colonoscope will be described, but the application of the insertion assist system 100 is not limited to a colonoscope.

[0069] Examples of latest situation information TYK (a) to (c) and an example of time-series situation information JIK (d) are shown.

[0070] (a) The immediate situation determination unit 121 acquires the following information by performing image recognition processing on the endoscopic image.

[0071] a1: The part where the endoscope insertion part is located. FIG. 7 is an explanatory diagram of the parts of the large intestine. The parts of the large intestine are the ascending colon, hepatic flexure, transverse colon, splenic flexure, descending colon, SDJ, sigmoid colon, and rectum from the cecum to the anus. The area near the hepatic flexure is also called the right colon, and the area near the splenic flexure is also called the left colon. SDJ stands for Sigmoid Descending colon Junction, and is the boundary between the sigmoid colon and the descending colon. The part is recognized from the image by utilizing the fact that the image features differ depending on the part. The length of the part of the endoscope insertion part that is inserted into the patient can be obtained from the insertion part shape information, and the part can also be estimated from this insertion length.

[0072] a2: Optimal distance, clay pipe shape, red ball, or residue. Optimal distance and clay pipe shape are the distance between the tip of the insertion part and the intestinal wall in the shaft holding shortening method. The optimal distance can be created by suction, pushing or pulling operation, and is the optimal distance between the insertion part and the intestinal wall at which the folds can be turned over by angle operation, torque operation, or both. By realizing the optimal distance, pushing operation is not required when passing the folds, and the occurrence of pain due to pushing operation can be suppressed. Clay pipe shape is a state where the distance is not optimal, and the tip of the insertion part and the intestinal wall are far away. Red ball is a state where the tip of the insertion part is in contact with the intestinal wall mucosa, and the image turns red with the mucosa due to this contact. Residue is a state where water, etc. remains on the intestinal wall.

[0073] a3: Intestinal movement. Intestinal movement can be back and forth, parallel movement, rotation, or peristalsis. Back and forth is the relative movement of the endoscope insertion part and the intestine in the direction of the optical axis. Parallel movement is the relative movement of the endoscope insertion part and the intestine in a direction perpendicular to the optical axis. Rotation is the relative rotation of the endoscope insertion part and the intestine around the optical axis. Peristalsis is the movement of the large intestine to expel its contents. transfer It is an exercise to achieve this.

[0074] a4: Suction or air insufflation. Suction is the process of sucking gas out of the large intestine. Suction draws the intestinal wall to the tip of the insertion tube. Air insufflation is the process of sending gas into the large intestine. Air insufflation causes the large intestine to expand, and the intestinal wall moves away from the tip of the insertion tube.

[0075] (b) The immediate situation determination unit 121 acquires the following information by performing recognition processing on the insertion portion shape information.

[0076] b1: Shape of the endoscope insertion part. The "shape" here refers to the shape at a certain moment, such as the present.

[0077] b2: The type of loop in the loop method. Also, the shape transition of the endoscope insertion part during, before and after the loop method is performed. The type of loop is an N loop, an alpha loop, an inverted alpha loop or a gamma loop, and is identified by the loop shape formed by the endoscope insertion part. FIG. 8 is an example of the shape transition of the endoscope insertion part. Here, the shape transition when the N loop is released is explained as an example. The loop method is used when passing the endoscope insertion part through the sigmoid colon. When the N loop is formed, the endoscope insertion part is approximately N-shaped. The N loop is gradually released by a pulling operation, and when released, the endoscope insertion part becomes approximately linear. In the middle of release, the endoscope insertion part becomes an intermediate shape between the approximately N-shaped and the approximately linear shape. By recognizing this shape transition, it is possible to recognize that the N loop has been properly released.

[0078] b3: Shape transition of the endoscope insertion part during, before, and after the axis-holding shortening technique. The axis-holding shortening technique is used when passing the endoscope insertion part through the sigmoid colon. In the axis-holding shortening technique, the endoscope passes through the sigmoid colon by repeatedly going over the folds of the intestinal wall through angle manipulation and folding the folds through torque manipulation. The shape transitions caused by these manipulations are detected.

[0079] b4: Shape transition of the endoscope insertion part during transverse colon manipulation. The transverse colon manipulation is a shortening of the transverse colon midtrans. As with the loop method, the shape transition caused by the transverse colon manipulation is detected.

[0080] b5: Deflection of the endoscope insertion part, or stretching of the intestinal wall by the endoscope insertion part. Deflection of the endoscope insertion part occurs when the tip of the insertion part does not move and the middle of the insertion part bends when a pushing operation is performed. Stretching of the intestinal wall by the endoscope insertion part occurs when the endoscope insertion part pushes the free colon of the large intestine, stretching the intestinal wall between the pushed part and the fixed colon of the large intestine. In Figure 7, the free colon is the sigmoid colon and the transverse colon, and the fixed colon is the rectum, the ascending colon, and the descending colon.

[0081] (c) The immediate situation determination unit 121 acquires the following operation recognition information based on the shape change of the endoscope insertion portion.

[0082] c1: Whether or not pushing, pulling, torqueing, or angling can be performed when performing the shaft holding shortening method, loop method, or shortening. Based on the displacement direction, displacement amount, or both of the endoscope insertion part, it is determined whether or not pain occurs when the operation is performed or continued.

[0083] (d) The time-series situation determination unit 122 acquires the following information based on the past examination information:

[0084] d1: Insertion support information, image recognition results, shape recognition results, or operation recognition information that has been generated in the past. This information is obtained by acquiring past inspection information stored in a storage device.

[0085] Next, a pain state recognized based on the latest state information TYK and the time-series state information JIK will be described.

[0086] The pain state recognition unit 130 recognizes the pain state by recognizing, based on the endoscope state information ESI, a situation in which pain occurs due to the intestinal wall of the large intestine stretching, a situation in which pain occurs due to the mesentery being pulled, a situation in which pain occurs due to the endoscope insertion part pushing the intestinal wall, or a situation in which pain occurs due to the relationship between the fixed colon of the large intestine and the endoscope insertion part. The "relationship between the fixed colon of the large intestine and the endoscope insertion part" refers to a relationship in which the endoscope insertion part pushes or pulls the fixed colon due to endoscope operation. Note that this relationship is estimated from the position, shape, positional displacement, shape displacement, etc. of the endoscope insertion part, rather than an estimate of force.

[0087] The above four situations are the main insertion situations that cause pain to the patient during colonoscopy. The pain situation recognition unit 130 recognizes these insertion situations, and can recognize the pain situation in which the patient is experiencing pain.

[0088] Specific examples of pain situations are shown below (1) to (23).

[0089] (1) A situation where a pushing operation was performed when no loop was formed. This painful situation is an insertion situation where the intestinal wall is stretched by the pushing operation when no loop is formed.

[0090] (2) A situation where a right torque operation is performed during an inverse alpha loop. The inverse alpha loop can be released by a left torque operation, but if a right torque operation is performed, the loop will not be released and pain may occur. The insertion situation where the inverse alpha loop is not released by this right torque operation is the present pain situation.

[0091] (3) A state in which a pushing operation was performed during the loop. This painful state is an insertion state in which the intestinal wall is stretched by a pushing operation when an N loop is formed.

[0092] (4) A situation in which an upward push operation toward the flank or head occurs near the SDJ due to angle manipulation of the endoscope. FIG. 9 is an explanatory diagram of this painful state. Since the SDJ is near the boundary between the sigmoid colon and the descending colon, when an upward push occurs near the SDJ, the descending colon, which is a fixed colon, is pushed, which may cause pain. The position of the source coil 18 provided in the endoscope insertion portion 14 is detected based on the position of the endoscope shape acquisition sensor 20. Therefore, the flank direction and head direction in the detected insertion portion shape can be determined based on the relative positional relationship between the patient and the endoscope shape acquisition sensor 20.

[0093] (5) When the endoscope is pushed up toward the head near the SDJ by torque and angle manipulation, as in (4), pain may occur because the descending colon, which is a fixed colon, is pushed up.

[0094] (6) A situation in which the endoscope insertion part was bent near the SDJ. This painful situation occurs when the endoscope insertion part is bent near the SDJ, which is close to the fixed colon, and the fixed colon is pushed. For example, this is a situation in which the endoscope insertion part is bent near the SDJ, which is close to the fixed colon, and the fixed colon is pushed.

[0095] (7) A situation where a pulling operation is performed when the endoscope is looped. Fig. 10 shows an explanatory diagram of this painful situation. Here, an α-loop is used as an example, but other loops may be used. An α-loop is released by a right torque operation. If a pulling operation is performed when an α-loop is formed, the loop is not released, and pain may occur.

[0096] (8) A situation where a pulling operation is performed when no loop is formed. This pain situation occurs when a pulling operation is performed when a loop is not formed, and the insertion is performed by pulling too far or in the wrong direction.

[0097] (9) A situation in which an operation is performed when adhesions are present. This painful situation is an insertion situation in which part of the endoscope insertion part gets caught on the intestinal wall during the pulling operation and does not move. Adhesion is suspected in the part of the intestinal wall where it gets caught. For example, the tip of the endoscope insertion part gets caught on an adhesion part. In this case, the tip does not move during the pulling operation and the shape of other parts changes. Or, the endoscopic image does not change. By recognizing these, this painful situation can be recognized.

[0098] (10) A situation in which a pushing or pulling operation is performed near the boundary between the fixed colon and the free colon. The boundary between the fixed colon and the free colon is the SDJ or the splenic flexure. As in (4), the fixed colon is pushed, which may cause pain.

[0099] (11) A situation in which the splenic flexure is pushed. Because the diaphragm is located cranially to the splenic flexure, pain may occur when the splenic flexure is pressed against the diaphragm by the pushing operation.

[0100] (12) A situation in which a pushing operation is performed when the tip of the endoscope insertion part is in either the splenic flexure, transverse colon, or hepatic flexure, and a re-loop is formed in the sigmoid colon. Figure 11 shows an explanatory diagram of this painful situation. Here, an example is shown in which the tip of the endoscope insertion part is in the splenic flexure. When a pushing operation is performed to pass through the splenic flexure, the tip of the insertion part may get caught on the splenic flexure and the insertion part near the sigmoid colon may bend. This bent part of the insertion part near the sigmoid colon is called a re-loop. In this situation, if a pushing operation is performed beyond a certain level, the re-loop will expand, causing the intestinal wall to stretch. This painful situation can be recognized by recognizing that the tip of the endoscope insertion part does not move and that the re-loop has been displaced beyond a certain level.

[0101] (13) A situation in which the tip of the endoscope insertion part is located at the splenic flexure, transverse colon, or hepatic flexure, and a pulling operation is performed when dealing with a re-loop in the sigmoid colon. Figure 12 is an explanatory diagram of this painful situation. Here, an example is shown in which the tip of the endoscope insertion part is located at the splenic flexure. The re-loop is released by a pulling operation, but if a pulling operation is performed while the bent tip of the insertion part is caught on the upper end of the descending colon, pain may occur. As with (12), this painful situation can be recognized from the difference between the movement of the tip of the insertion part and the movement of other parts.

[0102] (14) The left colon is pushed up from the splenic flexure by the pushing operation of the endoscope. Figure 13 shows an explanatory diagram of this pain state. When the pushing operation is performed while the endoscope insertion part is in contact with the splenic flexure, the splenic flexure is pushed up. This may cause the left colon to stretch or be pressed against the diaphragm.

[0103] (15) A situation in which the shortening operation pulls the descending colon down toward the anal side of the splenic flexure, or the right colon jumps up toward the head. When the shortening operation of the mittrans is performed, the tip of the insertion part of the right colon rises toward the head, and the insertion part near the splenic flexure drops toward the anal side. This may cause pain by pulling the ascending colon, which is a fixed colon, or pushing the descending colon, which is a fixed colon.

[0104] (16) A situation in which the splenic flexure is pushed up toward the head by the pushing operation when inserting the mitral transformer. When the endoscope insertion section is located in the hanging part of the transverse colon, the pushing operation may cause the insertion section near the splenic flexure to bend up toward the head, pushing up the splenic flexure.

[0105] (17) During the shortening operation, the right colon is pulled toward the left flank by the angle operation.

[0106] (18) A situation in which the hepatic curvature is pushed up by angle manipulation.

[0107] (19) A situation in which the hepatic curvature is pushed up by a pushing operation.

[0108] (20) When the tip of the endoscope insertion part is at the hepatic curvature, the splenic curvature is pushed up by the pushing operation.

[0109] (21) After reaching the hepatic flexure, the splenic flexure is pulled toward the anus by the pulling maneuver.

[0110] (22) A situation in which the splenic flexure is pulled toward the anus by a pulling maneuver on the splenic flexure.

[0111] (23) A situation in which the intestines expand due to air supply.

[0112] The pain state recognition unit 130 may recognize that a pain state occurs when the amount of displacement exceeds a predetermined value in each state. As shown in Fig. 14, in the state (3) above, when an N loop is detected and the amount of displacement of the convex portion of the loop is equal to or greater than a predetermined value, a pain state may be determined. Since the degree of extension changes depending on the amount of displacement, a pain state is recognized when the amount of displacement causes a certain level of extension or more.

[0113] The pain state recognition unit 130 may also determine the pain level based on the amount of displacement in each situation. For example, in Fig. 14, when the amount of displacement of the loop convex portion is equal to or greater than a first predetermined value, it may determine the pain level as the first pain level, and when the amount of displacement of the loop convex portion is equal to a second predetermined value, it may determine the pain level as the second pain level. When the second predetermined value is greater than the first predetermined value, the second pain level indicates that the pain is stronger than the first pain level.

[0114] Fig. 15 is a flowchart of the process performed by the insertion support system 100 in the first detailed configuration example. In this flowchart, the insertion support information generated in step S7 corresponds to the first insertion support information generated when there is no pain as described in Fig. 1. Also, the insertion support information generated in step S8 corresponds to the second insertion support information generated when there is pain as described in Fig. 1.

[0115] In step S1, the information acquisition unit 140 acquires information IFIN. In step S2, the latest situation determination unit 121 and the time-series situation determination unit 122 determine the endoscope insertion situation based on the information IFIN. In steps S3 and S4, the pain situation recognition unit 130 determines the presence or absence of a pain situation based on the latest situation information TYK and the time-series situation information JIK.

[0116] When the pain state recognition unit 130 determines in step S4 that a pain state exists, in step S5, the pain state recognition unit 130 determines the type and level of pain based on the insertion state determined in step S2. The pain state recognition unit 130 also determines the frequency of pain and the duration of pain occurrence based on the insertion state determined in step S2. In step S6, the pain state that occurred in the past is acquired based on the insertion state determined in step S2. In step S7, the insertion support information generation unit 150 generates insertion support information based on the pain state recognized by the pain state recognition unit 130 in steps S5 and S6. The insertion support information here is information indicating an operation to reduce pain or an operation to avoid pain. In step S8, the insertion support information generation unit 150 presents the insertion support information by outputting the generated insertion support information to a display device or the like.

[0117] The types of pain are the above-mentioned pain situations (1) to (23). Operations to reduce pain or avoid pain are, for example, an operation opposite to the operation that corresponds to the condition in which pain occurred or the condition in which pain is likely to occur, or manual pressure by an operation assistant, or changing the patient's position, etc. As an example, in the above-mentioned (3), when a pain situation occurs due to a push operation during an N-loop, a pull operation or manual pressure is presented.

[0118] In step S7, the insertion support information generating unit 150 changes the content of the operation to be presented depending on the type or level of the pain that has occurred. The insertion support information generating unit 150 also changes the content of the operation to be presented depending on how many times the same pain has occurred in the past, how long the same pain has occurred in the past, and whether a different pain has occurred in the past. If the number of times pain has occurred is high, the insertion support information generating unit 150 may also present a suggestion to change the scope, change the surgeon, or stop the examination. This pain information may also be recorded in the electronic medical record.

[0119] In step S4, when the pain state recognition unit 130 determines that there is no pain state, in step S9, a pain state that occurred in the past is acquired based on the insertion state determined in step S2. In step S10, the insertion support information generation unit 150 generates insertion support information based on the pain state recognized by the pain state recognition unit 130 in step S9. The insertion support information here is information indicating a normal operation, an operation that is unlikely to cause pain, or an operation that prevents the occurrence of pain. In step S11, the insertion support information generation unit 150 presents the insertion support information by outputting the generated insertion support information to a display device or the like.

[0120] An operation that is less likely to cause pain or that can prevent pain from occurring in the first place is, for example, a technique that does not stretch the intestine, such as the axial shortening technique.

[0121] In step S9, if there is no pain currently present in the insertion state but there has been pain in the past in that insertion state, the insertion support information generating unit 150 changes the information to insertion support information for an operation that will not cause that pain.

[0122] 16 is a flowchart showing a specific example of recognition of a pain state and generation of insertion support information according to the recognition result. Here, the axis holding shortening method is used as an example, but a processing flow corresponding to each insertion method or each of the above-mentioned pain states (1) to (23) is set.

[0123] In step S61, the immediate situation determination unit 121 determines whether or not the distance is optimal based on the information IFIN.

[0124] When the most recent situation determination unit 121 determines that the distance is optimal in step S61, the pain situation recognition unit 130 determines whether or not the situation is painful based on the insertion situation recognized by the most recent situation determination unit 121 and the time-series situation determination unit 122 in step S62. When the pain situation recognition unit 130 determines that the situation is not painful in step S62, the insertion support information generation unit 150 generates insertion support information indicating a right torque operation in step S63. When the pain situation recognition unit 130 determines that the situation is painful in step S62, the insertion support information generation unit 150 generates insertion support information having contents according to the type of pain or past pain situations, etc. in step S64.

[0125] When the immediate situation determination unit 121 determines in step S61 that the distance is not optimal, the pain situation recognition unit 130 determines in step S65 whether or not the situation is painful based on the insertion situation recognized by the immediate situation determination unit 121 and the time-series situation determination unit 122. When the pain situation recognition unit 130 determines in step S65 that the situation is not painful, the insertion support information generation unit 150 generates insertion support information indicating a pull operation in step S66. S67 In step S70, when the pain state recognition unit 130 determines that the pulling operation was performed when not in a loop, the pain state recognition unit 130 determines in steps S68 and S69 that the optimal distance cannot be achieved by the pulling operation, and determines that the optimal distance can be achieved by the pushing operation but that there is a possibility that pain will occur in the pushing operation. In step S70, the insertion support information generation unit 150 generates insertion support information indicating the combined use of manual pressure and the pushing operation.

[0126] Fig. 17 is a flow chart showing a specific example of recognition of a pain state. Here, a pain state in which stretching occurs in the sigmoid colon as shown in Fig. 23 will be described as an example.

[0127] In step S31, the immediate situation determination unit 121 determines the location where the endoscope insertion part is located based on the endoscope image and the insertion part shape information. Specifically, the immediate situation determination unit 121 performs image recognition on the endoscope image and shape recognition on the insertion part shape information, and determines the location where the endoscope insertion part is located based on the image recognition result and the shape recognition result.

[0128] When the immediate situation determining section 121 determines in step S31 that the endoscope insertion section is present in a part other than the sigmoid colon, the pain situation recognizing section 130 determines in step S32 that the pain situation is in another part.

[0129] When the immediate situation determination unit 121 determines in step S31 that the endoscope insertion portion is present in the sigmoid colon, the immediate situation determination unit 121 determines in step S33 whether the endoscope insertion portion is in an extended shape. The extended shape here is a walking stick shape that is convex toward the head and bent toward the left flank, as shown in FIG.

[0130] When the most recent situation determining section 121 determines in step S33 that the shape is not an extension shape, the pain situation recognizing section 130 determines in step S34 that the shape is another type of pain situation.

[0131] When the most recent situation determination unit 121 determines in step S33 that the shape is an extended shape, in step S35 the most recent situation determination unit 121 determines whether or not extension has occurred. Specifically, the most recent situation determination unit 121 determines extension based on whether the endoscope insertion unit has been displaced toward the head side while maintaining the extended shape, whether the endoscope image has stopped or receded in a section where extension has been recognized from the most recent recognition information, or both of these.

[0132] When the most recent situation determination section 121 determines in step S35 that stretching has not occurred, the pain situation recognition section 130 determines in step S36 that a pain situation caused by stretching has not occurred.

[0133] When the most recent situation determination unit 121 determines in step S35 that extension has occurred, the pain situation recognition unit 130 determines in step S37 whether the extension has exceeded a threshold value. Specifically, the pain situation recognition unit 130 determines whether the amount of displacement of the convex portion of the endoscope insertion portion has exceeded a threshold value, as shown in Fig. 23.

[0134] When the pain state recognizing section 130 determines in step S37 that the extension does not exceed the threshold value, the pain state recognizing section 130 determines in step S36 that a pain state caused by the extension has not occurred.

[0135] When the pain state recognizing section 130 determines in step S37 that the extension has exceeded the threshold, it determines in step S38 that a pain state caused by extension has occurred.

[0136] 4. Second detailed configuration example of the insertion support system 18 is a second detailed configuration example of the insertion support system 100. The insertion support system 100 includes an image acquisition unit 141, an endoscope shape acquisition unit 142, and a situation recognition unit 120 and an insertion support information generation unit 150. In this configuration example, for example, the image acquisition unit 141, the endoscope shape acquisition unit 142, and the latest situation determination unit 121 correspond to the endoscope situation information acquisition unit 110 in Fig. 1 and Fig. 2, and the time-series situation determination unit 122 corresponds to the examination condition information acquisition unit 160 in Fig. 2. Note that descriptions of components that are the same as those already described will be omitted as appropriate.

[0137] The pain information acquiring unit 500 is included in the endoscope system 400. For example, when the insertion assistance system 100 is included in the insertion shape observation device 200, the pain information acquiring unit 500 may be included in the insertion shape observation device 200.

[0138] The pain information acquisition unit 500 includes a transmission device 510 that can be operated by the patient or a medical professional depending on the patient's pain state. The transmission device 510 is, for example, a switch or a touch panel. Taking the case where the patient operates the switch as an example, the patient is asked to hold the switch and press the switch when pain occurs. The pain state recognition unit 130 recognizes the occurrence of pain based on information from the switch, and the insertion support information generation unit 150 presents insertion support information based on the recognition result. The switch may be a one-stage switch or a two-stage switch so that the strength and weakness of the pain can be known. The switch may be capable of transmitting only the presence or absence of pain, or may be capable of transmitting information other than the presence or absence of pain, such as the strength or weakness or the duration of pain. The switch may be held by an assistant such as a nurse, and the assistant may press the switch instead of the patient when the patient complains of pain.

[0139] The pain information acquisition unit 500 may also include a camera that captures the patient's facial expression, and may recognize pain by recognizing that the captured image is a facial expression of pain. The pain information acquisition unit 500 may also include a microphone that captures the voice emitted by the patient, and may recognize pain by recognizing from the voice that the patient has complained of pain. The output signal of the transmission device 510, the facial expression recognition result, or the voice recognition result is input to the situation recognition unit 500 as input pain information INPN. 120 is entered into

[0140] The image acquisition unit 141 acquires an endoscopic image by receiving the endoscopic image transmitted by the signal processing device 310 of the endoscopic device 300. The endoscope shape acquisition unit 142 acquires the position and shape of the endoscope insertion part based on the detection signal from the endoscope shape acquisition sensor 20. The endoscopic image and information on the position and shape of the endoscope insertion part are output to the situation recognition unit 142 as information IFIN. 120 is entered into

[0141] Fig. 19 is a flowchart of the process performed by the insertion support system 100 in the second detailed configuration example. In this flowchart, the insertion support information generated in steps S13 and S17 corresponds to the insertion support information generated when the input pain information INPN is not input, as described in Fig. 1. Also, the insertion support information generated in step S15 corresponds to the insertion support information generated when the input pain information INPN is input, as described in Fig. 1.

[0142] In step S11, the insertion support information generating unit 150 judges whether or not the pain state recognizing unit 130 has detected a pain state based on the internal information. The internal information is information that is not the input pain information INPN sent by the patient or medical staff, and in the example of Fig. 18, is the endoscopic image acquired by the image acquiring unit 141, the insertion part shape information acquired by the endoscope shape acquiring unit 142, the most recent situation information TYK output by the most recent situation determining unit 121, the time-series situation information JIK output by the time-series situation determining unit 122, or any combination of two or more of them.

[0143] When the pain state recognition unit 130 detects a pain state based on the internal information in step S11, the insertion support information generation unit 150 determines in step S16 whether the pain state recognition unit 130 detects a pain state based on external information. The external information is input pain information INPN transmitted by a patient or a medical professional.

[0144] In step S16, when the pain condition recognition unit 130 does not detect a pain condition based on the external information, the insertion support information generation unit 150 generates insertion support information of pain response guide A. Pain response guide A guides the operation corresponding to each of the above-mentioned pain conditions (1) to (23).

[0145] When the pain state recognition unit 130 detects a pain state based on external information in step S16, in step S18, the pain state recognition unit 130 records the detected pain state as a log in memory. In step S19, when the detected pain state corresponds to any of the above-mentioned predetermined pain states (1) to (23), the pain state recognition unit 130 updates the judgment condition of the corresponding pain state. For example, when the pain state is detected by judging the amount of displacement against a threshold, the threshold is updated. In step S15, the insertion support information generation unit 150 generates insertion support information of the pain response guide B. The pain response guide B guides an operation that corresponds to each of the above-mentioned pain states (1) to (23) and takes into account the patient's specific pain. The patient's specific pain is pain that occurs in an insertion state different from the pain states (1) to (23). The operation that takes into account the patient's specific pain is an operation that avoids the patient's specific pain state. In step S14, the insertion support information generating unit 150 causes the display device 220 to display the generated insertion support information.

[0146] In step S11, when the pain state recognition unit 130 does not detect a pain state based on internal information, in step S12, the insertion support information generation unit 150 determines whether the pain state recognition unit 130 has detected a pain state based on external information.

[0147] In step S12, when the pain state recognition unit 130 does not detect a pain state based on the external information, in step S13, the insertion support information generation unit 150 generates insertion support information of a normal guide for the current insertion state. The normal guide guides the operation when no pain occurs. In step S14, the insertion support information generation unit 150 causes the display device 220 to display the generated insertion support information.

[0148] When the pain state recognition unit 130 detects a pain state based on external information in step S12, the pain state is recorded in step S20. For example, the pain state is recorded in an electronic medical record or the like. When the pain state recognition unit 130 recognizes a pain state that does not fall under the above-mentioned pain states (1) to (23) based on external information, the pain state is regarded as a pain specific to the patient and recorded, so that if a similar pain state occurs again in the current examination or in another examination after the next one, the pain state specific to the patient can be recognized. Next, in step S15, the insertion support information generation unit 150 generates insertion support information of the pain response guide B. In step S14, the insertion support information generation unit 150 causes the display device 220 to display the generated insertion support information.

[0149] 5. Third and fourth detailed configuration examples of the insertion support system 20 shows a third detailed configuration example of the insertion support system 100. The insertion support system 100 includes an image acquisition unit 141, an endoscope shape acquisition unit 142, a patient information acquisition unit 143, an endoscope information acquisition unit 144, and a situation recognition unit. 120 and an insertion support information generating unit 150. FIG. 21 shows a fourth detailed configuration example of the insertion support system 100. The fourth detailed configuration example is obtained by further adding a pain information acquiring unit 500 to the third detailed configuration example. In the third and fourth detailed configuration examples, for example, the image acquiring unit 141, the endoscope shape acquiring unit 142, and the latest situation determining unit 121 correspond to the endoscope situation information acquiring unit 110 in FIG. 1 and FIG. 2, and the patient information acquiring unit 143, the endoscope information acquiring unit 144, and the time-series situation determining unit 122 correspond to the examination condition information acquiring unit 160 in FIG. 2. Note that the description of the same components as those already described will be omitted as appropriate.

[0150] The electronic medical record 600 accumulates patient information, which is information on the attributes of a patient. The electronic medical record 600 is stored in, for example, a storage device provided outside the endoscope system 400, and the patient information acquisition unit 143 acquires the electronic medical record 600 from the storage device.

[0151] The patient information is, for example, the patient's physical build, sex, age, medical history, body fat percentage, or any combination of two or more of them. Physical build is BMI, height, weight, or any combination of two or more of them. The pain condition or pain threshold that is likely to occur may differ depending on these patient attributes. As an example, there is a situation where thin women are prone to pain, or men feel pain when the N loop is extended 50 mm in the head direction. By recognizing the pain condition based on the patient information, the patient's unique pain condition is appropriately recognized, and appropriate insertion assistance is realized.

[0152] The patient information may also include past examination information, such as which insertion method was used, how many minutes it took to insert, whether pain occurred, the insertion trajectory, which scope was used, whether a sedative was used, or any combination of two or more of these. By recognizing the pain state based on this past examination information, the patient's unique pain state is appropriately recognized, and appropriate insertion assistance is realized.

[0153] The endoscope information acquisition unit 144 acquires endoscope type information from the signal processing device 310 of the endoscope device 300. The thickness of the endoscope insertion portion varies depending on the type of endoscope. Therefore, the pain that is likely to occur may vary depending on the type of endoscope. Therefore, by recognizing the pain state based on the endoscope type information, the pain state that differs depending on the type of endoscope is appropriately recognized, and appropriate insertion assistance is realized.

[0154] 6. Fifth detailed configuration example of the insertion support system 22 shows a fifth detailed configuration example of the insertion support system 100. The insertion support system 100 includes an image acquisition unit 141, an endoscope shape acquisition unit 142, a patient information acquisition unit 143, an endoscope information acquisition unit 144, and a situation recognition unit. 120 and an insertion support information generating unit 150. Note that the description of the same components as those already described will be omitted where appropriate.

[0155] The insertion support information generating unit 150 may control the automatic insertion / removal device 700 by outputting support information AST to the control device 710 of the automatic insertion / removal device 700. The automatic insertion / removal device 700 is a robot that automatically or semi-automatically inserts and removes an endoscope, and the control device 710 is a device that controls the robot. The support information AST is output as a control signal to the control device 710. When the pain state recognizing unit 130 recognizes a pain state, the insertion support information generating unit 150 may output, for example, a control signal to stop the operation of the automatic insertion / removal device 700, or may output a control signal to change the operation to a different operation from that when a pain state is not recognized.

[0156] Although the present embodiment and its modified examples have been described above, the present disclosure is not limited to each embodiment or its modified example as it is, and in the implementation stage, the components can be modified and embodied within the scope of the gist. In addition, a plurality of components disclosed in each of the above-mentioned embodiments and modified examples can be appropriately combined. For example, some components may be deleted from all the components described in each embodiment or modified example. Furthermore, the components described in different embodiments or modified examples may be appropriately combined. In this way, various modifications and applications are possible within the scope of the gist of the present disclosure. In addition, a term described at least once in the specification or drawings together with a different term having a broader meaning or the same meaning can be replaced with the different term in any part of the specification or drawings. [Explanation of symbols]

[0157] 10 endoscope, 12 operation unit, 14 endoscope insertion unit, 15 bending unit, 16 rigid unit, 18 source coil, 20 endoscope shape acquisition sensor, 100 insertion support system, 110 endoscope status information acquisition unit, 120 status recognition unit, 121 immediate status determination unit, 122 time-series status determination unit, 130 status recognition unit, 140 information acquisition unit, 141 image acquisition unit, 142 endoscope shape acquisition unit, 143 patient information acquisition unit, 144 endoscope information acquisition unit, 150 insertion support information generation unit, 160 examination condition information acquisition unit, 200 insertion shape observation device, 210 main body device, 220 display device, 300 endoscope device, 310 signal processing device, 320 display device, 330 light source device, 400 endoscope system, 500 information acquisition unit, 510 transmission device, 600 electronic medical record, 700 Automatic insertion / removal device, 710 control device, AST support information, ESI endoscope status information, INPN input pain information, JIK time-series status information, PSI pain status information, TYK immediate status information

Claims

1. A processor is included. The processor, In an endoscopic examination using an endoscope, endoscope status information including at least one of an endoscopic image, insertion part shape information which is information on the shape of an endoscope insertion part, and operation recognition information which is information on a change in at least one of the shape and the position of the endoscope insertion part is acquired; At least one of input pain information, which is information regarding the pain of the patient acquired from a patient or a medical professional in real time, and the endoscope status information is input, and a pain status, which is a status in which the patient is experiencing pain during the endoscopy, is recognized to acquire pain status information; Obtaining examination condition information including at least one of endoscope type information, which is information on the type of the endoscope insertion part used in the endoscopic examination, patient information, which is information on the attributes of the patient, and past examination information, which is information on past endoscopic examinations; An insertion assistance system, comprising: a first section for generating insertion assistance information according to the pain condition information, the endoscope condition information, and the examination condition information;

2. In claim 1, An insertion assistance system characterized in that, when both the input pain information and the endoscope status information are input, the processor recognizes the pain status based on the input pain information.

3. In claim 2, The processor, When the endoscope status information is input but the input pain information is not input, the pain status is recognized based on the endoscope status information, and the insertion support information is generated according to the pain status recognized based on the endoscope status information; An insertion assistance system characterized in that, when the input pain information is input in addition to the endoscope status information, the pain situation is recognized based on the input pain information, and the insertion assistance information is generated in accordance with the pain situation recognized based on the input pain information.

4. In claim 1, The processor, classifying an insertion status of the endoscope based on at least one of the endoscope image, the insertion portion shape information, or the operation recognition information, and outputting the endoscope status information including a classification result; An insertion assistance system characterized by recognizing whether the insertion situation indicated by the classification result corresponds to a situation in which pain occurs, thereby recognizing the painful situation.

5. In claim 4, the operation recognition information includes insertion portion shape change information which is information on a shape change of the endoscope insertion portion, The processor performs the classification based on the insertion portion shape information and the insertion portion shape displacement information.

6. In claim 5, The insertion portion shape displacement information includes shape displacement amount information which is information on the magnitude of the shape displacement, The processor performs the classification based on the insertion portion shape information and the shape displacement amount information.

7. In claim 1, The processor, when generating the insertion support information corresponding to the insertion status of the endoscope based on the endoscope status information, generates first insertion support information as the insertion support information when pain status information indicating that the situation is not one in which pain will occur is input, and generates second insertion support information different from the first insertion support information as the insertion support information when pain status information indicating that the situation is one in which pain will occur is input.

8. In claim 1, The processor recognizes, based on the endoscope status information, a situation in which the pain occurs due to the stretching of the intestinal wall of the large intestine, a situation in which the pain occurs due to the mesentery being pulled, a situation in which the pain occurs due to the endoscope insertion part pressing on the intestinal wall, or a situation in which the pain occurs due to the relationship between the fixed colon of the large intestine and the endoscope insertion part, thereby recognizing the pain situation.

9. In claim 1, The processor recognizes, based on the endoscope status information, that a pushing operation toward the flank or head is performed near the SDJ by angle manipulation of the endoscope, thereby recognizing that the pain is occurring.

10. In claim 1, The insertion assistance system is characterized in that when the processor recognizes, based on the endoscope status information, a situation in which a pushing operation toward the head is performed near the SDJ by torque operation and angle operation of the endoscope, it recognizes that the pain is occurring.

11. In claim 1, The insertion assistance system is characterized in that when the processor recognizes a situation in which the endoscope insertion portion is bent near the SDJ based on the endoscope status information, it recognizes that the pain is occurring.

12. In claim 1, The insertion assistance system is characterized in that, when the processor recognizes a situation in which a pulling operation is performed while the endoscope is looped, based on the endoscope status information, it recognizes that the pain is occurring.

13. In claim 1, The insertion assistance system is characterized in that, when the processor recognizes, based on the endoscope status information, a situation in which a pushing operation is performed when the tip of the endoscope insertion portion is present at either the splenic flexure, the transverse colon, or the hepatic flexure and a re-loop is formed in the sigmoid colon, it recognizes that the pain is occurring.

14. In claim 1, The insertion assistance system is characterized in that, when the processor recognizes, based on the endoscope status information, a situation in which the tip of the endoscope insertion portion is present at the splenic flexure, transverse colon, or hepatic flexure and a pulling operation is performed when dealing with re-looping in the sigmoid colon, it recognizes that the pain is occurring.

15. In claim 1, The insertion assistance system is characterized in that, when the processor recognizes, based on the endoscope status information, a situation in which the left colon is pushed up from the splenic flexure by the pushing operation of the endoscope, it recognizes that the pain is occurring.

16. In claim 1, An insertion assistance system characterized in that the input pain information is at least one of patient facial expression information, which is information regarding the patient's facial expression during the endoscopic examination, and pain transmission information, which is information from a transmission device operated by the patient or a medical professional.

17. In claim 1, The processor recognizes the pain situation by distinguishing between situations with different pain levels, situations with different frequencies of pain, or situations with different ease of avoiding pain.

18. In claim 1, The processor, Obtaining examination condition information including at least one of endoscope type information, which is information on the type of the endoscope insertion part used in the endoscopic examination, patient information, which is information on the attributes of the patient, and past examination information, which is information on past endoscopic examinations; An insertion assistance system characterized by recognizing the pain state based on the examination condition information.

19. An endoscope used for endoscopic examination; A processor; Including, The processor, Acquire endoscope status information including at least one of an endoscope image, insertion part shape information which is the shape of an endoscope insertion part, or operation recognition information which is information on at least one change in the shape or position of the endoscope insertion part during the endoscopic examination using the endoscope, At least one of input pain information, which is information regarding the pain of the patient acquired from a patient or a medical professional in real time, and the endoscope status information is input, and a pain status, which is a status in which the patient is experiencing pain during the endoscopy, is recognized to acquire pain status information; Obtaining examination condition information including at least one of endoscope type information, which is information on the type of the endoscope insertion part used in the endoscopic examination, patient information, which is information on the attributes of the patient, and past examination information, which is information on past endoscopic examinations; An endoscope system comprising: an endoscope generating unit generating insertion assistance information in accordance with the pain condition information, the endoscope condition information, and the examination condition information.

20. A method of operating an insertion assist system including a processor, comprising: The processor acquires endoscope status information including at least one of an endoscopic image, insertion part shape information which is a shape of an endoscope insertion part, and operation recognition information which is information on a change in at least one of a shape or a position of the endoscope insertion part during an endoscopic examination using an endoscope; The processor receives at least one of input pain information, which is information regarding the pain of the patient acquired from a patient or a medical professional in real time, and the endoscope status information, and recognizes a pain status, which is a status in which the patient is experiencing pain during the endoscopy, to acquire pain status information; The processor acquires examination condition information including at least one of endoscope type information, which is information on the type of the endoscope insertion part used in the endoscopic examination, patient information, which is information on the attributes of the patient, and past examination information, which is information on past endoscopic examinations; the processor generates insertion support information according to the pain state information, the endoscope state information, and the examination condition information; A method for operating an insertion assist system comprising:

Images