Endoscopic system, image processing method, and image processing program

Abstract

This endoscopic system comprises: an endoscope that includes an insertion part in which an imaging element is disposed at the distal end; a temperature sensor that detects a temperature on the outer peripheral surface of the insertion part; a gas supply device that supplies gas through a gas supply channel provided in the insertion part; and a processor. The processor generates diagnosis support information, which supports diagnosis of gastroesophageal reflux disease, on the basis of a captured image captured by the imaging element and esophageal temperature information relating to the temperature detected by the temperature sensor.

Description

Endoscope system, image processing method, and image processing program

[0001] The present invention relates to an endoscope system, an image processing method, and an image processing program.

[0002] Gastroesophageal reflux disease (GERD) is a pathological condition caused by the reflux of gastric contents rich in gastric acid into the esophagus. Conventionally, an endoscope system using an endoscope has been proposed as a system for diagnosing gastroesophageal reflux disease (see, for example, Patent Document 1). In the endoscope system described in Patent Document 1, the insertion portion of the endoscope is inserted into the stomach, the gastroesophageal junction is set to be imaged using the endoscope, and air is supplied into the stomach. Then, in this endoscope system, based on the pressure change in the stomach detected during the air supply into the stomach and the state of the gastroesophageal junction observed from the captured image captured using the endoscope, the function of the lower esophageal sphincter is evaluated.

[0003] International Publication No. 2021 / 166127

[0004] However, in the diagnosis of gastroesophageal reflux disease, it is necessary to evaluate not only the function of the lower esophageal sphincter but also the motility function of the esophagus. For this reason, in addition to the examination using the endoscope system described in Patent Document 1, it is necessary to additionally evaluate the motility function of the esophagus by, for example, high-resolution manometry (HRM). This high-resolution manometry cannot be performed with the endoscope system described in Patent Document 1. For this reason, in addition to the examination using the endoscope system described in Patent Document 1, it is necessary to perform this high-resolution manometry with another examination device, which imposes a burden on the examinee. Therefore, there is a demand for a technique capable of diagnosing gastroesophageal reflux disease while reducing the burden on the examinee.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an endoscope system, an image processing method, and an image processing program capable of reducing the burden on an examinee.

[0006] To solve the above-mentioned problems and achieve the objective, the endoscope system according to the present invention comprises an endoscope having an insertion section with an image sensor positioned at its tip, a temperature sensor for detecting the temperature applied to the outer surface of the insertion section, an air supply device for supplying gas through an air supply conduit provided in the insertion section, and a processor. The processor generates diagnostic support information to assist in the diagnosis of gastroesophageal reflux disease based on the image captured by the image sensor and esophageal temperature information related to the temperature detected by the temperature sensor.

[0007] The image processing method according to the present invention generates diagnostic support information that assists in the diagnosis of gastroesophageal reflux disease, based on an image captured by an image sensor provided at the tip of the insertion portion of an endoscope and esophageal temperature information related to the temperature detected by a temperature sensor that detects the temperature applied to the outer surface of the insertion portion.

[0008] The image processing program according to the present invention causes a processor to execute a process that generates diagnostic support information to assist in the diagnosis of gastroesophageal reflux disease, based on an image captured by an image sensor provided at the tip of the insertion portion of an endoscope and esophageal temperature information related to the temperature detected by a temperature sensor that detects the temperature applied to the outer surface of the insertion portion.

[0009] According to the endoscopic system, image processing method, and image processing program of the present invention, the burden on the person being examined can be reduced.

[0010] Figure 1 is a diagram illustrating the configuration of an endoscope system according to an embodiment. Figure 2 is a diagram illustrating the configuration of an endoscope system according to an embodiment. Figure 3 is a diagram illustrating the configuration and arrangement of a temperature sensor and a pressure sensor. Figure 4 is a diagram illustrating the temperature sensor when the insertion part is inserted into the esophagus. Figure 5 is a diagram illustrating an example of the temperature measured by each temperature sensor. Figure 6 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. Figure 7 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. Figure 8 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. Figure 9 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. Figure 10 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. Figure 11 is a diagram illustrating an example of the display of diagnostic support information. Figure 12 is a diagram illustrating an example of the display of diagnostic support information. Figure 13 is a diagram illustrating an example of the display of diagnostic support information.

[0011] The embodiments for carrying out the present invention (hereinafter referred to as "embodiments") will be described below with reference to the drawings. However, the present invention is not limited to the embodiments described below. Furthermore, in the drawings, the same parts are denoted by the same reference numerals.

[0012] [Configuration of the Endoscope System] Figures 1 and 2 show the configuration of the endoscope system 1 according to an embodiment. The endoscope system 1 is used in the medical field and is a system for diagnosing gastroesophageal reflux disease in a patient using an endoscope 2. As shown in Figures 1 and 2, the endoscope system 1 comprises an endoscope 2, a light source device 3, a processing device 4, a display device 5, an air insufflation device 6, a gastric pressure measuring device 7, and a microphone 8.

[0013] In this embodiment, the endoscope 2 is a so-called flexible endoscope. Part of this endoscope 2 is inserted into the living body, images are captured within the living body, and an image signal generated by the imaging is output. As shown in Figure 1, the endoscope 2 comprises an insertion section 21, an operating section 22, and a universal cord 23.

[0014] The insertion portion 21 is a part that is inserted into the living body and is at least partially flexible. As shown in Figures 1 and 2, the insertion portion 21 comprises a tip portion 24, a flexible curved portion 25 (Figure 1) composed of multiple curved pieces, and a long, flexible flexible tube portion 26 (Figure 1) connected to the base end of the curved portion 25. An image sensor 244 (Figure 2) is built into the tip portion 24. The insertion portion 21 is inserted into the body cavity of the subject and uses the image sensor 244 to image subjects such as living tissue located in a position where external light cannot reach.

[0015] Here, the outer circumferential surface of the insertion portion 21 is provided with a plurality of temperature sensors 9 for detecting the temperature on the outer circumferential surface and a pressure sensor 10 for detecting the pressure applied to the outer circumferential surface. The detailed configuration and arrangement of the temperature sensors 9 and pressure sensors 10 will be explained later in the section "Configuration and Arrangement of Temperature Sensors and Pressure Sensors".

[0016] The operating unit 22 is connected to the proximal end portion of the insertion unit 21. The operating unit 22 receives various operations on the endoscope 2. As shown in Figure 1, the operating unit 22 includes a bending knob 221 that bends the bending unit 25 in the vertical and horizontal directions, a treatment instrument insertion unit 222 that extends from the operating unit 22 to the tip of the insertion unit 21 and inserts treatment instruments such as biopsy forceps, electrosurgical units and examination probes into the body cavity of the subject, an air supply line 223 (Figure 2) that extends from the operating unit 22 to the tip of the insertion unit 21 and supplies air into the body cavity of the subject, and a plurality of switches 224 for operating peripheral equipment such as an air supply device 6 and a water supply device (not shown).

[0017] The universal cord 23 incorporates at least a light guide 241 (Figure 2) and a bundled cable 245 (Figure 2) that bundles one or more signal lines. The light guide 241 is made of glass fiber or the like and forms a light guide path for the light emitted by the light source device 3. As shown in Figure 1, the universal cord 23 branches at the end opposite to the end connected to the operation unit 22. At the branched end of the universal cord 23, a connector 231 that can be attached to the light source device 3 and a connector 232 that can be attached to the processing unit 4 are provided. A part of the light guide 241 extends from the end of the connector 231. The universal cord 23 propagates the illumination light emitted from the light source device 3 to the tip 24 via the connector 231 (light guide 241), the operation unit 22, and the flexible tube section 26. The universal cord 23 also transmits the image signal captured by the image sensor 244 provided at the tip 24 to the processing unit 4 via the connector 232. The bundled cable 245 includes signal lines for transmitting image signals, signal lines for transmitting drive signals for driving the image sensor 244, and signal lines for sending and receiving information including unique information about the endoscope 2 (image sensor 244). In this embodiment, it is described as transmitting electrical signals using signal lines, but optical signals may be transmitted, or signals may be transmitted between the endoscope 2 and the processing unit 4 by wireless communication.

[0018] The exit end of the light guide 241 is inserted into the tip portion 24. As shown in Figure 2, this tip portion 24 includes an illumination lens 242, a light-gathering optical system 243, and an image sensor 244 provided at the imaging position of the optical system 243, which receives the light gathered by the optical system 243, converts it into an electrical signal via photoelectricity, and performs predetermined signal processing.

[0019] The optical system 243 is composed of one or more lenses and forms an observation image on the light-receiving surface of the image sensor 244. The optical system 243 may also have an optical zoom function to change the angle of view and a focus function to change the focal point.

[0020] The image sensor 244 converts light from the optical system 243 into electrical signals (image signals). This image sensor 244 is made up of multiple pixels arranged in a matrix, each having a photodiode that stores charge according to the amount of light, and a capacitor that converts the charge transferred from the photodiode into a voltage level. The image sensor 244 generates electrical signals by converting the light incident on it through the optical system 243 into electrical signals, and sequentially reads out the electrical signals generated by pixels arbitrarily set as readout targets from among the multiple pixels, and outputs them as image signals. This image sensor 244 can be realized using, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0021] For the sake of clarity, in the following explanation, the image signal generated by the image sensor 244 will be referred to as the captured image.

[0022] Here, the endoscope 2 has a memory (not shown) that stores data including execution programs and control programs for the image sensor 244 to perform various operations, as well as identification information for the endoscope 2. This identification information includes the endoscope 2's unique information (ID), year of manufacture, specifications, and transmission method. The memory may also temporarily store images generated by the image sensor 244.

[0023] As shown in Figure 2, the light source device 3 comprises a light source unit 31, an illumination control unit 32, and a light source driver 33.

[0024] The light source unit 31 emits light under the control of the illumination control unit 32. This light source unit 31 emits light having a visible light wavelength range (white light (illumination light)). This light source unit 31 can be implemented using an LED (Light Emitting Diode) light source, as well as a laser light source, xenon lamp, halogen lamp, or any other light source. The light source unit 31 may also have one or more lenses. The light generated by the light source unit 31 is then emitted from the tip of the tip unit 24 toward the subject via the light guide 241 and illumination lens 242.

[0025] Furthermore, the light emitted from the light source unit 31 is not limited to white light; it may also be narrowband light having a specific wavelength range, or excitation light that excites substances contained in the object being observed.

[0026] The light source driver 33, under the control of the lighting control unit 32, supplies current to the light source unit 31, thereby causing the light source unit 31 to emit light.

[0027] The processing unit 4 corresponds to the processor according to the present invention. As shown in Figure 2, the processing unit 4 comprises an image processing unit 41, a synchronization signal generation unit 42, an input unit 43, a control unit 44, and a storage unit 45.

[0028] The image processing unit 41, under the control of the control unit 44, performs predetermined image processing on the captured image received from the endoscope 2 to generate an endoscopic image.

[0029] Examples of image processing performed by the image processing unit 41 include optical black subtraction (clamping), white balance adjustment, demosaicing, color correction matrix processing, gamma correction, YC processing to convert RGB signals into luminance chromatic difference signals (Y, Cb / Cr signals), digital gain adjustment to multiply digital gain, noise reduction, and filtering to enhance structure.

[0030] Furthermore, the image processing unit 41 generates first support information based on the pressure signal (intragastric pressure information) detected by the intragastric pressure measuring device 7, under the control of the control unit 44.

[0031] Furthermore, the image processing unit 41 generates second support information based on the temperature signal (esophageal temperature information) detected by the temperature sensor 9, under the control of the control unit 44.

[0032] Furthermore, the image processing unit 41 generates third support information based on signals related to the muffled sound (muffled sound information) collected by the microphone 8, under the control of the control unit 44.

[0033] The image processing unit 41 then generates diagnostic support information for diagnosing gastroesophageal reflux disease based on the endoscopic image and the first to third support information. This diagnostic support information is output to the display device 5 and displayed on the display device 5.

[0034] Further details regarding the first to third support information and diagnostic support information will be explained in "Examples of Displaying Diagnostic Support Information" below.

[0035] The image processing unit 41 described above is configured using a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor such as an ASIC (Application Specific Integrated Circuit) or various arithmetic circuits that perform specific functions.

[0036] The synchronization signal generation unit 42 generates a clock signal (synchronization signal) that serves as the reference for the operation of the processing unit 4, and outputs the generated synchronization signal to the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2. Here, the synchronization signal generated by the synchronization signal generation unit 42 includes a horizontal synchronization signal and a vertical synchronization signal. As a result, the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2 operate in synchronization with each other based on the generated synchronization signal.

[0037] The input unit 43 is implemented using a keyboard, mouse, switches, or touch panel, and accepts various operations to instruct the operation of the endoscope system 1. The input unit 43 may also include switches provided on the operation unit 22 or portable terminals such as external tablet computers.

[0038] The control unit 44 is configured using a general-purpose processor such as a CPU or a dedicated processor such as an ASIC that performs specific functions using various arithmetic circuits.

[0039] The storage unit 45 stores data including various programs executed by the control unit 44 and various parameters necessary for the processing of the control unit 44. These programs can be recorded on computer-readable recording media such as hard disks, flash memory, CD-ROMs, DVD-ROMs, and flexible disks and widely distributed. These programs can also be obtained by downloading them via a communication network. The communication network referred to here can be implemented by existing public telephone networks, LANs (Local Area Networks), WANs (Wide Area Networks), etc., and can be wired or wireless.

[0040] The memory unit 45 having the above configuration is implemented using a ROM (Read Only Memory) on which various programs are pre-installed, and a RAM or hard disk that stores calculation parameters and data for each process.

[0041] In this embodiment, the light source device 3 and the processing device 4 were housed in separate enclosures, but this is not limited to this configuration; they may be housed integrally within the same enclosure.

[0042] The display device 5 displays the display image received from the processing device 4 (image processing device 41) via a video cable. This display device 5 is configured using a monitor such as a liquid crystal or organic EL (Electro Luminescence).

[0043] The air supply device 6 adjusts the pressure of the gas supplied from a gas supply source (not shown, for example, a carbon dioxide cylinder) to a predetermined pressure and discharges it through the air supply pipe 223 from the tip of the insertion part 21 into the space where the tip is located. As shown in Figure 1, the air supply device 6 comprises an air supply unit 61, a flow rate measuring unit 62, and a control unit 63.

[0044] Although not shown in detail, the air supply unit 61 includes a primary pressure reducer, a secondary pressure reducer, and a flow control valve. These primary pressure reducer, secondary pressure reducer, and flow control valve are connected in this order by an air supply pipeline formed of silicon, fluororesin, or the like. The gas supplied from a gas supply source not shown passes through the air supply pipeline in the order of the primary pressure reducer, secondary pressure reducer, and flow control valve, is adjusted to a predetermined pressure and flow rate, and then is discharged from the air supply tube TU (FIG. 1) via the flow measurement unit 62. The control unit 63 controls the flow control valve provided in the air supply unit 61 to adjust the flow rate of the gas supplied to the endoscope 2 to a predetermined value. The flow control valve is, for example, a kind of electromagnetic drive valve and is constituted by a regulating valve using an electromagnetic coil in a drive unit. By controlling the position of the plunger according to the magnitude of the current flowing through the electromagnetic coil, the opening degree of the valve unit is controlled, and the flow rate of the gas flowing through the air supply pipeline is adjusted to a predetermined value.

[0045] The intragastric pressure measuring device 7 corresponds to the internal pressure sensor according to the present invention. This intragastric pressure measuring device 7 detects the pressure inside the space where the tip is located via the pressure measuring probe 71 inserted to the tip of the insertion unit 21 through the treatment tool insertion unit 222. Then, a signal regarding the pressure detected by the intragastric pressure measuring device 7 (hereinafter referred to as intragastric pressure) (hereinafter referred to as intragastric pressure information) is output to the processing device 4.

[0046] The microphone 8 is disposed on the throat or the like of the subject and collects the eructation sound (gup sound) emitted from the esophagus of the subject. Then, a signal regarding the eructation sound collected by the microphone 8 (hereinafter referred to as eructation sound information) is output to the processing device 4.

[0047] 〔Regarding the configuration and arrangement of the temperature sensor and the pressure sensor〕 FIG. 3 is a diagram for explaining the configuration and arrangement of the temperature sensor 9 and the pressure sensor 10.

[0048] The temperature sensor 9 detects temperature by a known method. For example, it is constituted by a sensor forming a thermocouple or a thermistor.

[0049] As shown in Figure 3, the temperature sensor 9 in this embodiment is a circular temperature sensor provided around the entire circumference in the rotational direction of the insertion portion 21, centered on a central axis along the axial direction. For example, a sheet-type sensor can be used for the temperature sensor 9. In this case, it is preferable to use a flexible sensor that has flexibility. Note that the temperature sensor 9 is not limited to a circular temperature sensor; a point-type temperature sensor provided only on a part of the entire circumference in the rotational direction may also be used.

[0050] The temperature sensors 9 are arranged on the outer surface of the insertion portion 21 at 1 cm intervals, totaling 36 sensors, along the axial direction of the insertion portion 21, as shown in Figure 3. This ensures that when the insertion portion 21 is inserted into the stomach, the temperature sensors 9 are positioned to detect the temperature of the entire esophagus, from the upper esophageal sphincter (UES) to the lower esophageal sphincter (LES).

[0051] Note that the number of temperature sensors 9 is not limited to 36; for example, there may be 6 or more, 12 or more, or even more sensors arranged.

[0052] Here, the temperature sensor 9 can be positioned at a predetermined distance from one or more other temperature sensors 9 that are closest to it. Optionally, the spacing between each temperature sensor 9 can be approximately the same. This spacing may be 3 centimeters or less, for example, 2 centimeters or less, for example, 1 centimeter, or even less than 1 centimeter.

[0053] The pressure sensor 10 is configured to detect pressure using a known method, for example, a resistive pressure sensor or a capacitive pressure sensor. As shown in Figure 3, the pressure sensor 10 according to this embodiment is a circular pressure sensor provided around the entire circumference in the rotational direction of the insertion portion 21, centered on a central axis along the axial direction. The pressure sensor 10 is provided further forward than the tip of the temperature sensors 9, which are arranged in the longitudinal direction of the insertion portion 21. Note that the pressure sensor 10 is not limited to a circular pressure sensor; a point-type pressure sensor provided only on a part of the entire circumference in the rotational direction may also be used.

[0054] The temperature signals (hereinafter referred to as esophageal temperature information) and pressure signals detected by the temperature sensor 9 and pressure sensor 10 described above are output to the processing unit 4.

[0055] Figure 4 illustrates the temperature sensor when the insertion part is inserted into the esophagus. When the insertion part 21 is inserted into the esophagus, the detected temperature changes depending on the distance to the esophageal lumen wall and the manner of contact, due to the shape of the esophageal lumen wall and the contraction and relaxation at various points inside the esophagus.

[0056] Figure 5 illustrates an example of the temperature measured by each temperature sensor. In Figure 5, six temperature sensors S1 to S6 are arranged in the axial direction of the insertion section 21, starting from the tip side (stomach side). Figure 5 shows the temperatures detected by temperature sensors S1 to S6 at a certain time. As shown in Figure 5, the temperature detected differs depending on the distance and contact method between each temperature sensor and the esophageal wall. For example, when a temperature sensor is in contact with the esophageal wall, the detected temperature is higher than when the temperature sensor is not in contact with the esophageal wall. Also, if the pressure the temperature sensor receives from the esophageal wall is high, the detected temperature will be higher than when the pressure is low.

[0057] [Method for Diagnosing Gastroesophageal Reflux Disease] Next, a method for diagnosing gastroesophageal reflux disease will be described. Figures 6 to 10 are diagrams illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. Specifically, Figure 6 is a flowchart illustrating a method for diagnosing gastroesophageal reflux disease. Figure 7 is a cross-sectional view showing the area around the gastric cardia, illustrating steps S1 to S3. Note that in Figure 7, for the sake of explanation, the temperature sensor 9 provided on the outer circumferential surface of the insertion portion 21 is not shown. Figures 8 and 7 are diagrams showing the structure of the stomach and esophagus. Note that Figure 8 is a cross-sectional view showing the structure of the stomach and esophagus. Also, Figure 7 is a view of the stomach and esophagus from the outside. Figure 9 is a diagram showing images taken in Phase 1 to Phase 3, which correspond to the dynamic changes that occur when air is insufflated into the stomach (during pneumoperitoneum).

[0058] First, the user, such as a physician, inserts the insertion part 21 into the person being examined through a natural opening, such as the mouth or nose, and guides the insertion part 21 into the stomach through the esophagus, as shown in Figure 7 (Step S1).

[0059] After step S1, the user, such as a physician, operates the bending knob 221 (bending operation) to set the gastric cardia as the observation field of view (step S2). In step S2, in response to the operation of the bending knob 221, the insertion part 21 is set into a J-shape with its tip facing the gastric cardia, as shown in Figure 7. In this state, the temperature sensor 9 is positioned to detect the temperature at various points inside the esophagus from the upper esophageal sphincter to the lower esophageal sphincter, and performs temperature measurements at, for example, a preset time interval.

[0060] After step S2, the user, such as a physician, operates switch 224 to start insufflating air from the air insufflation device 6 into the stomach through the air insufflation tube 223 (step S3). In Figure 7, the increase in gastric pressure due to air insufflation is represented by a white arrow.

[0061] During insufflation of air into the stomach (pneumoperitoneum), if the subject is healthy, the following three dynamic changes occur: Phase 1 to Phase 3. Before explaining Phases 1 to 3, the structure of the stomach and esophagus will be described with reference to Figures 8 and 9.

[0062] The Intramural Anti-Reflux Barrie Complex (IM-ARB complex), which is part of the anti-reflux mechanism at the gastroesophageal junction, is composed of three main components: Collar Sling Muscle fibers (Figure 9), Clasp Muscle fibers (Figure 9), and the lower esophageal sphincter (Figures 8 and 9).

[0063] Collar sling muscle fibers are obliquely running muscles located along the greater curvature of the stomach (Figure 8), forming a sling-like structure that surrounds the upper part of the stomach (Figure 9). These collar sling muscle fibers function to prevent stomach contents from refluxing into the esophagus by constricting the gastric cardia. From inside the stomach, the collar sling muscle fibers can be identified as a gastroesophageal flap valve (Figure 8). This gastroesophageal flap valve (GEFV) is a projection within the gastroesophageal junction, created by the sharp angle between the esophagus and the gastric cardia, and is a type of mucosal flap valve (MFV). An MFV located at the gastroesophageal junction is called a gastroesophageal flap valve. An MFV is a part where the mucosa is formed in a flap shape. The morphology of the gastroesophageal flap valve changes with the contraction and relaxation of muscles such as collar sling muscle fibers and clasp muscle fibers.

[0064] Clasp muscle fibers are located on the lesser curvature side of the stomach (Figure 8) and consist of a circular muscular layer. These clasp muscle fibers function to prevent stomach contents from refluxing into the esophagus by constricting the gastric cardia.

[0065] The lower esophageal sphincter (LES) is a ring of muscle located at the junction of the esophagus and stomach. It normally contracts to close the esophagus and prevent the reflux of stomach contents.

[0066] Phase 1 begins with zero air insufflation into the stomach. In Phase 1, as shown in Figures 10(a) and 10(b), the gastroesophageal flap valve and the longitudinal folds of the lesser curvature are observed. These longitudinal folds of the lesser curvature are mucosal folds that extend longitudinally along the inner wall of the stomach. Their morphology changes with the contraction and relaxation of muscles such as clasp muscle fibers, and they gradually stretch and flatten as the intragastric pressure increases. In Phase 1, the esophageal mucosa is not observed. Then, in Phase 1, as the air insufflation into the stomach increases, as shown in Figure 10(b), the longitudinal folds of the lesser curvature are stretched, the gastroesophageal flap valve gradually flattens, and its legs spread, but the esophageal mucosa is not observed.

[0067] In other words, Phase 1 makes it possible to evaluate the valve function (reflux prevention mechanism) of the gastric structure formed by the gastroesophageal flap valve and the longitudinal folds of the lesser curvature.

[0068] Phase 2 is the phase that occurs after Phase 1. In Phase 2, as shown in Figure 10(c), the esophageal mucosa is observed beyond the squamous columnar junction (Figure 8, Squamocolumnar Junction (SCJ)). This SCJ is the point where the squamous epithelium of the esophagus and the columnar epithelium of the stomach intersect, located at the gastroesophageal junction (GEJ), and marks the boundary between different epithelial cells of the digestive tract. The GEJ is located near the boundary between the esophagus and the stomach and is composed of various anatomical components that form a barrier to prevent reflux of gastric contents. Collar sling muscle fibers, clasp muscle fibers, lower esophageal sphincter, gastroesophageal flap valve, and the SCJ are all included in the GEJ. If the subject is healthy, the Scope Holding Sign (SHS) is observed. This SHS refers to the phenomenon in which the insertion portion 21 is held in place by the contraction of the lower esophageal sphincter when the intragastric pressure increases. The state shown in Figure 10(c) is SHS.

[0069] In other words, Phase 2 makes it possible to evaluate the valve function (reflux prevention mechanism) of the lower esophageal sphincter.

[0070] Phase 3 is the phase that occurs after Phase 2. In Phase 3, the intragastric pressure exceeds the contractile force of the lower esophageal sphincter, causing relaxation of the lower esophageal sphincter. In Figure 10(d), the arrows indicate that gas leaks into the esophagus (causing a babbling sound) due to the relaxation of the lower esophageal sphincter. If the subject is healthy, peristaltic waves will then descend from the upper esophagus, and SHS will be observed again.

[0071] In other words, Phase 3 makes it possible to evaluate the acid clearance function mediated by esophageal peristalsis.

[0072] After step S3, the user, such as a physician, determines whether the situation corresponds to Phase 1 described above based on the diagnostic support information displayed on the display device 5, and evaluates the valve function (condition of the gastric cardia) due to the gastric structure (step S4).

[0073] After step S4, the user, such as a physician, determines whether the state corresponds to Phase 2 described above based on the diagnostic support information displayed on the display device 5, and evaluates the valve function of the lower esophageal sphincter (state of the lower esophageal sphincter) (step S5).

[0074] After step S5, the user, such as a physician, determines whether the patient is in a state corresponding to Phase 3 described above, based on the diagnostic support information displayed on the display device 5, and evaluates the acid clearance function by esophageal peristalsis (the state of peristaltic movement in the esophagus) (step S6).

[0075] Further details regarding the diagnostic support information will be explained in the section "Examples of Displaying Diagnostic Support Information" below.

[0076] After step S6, the user, such as a physician, diagnoses gastroesophageal reflux disease based on the evaluation results from steps S4 to S6 (step S7).

[0077] [Examples of Display of Diagnostic Support Information] Next, we will explain the diagnostic support information. Figures 11 to 13 are diagrams illustrating examples of the display of diagnostic support information I0. Specifically, Figure 11 is a diagram showing an example of diagnostic support information I0 displayed on the display device 5 when the person being examined is healthy. Figure 12 is a diagram showing an example of diagnostic support information I0 displayed on the display device 5 when the person being examined is suspected of having gastroesophageal reflux disease. Figures 13(a) to 13(c) are diagrams showing examples of the first to third support information I2 to I4 that constitute the diagnostic support information I0 when the person being examined is healthy. Figures 13(d) to 13(f) are diagrams showing examples of the first to third support information I2 to I4 that constitute the diagnostic support information I0 when the person being examined is suspected of having gastroesophageal reflux disease.

[0078] As shown in Figures 11 to 13, the processing unit 4 generates an endoscopic image I1 by applying predetermined image processing to the image received from the endoscope 2 when air is insufflated into the stomach by the air insufflation device 6. The processing unit 4 also generates first support information I2 based on the gastric pressure information detected by the gastric pressure measuring device 7 during the air insufflation. Furthermore, the processing unit 4 generates second support information I3 based on the esophageal temperature information detected by the temperature sensor 9 during the air insufflation. Furthermore, the processing unit 4 generates third support information I4 based on the aspiration sound information collected by the microphone 8 during the air insufflation. Finally, the processing unit 4 generates diagnostic support information I0 for diagnosing gastroesophageal reflux disease based on the endoscopic image I1, the first to third support information I2 to I4, and time information I5 related to the air insufflation time into the stomach by the air insufflation device 6 (hereinafter referred to as pneumoperitoneum time).

[0079] Here, the processing unit 4 processes the captured image, gastric pressure information, esophageal temperature information, and aerated sound information in a time-synchronized manner based on the synchronization signal generated by the synchronization signal generation unit 42. The processing unit 4 also processes the esophageal temperature information detected by the multiple temperature sensors 9 in a time-synchronized manner based on the synchronization signal.

[0080] The first support information I2 includes a pressure waveform I21 showing the change in pressure (intragastric pressure) based on the increase in intragastric pressure due to air delivery into the stomach by the air delivery device 6, as shown in Figures 11 to 13. This pressure is measured by a pressure sensor 10 or an intragastric pressure measuring device 7 (pressure measuring probe 71). The first support information I2 also includes the current intragastric pressure (Current IGP) I22, the maximum intragastric pressure (Maximum IGP) I23 when the intragastric pressure increases, and the basal intragastric pressure (Basal IGP) I24 before the increase in intragastric pressure, as shown in Figures 11 and 12. The first support information I2 is not limited to the pressure waveform I21, current intragastric pressure I22, maximum intragastric pressure I23, and basal intragastric pressure I24, but may also include the pressure difference between the maximum intragastric pressure I23 and the basal intragastric pressure I24, or the pressure gradient obtained by dividing the pressure difference by the pneumoperitoneum time.

[0081] Furthermore, the current intragastric pressure I22, the maximum intragastric pressure I23 when intragastric pressure increases, and the basal intragastric pressure I24 before the increase in intragastric pressure may be represented using any appropriate representation, such as bar graphs, line graphs, contour graphs, other representations, or any appropriate combination thereof.

[0082] The second support information I3 includes a temperature waveform I31 showing the temperature changes detected by multiple temperature sensors 9, as shown in Figures 11 to 13. In this embodiment, the temperature waveform I31 is composed of a heat map or graph, as shown in Figures 13(c) and 13(f). Specifically, the heat map (temperature waveform I31) is a diagram in which time is the horizontal axis, and the temperature changes detected by the temperature sensors 9, which are positioned from the upper esophageal sphincter side to the lower esophageal sphincter side, are represented as a color pattern from the top to the bottom of the vertical axis. For example, the temperature changes shown in Figure 5 are displayed in color along the vertical axis, and the temperature measured by each temperature sensor 9 at each time point is displayed along the horizontal axis. That is, the time change of the temperature measured by the temperature sensors is displayed in color along the horizontal axis.

[0083] Here, the color patterns shown in Figure 13(c) and Figure 13(f) can, for example, use blue to represent the low temperature range and red to represent the high temperature range. The temporal representation shown on the horizontal axis can move horizontally along the time dimension to show the passage of time within the time interval in which the displayed temperature was measured. For example, the far right of the temporal representation can correspond to the most recent time within the time interval displayed in the temporal representation, and the far left can represent the earliest time. To show the passage of time within the time interval, the temporal representation moves continuously to the left on the screen, allowing the user to observe the representation of temperature measured in the esophagus over time. This continuous movement to the left allows the user to confirm the temporal change in temperature at the displayed position (if any) and to confirm the occurrence of events that caused the temperature change (e.g., relaxation of the lower esophageal sphincter or peristalsis in the esophagus).

[0084] Furthermore, information indicating the location of multiple temperature sensors 9 within the esophageal tube may be added to the heatmap (temperature waveform I31). For example, markers indicating the location of the upper and lower esophageal sphincters may be added to the vertical axis.

[0085] Furthermore, even if the temperature data detected by the number of temperature sensors 9 may be detected at discrete locations, the temperature data can be made quasi-continuous in spatial dimensions. Temperature data can be made quasi-continuous by including interpolated temperatures in the temperature data. Based on quasi-continuous temperature data, a quasi-continuous visual representation (e.g., one with smooth changes) can be provided. Any suitable visual representation described below may be quasi-continuous.

[0086] Furthermore, the second support information I3 includes the basal LES temperature I32 detected by the temperature sensor 9 located on the lower esophageal sphincter side before the rise in gastric pressure, as shown in Figures 11 and 12. Note that the second support information I3 is not limited to the temperature waveform I31 and the basal temperature I32; it may also include the maximum and minimum temperatures detected by multiple temperature sensors 9.

[0087] Furthermore, when the image processing unit 41 creates a heat map (temperature waveform I31), it may apply baseline correction to the temperature waveform to represent the change from a reference temperature. The reference temperature used as the baseline in this case can be a preset temperature or the temperature inside the esophagus or stomach before gas is delivered into the stomach. For example, for the temperature distribution shown in Figure 5, the reference temperature T BL Alternatively, the difference between the two values ​​can be calculated, and this difference can be used to create a heatmap (temperature waveform I31).

[0088] The third support information I4 includes a waveform I41 of the muffled sound based on the muffled sound information detected by the microphone 8, as shown in Figures 11 to 13. The third support information I4 also includes the maximum sound pressure of the muffled sound detected by the microphone 8, as shown in Figures 11 and 12.

[0089] Then, the user, such as a doctor, checks the diagnostic support information I0 displayed on the display device 5 and diagnoses the patient with gastroesophageal reflux disease.

[0090] In step S4, the user, such as a physician, checks the diagnostic support information I0 displayed on the display device 5, determines whether the state corresponds to Phase 1, and evaluates the valve function due to the structure of the stomach. For example, if the person being examined is healthy, the pressure waveform I21 will be a sloped pressure waveform as shown in Figure 13(a). On the other hand, if the person being examined is suspected of having an abnormality, the pressure waveform I21 will be a flat pressure waveform as shown in Figure 13(d). Therefore, the user, such as a physician, checks, for example, the endoscopic image I1 and the first support information I2 that constitute the diagnostic support information I0, and evaluates the valve function due to the structure of the stomach.

[0091] In step S5, the user, such as a physician, checks the diagnostic support information I0 displayed on the display device 5, determines whether the state corresponds to Phase 2, and evaluates the valve function of the lower esophageal sphincter. For example, if the person being examined is healthy, SHS will be observed, and the contraction pressure of the lower esophageal sphincter will also be observed. Therefore, the user, such as a physician, checks, for example, the endoscopic image I1, and the first and second support information I2 and I3 that constitute the diagnostic support information I0, and evaluates the valve function of the lower esophageal sphincter.

[0092] In step S6, the user, such as a physician, checks the diagnostic support information I0 displayed on the display device 5 and determines whether the state corresponds to Phase 3, and evaluates the acid clearance function by esophageal peristalsis. For example, if the subject is healthy, a babbling sound occurs and the lower esophageal sphincter relaxes when the gastric pressure is around 19 mmHg (Figures 13(a) to 13(c)). On the other hand, if the subject is suspected of having an abnormality, the lower esophageal sphincter relaxes when the gastric pressure is around 14 mmHg. Also, if the subject is healthy, peristaltic movement in the esophagus is observed (Figure 13(c)), and SHS is observed again. For example, on the heat map (temperature waveform I31), as time progresses (to the right on the time axis, but the representation itself moves to the left), it can be seen that the contractile pressure in each part of the esophagus progresses from the top to the bottom (from the upper esophageal sphincter to the lower esophageal sphincter).

[0093] On the other hand, if an abnormality is suspected in the subject being examined, peristaltic movement in the esophagus will not be observed (Figure 13(f)). Therefore, users such as physicians can, for example, check the endoscopic image I1 and the first to third support information I2 to I4 that constitute the diagnostic support information I0 to evaluate the acid clearance function due to esophageal peristalsis.

[0094] Then, in step S7, if the user, such as a physician, suspects an abnormality in any of the valve function due to the gastric structure, the valve function due to the lower esophageal sphincter, or the acid clearance function due to esophageal peristalsis, which were evaluated in steps S4 to S6, the user diagnoses suspected gastroesophageal reflux disease.

[0095] Furthermore, the diagnostic support information I0 shown in Figures 11 and 12 includes diagnostic result information I6. The diagnostic result information I6 is information regarding the diagnostic result that the processing device 4 automatically diagnoses whether or not the person being examined has gastroesophageal reflux disease (or whether or not there is a suspicion of gastroesophageal reflux disease) based on the captured images, gastric pressure information, esophageal temperature information, and aerated sound information. This diagnostic result information I6 includes an evaluation result I61 that evaluates which phase the current patient is in, an evaluation result I62 that evaluates whether or not the patient is in a state corresponding to each of Phases 1 to 3, and a diagnostic result I63 that evaluates whether or not the patient has gastroesophageal reflux disease (or whether or not there is a suspicion of gastroesophageal reflux disease) based on the evaluation result I62.

[0096] The processing unit 4 stores the captured image, gastric pressure information, esophageal temperature information, and aerated sound information in memory, which are processed in a time-synchronized manner based on the synchronization signal. In other words, the captured image and various support information can be referenced in a time-synchronized state. Therefore, for example, a user such as a doctor can view the endoscopic image at a time specified by the user, along with the various support information for that timing.

[0097] The embodiment described above provides the following advantages. According to this embodiment, it is possible to evaluate the valve function due to the gastric structure, the valve function due to the lower esophageal sphincter, and the acid clearance function due to esophageal peristalsis using only the endoscope system 1, without using other examination equipment, and to diagnose whether or not the person being examined has gastroesophageal reflux disease. Therefore, according to this embodiment, the burden on the person being examined when being tested for gastroesophageal reflux disease can be reduced.

[0098] Furthermore, in this embodiment, multiple temperature sensors 9 are arranged in the insertion section 21, and the temperature measurement results are used as data for evaluating the peristaltic movement of the esophagus. In this way, by using temperature as the evaluation of peristaltic movement and by using temperature sensors 9 arranged in the insertion section 21, it is possible to suppress the increase in diameter of the insertion section 21. Note that, for example, a pressure sensor requires a configuration to detect membrane vibration, so if a pressure sensor is used instead of a temperature sensor 9, the diameter will be slightly larger compared to a temperature sensor.

[0099] In the embodiment described above, an example was given in which the user evaluates the acid clearance function due to esophageal peristalsis (the state of peristaltic movement in the esophagus) based on esophageal temperature information. However, the image processing unit 41 may also be configured to estimate esophageal peristalsis based on information showing the temporal changes in temperature at multiple locations in the esophagus, as described above. In this case, the image processing unit 41 generates an evaluation result that evaluates the acid clearance function due to esophageal peristalsis (the state of peristaltic movement in the esophagus) based on the maximum, minimum, average, and mode values ​​of the temperature.

[0100] Furthermore, the image processing unit 41 may generate evaluation results that evaluate the acid clearance function due to esophageal peristalsis (the state of peristaltic movement in the esophagus) using a trained model. This trained model may be generated by an external learning device different from the processing unit 4, or it may be generated in the processing unit 4. If the trained model is generated by a learning device, the processing unit 4 obtains the trained model by referring to that learning device or an external server where the trained model is stored, or generates evaluation results based on estimated information obtained by inputting the target information into the trained model. These evaluation results include, for example, information that associates the change in esophageal temperature of the subject over time with the estimated information. The learning device is configured using a general-purpose processor such as a CPU, a dedicated processor such as various arithmetic circuits that perform specific functions such as ASICs, and a memory that stores various programs executed by the learning device.

[0101] A trained model is generated by training it using training data consisting of pairs of esophageal temperature information and evaluation results associated with that esophageal temperature information. The training data is, for example, data in which a heatmap (temperature waveform I31) as shown in Figure 13 is associated with an evaluation result (e.g., normal / abnormal) for this heatmap (temperature waveform I31), and evaluation results are associated with multiple patterns of heatmaps.

[0102] The trained model is a neural network consisting of an input layer, hidden layers, and output layers, generated by supervised learning with, for example, a heatmap (temperature waveform I31) as the explanatory variable and the evaluation result as the objective variable. In addition to neural networks, models generated by known machine learning or deep learning methods can also be used. This trained model simply outputs whether the input heatmap corresponds to normal or abnormal peristaltic motion.

[0103] Alternatively, the system may output information estimating regions where peristaltic motion is abnormal, using a trained model generated by supervised learning, where the partial temperature distribution in the heatmap (temperature waveform I31) is the explanatory variable and the evaluation result of the normal / abnormality of that partial temperature distribution is the dependent variable. Here, the region may be specified as a region on a time-temperature graph, or as a region on a three-dimensional model.

[0104] Furthermore, the system may output information estimating normal / abnormal symptoms using a pre-trained model generated by supervised learning, where the partial temperature distribution in the heatmap (temperature waveform I31) is the explanatory variable and the evaluation result of the normal / abnormal symptoms of that partial temperature distribution is the dependent variable.

[0105] The evaluation results of the estimation by the trained model are used as supporting information for the user's evaluation. For example, the user evaluates the acid clearance function by esophageal peristalsis (the state of peristaltic movement in the esophagus) by referring to the second supporting information I3 described above and the evaluation results generated by the trained model.

[0106] (Other Embodiments) While embodiments for carrying out the present invention have been described so far, the present invention should not be limited to the embodiments described above. In the embodiments described above, in order to diagnose gastroesophageal reflux disease, at least two pieces of information, namely endoscopic images and esophageal temperature information, are sufficient, and it is not necessary to use all of the information, including the endoscopic images, esophageal temperature information, gastric pressure information, and aerate sound information.

[0107] While the above-described embodiment was explained as a technique for diagnosing gastroesophageal reflux disease, it is not limited to that. For example, it can also evaluate relaxation failure of the lower esophageal sphincter, and thus can be useful in diagnosing esophageal achalasia. Furthermore, it can evaluate the function of major sphincters present in the human digestive tract. Specifically, it can evaluate the external anal sphincter, that is, physiological anorectal function associated with aging.

[0108] 1 Endoscope system 2 Endoscope 3 Light source device 4 Processing unit 5 Display device 6 Air supply device 7 Intragastric pressure measuring device 8 Microphone 9 Temperature sensor 10 Pressure sensor 21 Insertion section 22 Operation section 23 Universal cord 24 Tip section 25 Bending section 26 Flexible tube section 31 Light source section 32 Illumination control section 33 Light source driver 41 Image processing section 42 Synchronization signal generation section 43 Input section 44 Control section 45 Memory section 61 Air supply section 62 Flow rate measurement section 63 Control section 71 Pressure measuring probe 221 Bending knob 222 Instrument insertion section 223 Air supply line 224 Switch 231, 232 Connector 241 Light guide 242 Illumination lens 243 Optical system 244 Image sensor 245 Cable Assembly I0 Diagnostic support information I1 Endoscopic image I2 First support information I21 Pressure waveform I22 Current intragastric pressure I23 Maximum intragastric pressure I24 Basal intragastric pressure I3 Second support information I31 Temperature waveform I32 Basal temperature I4 Third support information I41 Waveform I5 Time information I6 Diagnostic result information I61, I62 Evaluation results I63 Diagnostic result TU Air supply tube

Claims

1. An endoscope system comprising: an endoscope having an insertion section with an image sensor positioned at its tip; a temperature sensor for detecting the temperature applied to the outer surface of the insertion section; an air supply device for supplying gas through an air supply line provided in the insertion section; and a processor, wherein the processor generates diagnostic support information to assist in the diagnosis of gastroesophageal reflux disease based on the image captured by the image sensor and esophageal temperature information related to the temperature detected by the temperature sensor.

2. The endoscope system according to claim 1, wherein the temperature sensor is positioned to detect at least the contraction pressure and relaxation pressure of the lower esophageal sphincter when the insertion portion is inserted into the stomach and the gastric cardia is made image-capable by the image sensor as the endoscope is bent.

3. The endoscope system according to claim 1, wherein the temperature sensor is positioned to detect at least the contraction pressure and relaxation pressure of the upper esophageal sphincter when the insertion portion is inserted into the stomach and the gastric cardia is made image-capable by the image sensor as the endoscope is bent.

4. The endoscope system according to claim 1, wherein the temperature sensor is positioned to detect contraction and relaxation pressure at various locations within the esophagus from the upper esophageal sphincter to the lower esophageal sphincter, when the insertion portion is inserted into the stomach and the gastric cardia is made image-capable by the image sensor due to the bending operation of the endoscope.

5. The endoscope system according to claim 1, wherein 36 temperature sensors are arranged on the outer circumferential surface of the insertion portion at 1 cm intervals along the axial direction of the insertion portion.

6. The endoscope system according to claim 1, wherein the temperature sensor is a sheet-type temperature sensor provided over the entire circumference in the rotational direction with respect to the central axis along the axial direction in the insertion portion.

7. The endoscope system according to claim 6, wherein the temperature sensor is a flexible sensor having flexibility.

8. The endoscope system according to claim 1, wherein a plurality of temperature sensors are provided along the axial direction of the insertion portion, and the processor performs baseline correction on the temperature detected by each temperature sensor.

9. The endoscopic system according to claim 8, wherein the processor performs the baseline correction using the temperature before gas is delivered into the stomach as the reference temperature.

10. The endoscopic system according to claim 1, wherein the processor generates support information, which is the diagnostic support information, based on the esophageal temperature information.

11. The endoscopic system according to claim 10, wherein the processor processes the esophageal temperature information detected by each of the multiple temperature sensors in a time-synchronized manner.

12. The endoscope system according to claim 10, wherein the support information includes temperature waveforms detected by each of the multiple temperature sensors.

13. The endoscope system according to claim 12, wherein the support information includes a heat map showing the change in temperature over time.

14. The endoscopic system according to claim 1, wherein the processor estimates the peristaltic movement of the esophagus based on the esophageal temperature information in the esophagus.

15. The endoscopic system according to claim 14, wherein the processor generates an evaluation result for evaluating the acid clearance function by esophageal peristalsis based on estimated information obtained by inputting esophageal temperature information to be evaluated into a trained model that has learned the time change of temperature in the esophagus as an explanatory variable and the evaluation result for the time change of temperature as an objective variable.

16. The endoscopic system according to claim 15, wherein the evaluation result includes normal or abnormal peristaltic movement.

17. The endoscope system according to claim 15, wherein the evaluation results include information that estimates areas where peristaltic movement is abnormal.

18. The endoscope system according to claim 15, wherein the evaluation results include information that estimates whether the esophagus is normal or abnormal.

19. An image processing method in which a processor generates diagnostic support information to assist in the diagnosis of gastroesophageal reflux disease, based on an image captured by an image sensor provided at the tip of the insertion section of an endoscope and esophageal temperature information related to the temperature detected by a temperature sensor that detects the temperature applied to the outer surface of the insertion section.

20. An image processing program that causes a processor to execute a process to generate diagnostic support information to assist in the diagnosis of gastroesophageal reflux disease, based on an image captured by an image sensor provided at the tip of the insertion section of an endoscope and esophageal temperature information related to the temperature detected by a temperature sensor that detects the temperature applied to the outer surface of the insertion section.

Images