Endoscope system, image processing method, and image processing program

Abstract

An endoscope system according to the present invention comprises: an endoscope having an insertion part with an image-capture element disposed at the distal end thereof; a contact pressure sensor provided to the insertion part to detect contact pressure with a body cavity of a subject; and a processor. On the basis of a captured image captured by the image-capture element and elasticity characteristic information detected by the contact pressure sensor, the processor generates a mapping image in which an acquisition position of the elasticity characteristic information estimated on a three-dimensional model representing the body cavity is associated with the three-dimensional model.

Description

Endoscope system, image processing method, and image processing program

[0001] The present invention relates to an endoscope system, an image processing method, and an image processing program.

[0002] Gastroesophageal reflux disease (GERD) is a pathological condition caused by the reflux of stomach contents, including a large amount of stomach acid, into the esophagus. Conventionally, an endoscopic system using an endoscope has been proposed as a system for diagnosing gastroesophageal reflux disease (see, for example, Patent Document 1). In the endoscopic system described in Patent Document 1, an insertion section of an endoscope is inserted into the stomach, the endoscope is set to be able to image the gastric cardia, and air is blown into the stomach. The endoscopic system then evaluates the function of the lower esophageal sphincter based on pressure changes within the stomach detected during gas blowing into the stomach and the state of the gastric cardia observed from the image captured using the endoscope.

[0003] International Publication No. 2021 / 166127

[0004] However, in diagnosing gastroesophageal reflux disease, it is necessary to evaluate not only the function of the lower esophageal sphincter but also the motility function of the esophagus. Therefore, in addition to the examination using the endoscopic system described in Patent Document 1, it is necessary to additionally evaluate the motility function of the esophagus using, for example, a high-resolution manometry (HRM) examination. This high-resolution manometry examination cannot be performed with the endoscopic system described in Patent Document 1. Therefore, in addition to the examination using the endoscopic system described in Patent Document 1, the high-resolution manometry examination must be performed using a separate examination device, which places a burden on the examinee. Therefore, there is a need for a technology that can diagnose gastroesophageal reflux disease with high accuracy while reducing the burden on the examinee.

[0005] The present invention has been made in consideration of the above, and aims to provide an endoscopic system, an image processing method, and an image processing program that can diagnose gastroesophageal reflux disease with high accuracy while reducing the burden on the subject.

[0006] In order to solve the above-mentioned problems and achieve the object, the endoscopic system of the present invention comprises an endoscope having an insertion section with an imaging element disposed at the tip, a contact pressure sensor provided in the insertion section for detecting contact pressure with the body cavity of the subject, and a processor, and the processor generates a mapping image in which the acquisition position of the elastic property information estimated on a three-dimensional model representing the body cavity is associated with the three-dimensional model based on an image captured by the imaging element and elastic property information detected by the contact pressure sensor.

[0007] In the image processing method of the present invention, a computer generates a mapping image in which the acquisition position of the elastic property information estimated on a three-dimensional model representing the body cavity is associated with the three-dimensional model, based on an image captured by an imaging element provided at the tip of an insertion portion of an endoscope and elastic property information detected by a contact pressure sensor provided in the insertion portion that detects the contact pressure with the body cavity of the subject.

[0008] The image processing program of the present invention causes a computer to execute a process of generating a mapping image in which the acquisition position of the elastic property information estimated on a three-dimensional model representing the body cavity is associated with the three-dimensional model, based on an image captured by an imaging element provided at the tip of the insertion portion of an endoscope and elastic property information detected by a contact pressure sensor provided in the insertion portion that detects the contact pressure with the body cavity of the subject.

[0009] According to the endoscope system, image processing method, and image processing program of the present invention, it is possible to diagnose gastroesophageal reflux disease with high accuracy while reducing the burden on the subject.

[0010] FIG. 1 is a diagram illustrating a configuration of an endoscopic system according to an embodiment. FIG. 2 is a diagram illustrating a configuration of an endoscopic system according to an embodiment. FIG. 3 is a diagram illustrating the configuration and arrangement of a pressure sensor. FIG. 4 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. FIG. 5 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. FIG. 6 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. FIG. 7 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. FIG. 8 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. FIG. 9 is a diagram illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. FIG. 10 is a diagram illustrating a display example of diagnostic support information. FIG. 11 is a diagram illustrating a display example of diagnostic support information. FIG. 12 is a diagram illustrating a display example of diagnostic support information. FIG. 13 is a flowchart illustrating a process for generating a trained model that generates a three-dimensional model of a lumen. FIG. 14 is a diagram illustrating an example of an endoscopic image acquired in a human body model for endoscopic training. FIG. 15 is a diagram illustrating an example of an organ image obtained by converting a subject shown in the endoscopic image shown in FIG. 14 into an organ.

[0011] Hereinafter, a mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the description of the drawings, the same parts are given the same reference numerals.

[0012] 1 and 2 are diagrams showing the configuration of an endoscopic system 1 according to an embodiment. The endoscopic system 1 is used in the medical field and is a system for diagnosing gastroesophageal reflux disease in a subject using an endoscope 2. As shown in FIGS. 1 and 2 , the endoscopic system 1 includes an endoscope 2, a light source device 3, a processing device 4, a display device 5, an air supply device 6, a gastric pressure measuring device 7, and a microphone 8.

[0013] In this embodiment, the endoscope 2 is a so-called flexible endoscope. A portion of the endoscope 2 is inserted into a living body, captures images of the living body, and outputs image signals generated by the image capture. As shown in FIG. 1 , the endoscope 2 includes an insertion section 21, an operation section 22, and a universal cord 23.

[0014] The insertion section 21 is a section that has at least a portion that is flexible and is inserted into a living body. As shown in Figures 1 and 2, the insertion section 21 includes a tip section 24, a freely bendable bending section 25 (Figure 1) that is composed of a plurality of bending pieces, and a long, flexible flexible tube section 26 (Figure 1) that is connected to the base end side of the bending section 25. An imaging element 244 (Figure 2) is built into the tip section 24. The insertion section 21 is inserted into a body cavity of a subject, and captures an image of a subject, such as biological tissue, that is located in a position that is not accessible by external light, using the imaging element 244.

[0015] Here, an outer peripheral surface pressure sensor 9 that detects pressure applied to the outer peripheral surface is provided on the outer peripheral surface of the insertion portion 21. The detailed configuration and arrangement of the outer peripheral surface pressure sensor 9 will be described later in the section "Configuration and Arrangement of Pressure Sensor."

[0016] The operation unit 22 is connected to the base end portion of the insertion section 21. The operation unit 22 receives various operations for the endoscope 2. As shown in Fig. 1 , the operation unit 22 includes a bending knob 221 for bending the bending section 25 in the up-down and left-right directions, a treatment tool insertion section 222 that extends from the operation unit 22 to the tip of the insertion section 21 and inserts treatment tools such as biopsy forceps, an electric scalpel, and an examination probe into the body cavity of the subject, an air supply conduit 223 (Fig. 2) that extends from the operation unit 22 to the tip of the insertion section 21 and supplies air into the body cavity of the subject, and a plurality of switches 224 for operating peripheral devices such as the air supply device 6 and a water supply device (not shown).

[0017] The universal cord 23 incorporates at least a light guide 241 ( FIG. 2 ) and a cable assembly 245 ( FIG. 2 ) that bundles one or more signal lines. The light guide 241 is made of glass fiber or the like and serves as a light guide path for light emitted by the light source device 3. As shown in FIG. 1 , the universal cord 23 branches at the end opposite to the end connected to the operation unit 22. The branched ends of the universal cord 23 are provided with a connector 231 that is detachable from the light source device 3 and a connector 232 that is detachable from the processing device 4. A portion of the light guide 241 extends from the end of the connector 231. The universal cord 23 transmits illumination light emitted from the light source device 3 to the distal end 24 via the connector 231 (light guide 241), the operation unit 22, and the flexible tube portion 26. The universal cord 23 also transmits image signals captured by an image sensor 244 provided in the distal end 24 to the processing device 4 via the connector 232. The cable assembly 245 includes a signal line for transmitting an image signal, a signal line for transmitting a drive signal for driving the image sensor 244, and a signal line for transmitting and receiving information including unique information related to the endoscope 2 (image sensor 244). Note that, in this embodiment, the signal lines are described as transmitting electrical signals, but they may also be used to transmit optical signals, or may be used to transmit signals between the endoscope 2 and the processing device 4 by wireless communication.

[0018] The output end side of the light guide 241 is inserted into the tip portion 24. As shown in Fig. 2, the tip portion 24 includes an illumination lens 242, an optical system 243 for collecting light, and an image sensor 244 that is provided at the imaging position of the optical system 243 and receives the light collected by the optical system 243, photoelectrically converts the light into an electrical signal, and performs predetermined signal processing.

[0019] The optical system 243 is configured using one or more lenses, and forms an observation image on the light receiving surface of the image sensor 244. The optical system 243 may have an optical zoom function that changes the angle of view and a focus function that changes the focus.

[0020] The image sensor 244 photoelectrically converts light from the optical system 243 to generate an electrical signal (image signal). The image sensor 244 is configured with a plurality of pixels arranged in a matrix, each of which has a photodiode that accumulates an electric charge according to the amount of light and a capacitor that converts the electric charge transferred from the photodiode into a voltage level. The image sensor 244 photoelectrically converts light incident on each pixel via the optical system 243 to generate an electric signal, sequentially reads out the electric signals generated by pixels arbitrarily designated as readout targets among the plurality of pixels, and outputs the electric signals as an image signal. The image sensor 244 is realized, for example, using a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0021] For ease of explanation, the image signal generated by the image sensor 244 capturing an image will be referred to as a captured image below.

[0022] A contact pressure sensor 246 is provided at the tip of the distal end portion 24 on the side opposite to the bending portion 25. The contact pressure sensor 246 is provided, for example, on the distal end surface of the distal end portion 24, protruding from the distal end surface where the illumination lens 242 and the optical system 243 are exposed. The contact pressure sensor 246 may be a hardness sensor or the like that can acquire elasticity property information (Young's modulus (elastic coefficient)) of the contact point. In the following description, the contact pressure sensor 246 will be described as outputting a signal related to the elasticity of the subject's lumen (hereinafter referred to as elasticity property information).

[0023] Here, the endoscope 2 has a memory (not shown) that stores execution programs and control programs for the image sensor 244 to execute various operations, as well as data including identification information of the endoscope 2. The identification information includes the endoscope 2's unique information (ID), model year, specification information, transmission method, etc. The memory may also temporarily store captured images generated by the image sensor 244.

[0024] As shown in FIG. 2 , the light source device 3 includes a light source unit 31 , an illumination control unit 32 , and a light source driver 33 .

[0025] The light source unit 31 emits light under the control of the illumination control unit 32. The light source unit 31 emits light having a wavelength band of visible light (white light (illumination light)). The light source unit 31 is realized using any light source such as an LED (Light Emitting Diode) light source, a laser light source, a xenon lamp, or a halogen lamp. The light source unit 31 may also include one or more lenses. The light generated by the light source unit 31 passes through the light guide 241 and the illumination lens 242 and is emitted from the tip of the tip unit 24 toward the subject.

[0026] The light emitted from the light source unit 31 is not limited to white light, but may be narrowband light having light in a specific wavelength band, or excitation light that excites substances contained in the object of observation.

[0027] The light source driver 33 supplies current to the light source unit 31 under the control of the illumination control unit 32, thereby causing the light source unit 31 to emit light.

[0028] The processing device 4 corresponds to a processor according to the present invention. As shown in FIG. 2 , the processing device 4 includes an image processing unit 41, a synchronization signal generating unit 42, an input unit 43, a control unit 44, and a storage unit 45.

[0029] Under the control of the control unit 44, the image processing unit 41 performs predetermined image processing on the captured image received from the endoscope 2 to generate an endoscopic image.

[0030] Examples of image processing performed by the image processing unit 41 include optical black subtraction processing (clamping processing), white balance adjustment processing, demosaic processing, color correction matrix processing, gamma correction processing, YC processing that converts RGB signals into luminance color difference signals (Y, Cb / Cr signals), digital gain adjustment that multiplies by digital gain, noise removal, and filter processing that emphasizes structure.

[0031] Furthermore, under the control of the control unit 44, the image processing unit 41 generates first support information based on a signal relating to pressure (gastric pressure information) detected by the gastric pressure measuring device 7.

[0032] Furthermore, under the control of the control unit 44, the image processing unit 41 generates second support information based on a signal relating to pressure detected by the outer peripheral surface pressure sensor 9 (esophageal pressure information).

[0033] Furthermore, under the control of the control unit 44, the image processing unit 41 generates third support information based on a signal relating to a belching sound (belching sound information) collected by the microphone 8.

[0034] The image processing unit 41 then generates diagnostic support information for diagnosing gastroesophageal reflux disease based on the endoscopic image and the first to third support information. The diagnostic support information is output to the display device 5 and displayed on the display device 5.

[0035] Details of the first to third support information and the diagnostic support information will be explained in the section "Display Examples of Diagnostic Support Information" below.

[0036] The image processing unit 41 also includes a mapping image generating unit 411 that generates a mapping image using the captured image received from the endoscope 2 and the elasticity property information detected by the contact pressure sensor 246 .

[0037] The image processing unit 41 described above is configured using a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor such as various arithmetic circuits that execute specific functions, such as an ASIC (Application Specific Integrated Circuit).

[0038] The synchronization signal generation unit 42 generates a clock signal (synchronization signal) that serves as a reference for the operation of the processing device 4, and outputs the generated synchronization signal to the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2. Here, the synchronization signal generated by the synchronization signal generation unit 42 includes a horizontal synchronization signal and a vertical synchronization signal. Therefore, the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2 operate in synchronization with each other using the generated synchronization signal.

[0039] The input unit 43 is realized using a keyboard, a mouse, a switch, and a touch panel, and accepts various operations for instructing the operation of the endoscope system 1. The input unit 43 may include a switch provided on the operation unit 22 or a portable terminal such as an external tablet computer.

[0040] The control unit 44 is configured using a general-purpose processor such as a CPU or a dedicated processor such as an ASIC or various arithmetic circuits that execute specific functions.

[0041] The storage unit 45 stores various programs executed by the control unit 44 and data including various parameters necessary for the processing of the control unit 44. The various programs can be recorded on computer-readable recording media such as a hard disk, flash memory, CD-ROM, DVD-ROM, or flexible disk and widely distributed. The various programs can also be obtained by downloading them via a communications network. The communications network referred to here is realized by, for example, an existing public line network, a LAN (Local Area Network), a WAN (Wide Area Network), or the like, and can be wired or wireless.

[0042] The storage unit 45 having the above configuration is realized using a ROM (Read Only Memory) in which various programs and the like are pre-installed, and a RAM, hard disk, and the like for storing calculation parameters and data for each process.

[0043] In this embodiment, the light source device 3 and the processing device 4 are provided in separate housings, but this is not limiting, and they may be provided integrally in the same housing.

[0044] The display device 5 displays the display image received from the processing device 4 (image processing unit 41) via the video cable. The display device 5 is configured using a monitor such as a liquid crystal or organic EL (Electro Luminescence) monitor.

[0045] The gas supply device 6 adjusts the pressure of gas supplied from a gas supply source (not shown, for example, a carbon dioxide gas cylinder) to a predetermined pressure and discharges the gas from the tip of the insertion section 21 into the space where the tip is located through the gas supply conduit 223. As shown in Fig. 1, the gas supply device 6 includes a gas supply unit 61, a flow rate measurement unit 62, and a control unit 63.

[0046] Although not specifically shown, the gas supply unit 61 includes a primary pressure reducer, a secondary pressure reducer, and a flow control valve. These primary pressure reducer, secondary pressure reducer, and flow control valve are connected in this order by an air supply conduit made of silicone, fluororesin, or the like. Gas supplied from a gas supply source (not shown) passes through the air supply conduit, in this order, through the primary pressure reducer, secondary pressure reducer, and flow control valve. After being adjusted to a predetermined pressure and flow rate, the gas is discharged from the air supply tube TU ( FIG. 1 ) via the flow rate measuring unit 62. The control unit 63 controls the flow control valve provided in the gas supply unit 61 to adjust the flow rate of gas supplied to the endoscope 2 to a predetermined value. The flow control valve is, for example, a type of electromagnetically driven valve, and is configured as an adjustment valve using an electromagnetic coil in the drive unit. The opening degree of the valve unit is controlled by controlling the position of the plunger depending on the magnitude of the current flowing through the electromagnetic coil, thereby adjusting the flow rate of gas flowing through the air supply conduit to a predetermined value.

[0047] The gastric pressure measuring device 7 corresponds to an internal pressure sensor according to the present invention. This gastric pressure measuring device 7 detects the pressure inside the space where the tip is located via a pressure measurement probe 71 inserted to the tip of the insertion section 21 through the treatment tool insertion section 222. A signal (hereinafter referred to as gastric pressure information) related to the pressure (hereinafter referred to as gastric pressure) detected by the gastric pressure measuring device 7 is output to the processing device 4.

[0048] The microphone 8 is placed on the throat of the subject and collects belching sounds (burping sounds) emitted from the esophagus of the subject. The signal related to the belching sounds collected by the microphone 8 (hereinafter referred to as belching sound information) is output to the processing device 4.

[0049] [Configuration and Arrangement of Pressure Sensor] Figure 3 is a diagram illustrating the configuration and arrangement of the outer peripheral surface pressure sensor 9. The outer peripheral surface pressure sensor 9 detects pressure using a known method, and is configured as, for example, a resistance-type pressure sensor or a capacitance-type pressure sensor. As shown in Figure 3, the outer peripheral surface pressure sensor 9 according to this embodiment is a circular pressure sensor provided around the entire circumference in the rotational direction around a central axis along the axial direction of the insertion portion 21. Note that the outer peripheral surface pressure sensor 9 is not limited to a circular pressure sensor, and a point-type pressure sensor provided only around a portion of the entire circumference in the rotational direction may also be used.

[0050] 3, a total of 36 outer peripheral surface pressure sensors 9 are arranged at 1 cm intervals along the axial direction of the insertion section 21 on the outer peripheral surface of the insertion section 21. As a result, when the insertion section 21 is inserted into the stomach, the outer peripheral surface pressure sensors 9 are arranged at positions that detect contraction pressure and relaxation pressure at various locations inside the esophagus from the upper esophageal sphincter to the lower esophageal sphincter.

[0051] The number of outer peripheral surface pressure sensors 9 is not limited to 36, but may be, for example, 6 or more, 12 or more, or even more.

[0052] Here, the outer periphery pressure sensor 9 may be positioned a predetermined distance from one or more of the nearest other outer periphery pressure sensors 9. Optionally, the spacing between each outer periphery pressure sensor 9 may be substantially the same. The spacing may be 3 centimeters or less, for example 2 centimeters or less, for example 1 centimeter, or even less than 1 centimeter.

[0053] The signal relating to the pressure detected by the outer peripheral surface pressure sensor 9 described above (hereinafter referred to as esophageal pressure information) is output to the processing device 4.

[0054] [Method for Diagnosing Gastroesophageal Reflux Disease] Next, a method for diagnosing gastroesophageal reflux disease will be described. FIGS. 4 to 9 are diagrams illustrating a method for diagnosing gastroesophageal reflux disease according to an embodiment. Specifically, FIG. 4 is a flowchart illustrating the method for diagnosing gastroesophageal reflux disease. FIG. 5 is a diagram illustrating captured images acquired during passage through the esophagus and mapping images generated based on elastic property information from the contact pressure sensor. FIG. 6 is a cross-sectional view showing the vicinity of the cardiac region of the stomach, illustrating steps S1 to S4. For ease of explanation, the outer peripheral surface pressure sensor 9 provided on the outer peripheral surface of the insertion section 21 is omitted from FIG. 6. FIGS. 7 and 8 are diagrams illustrating the structures of the stomach and esophagus. FIG. 7 is a cross-sectional view showing the structures of the stomach and esophagus. FIG. 8 is a view of the stomach and esophagus viewed from the outside. FIG. 9 is a diagram illustrating captured images in Phases 1 to 3, which correspond to dynamic changes that occur when air is supplied to the stomach (pneumoperitoneum).

[0055] First, a user such as a doctor inserts the insertion portion 21 into the subject through a natural opening such as the mouth or nose, and introduces the insertion portion 21 through the esophagus into the stomach as shown in Fig. 6 (step S1). In this embodiment, when the insertion portion 21 is inserted into the esophagus, the image sensor 244 starts capturing images, and captured images are sequentially generated as the insertion portion 21 moves through the esophagus. Elasticity property information is also acquired through measurements by the contact pressure detection sensor 246. Note that detection by the contact pressure detection sensor 246 is performed, for example, when the contact pressure detection sensor 246 comes into contact with the organ to be measured by a user operation such as a doctor. Note that the detection may be performed at a timing specified by the user via a button or the like, or at predetermined time intervals.

[0056] While the insertion section 21 passes through the esophagus, the image processing unit 41 generates a mapping image and displays it on the display device 5 (step S2). Specifically, the mapping image generation unit 411 estimates global coordinates using the captured images and elastic property information from the contact pressure sensor 246, and generates a mapping image in which the coordinates are associated with the detection results of the contact pressure sensor 246. The mapping image generation unit 411 first estimates depth using feature points or brightness values ​​in the multiple captured images, and generates a three-dimensional model that estimates the self-position based on the estimation results. This three-dimensional model can be generated using, for example, SLAM (Simultaneous Localization and Mapping). Other well-known techniques can also be used, such as generating a three-dimensional model based on a set of multiple images acquired using a stereo camera equipped with multiple image sensors with different optical axis angles. When a stereo camera is used, for example, the image sensor 244 is configured by a stereo camera, but a stereo camera may be provided separately from the image sensor 244.

[0057] The mapping image generator 411 then generates a mapping image by associating the detected position in the three-dimensional image with the elasticity information based on the detection time of the contact pressure sensor 246. This mapping image may be generated using, for example, an image acquired after passing through the esophagus and the elasticity property information detected by the contact pressure sensor 246, or the mapping image may be updated each time (in substantially real time) using newly acquired image and / or elasticity property information at the timing of acquiring the image and / or elasticity property information.

[0058] For example, as shown in FIG. 5 , a virtual esophagus image M1 corresponding to a three-dimensional model of the esophagus is generated, and detection positions S1 to S5 of the contact pressure sensor 246 in the esophagus are set. Elasticity property information detected at each detection position is associated with the corresponding position. In this case, a visually distinguishable display mode may be changed depending on the elasticity property information. For example, different colors may be used depending on the value indicated by the elasticity property information. Furthermore, if the elasticity property information (value) is an abnormal value outside a predetermined normal range, the abnormal area may be displayed in a different color, or a different color from normal areas, to identify the abnormal area. Alternatively, a region where abnormal values ​​are concentrated may be set and this region may be displayed as an abnormal area. The threshold value and normal range for determining normality / abnormality may be preset based on the elasticity (softness) of a normal lumen, or may be set based on previously acquired elasticity property information or statistical analysis.

[0059] Furthermore, a trained model generated by learning elastic property information acquired in the past may be used to determine whether an object at a detection position is normal or abnormal. The trained model in this case may be, for example, a machine learning model such as a neural network generated by learning using the past elastic property information as an explanatory variable and the normal / abnormal label associated with the elastic property information as a target variable.

[0060] In this case, an evaluation result may be generated that evaluates the presence or absence of inflammation or stenosis of the esophagus as the "abnormality," and the elasticity characteristic information may include this evaluation result. These determinations and evaluations may be performed by the mapping image generation unit 411 or the control unit 44, or may be performed by a different functional block.

[0061] The generated mapping image is then displayed immediately (in real time). This allows the user to recognize a lack of observation points or a bias in the observation points, and to observe additional locations. Note that the display timing is not limited to real time, and for example, the mapping image may be generated and displayed after a predetermined number of measurements have been taken by the contact pressure detection sensor 246.

[0062] After step S2, a user such as a doctor operates (bends) the bending knob 221 to set the field of view to include the gastric cardia (step S3). In step S2, in response to the operation of the bending knob 221, the insertion section 21 is set into a J-shape with the tip pointing toward the gastric cardia, as shown in Fig. 6. In this state, the outer peripheral surface pressure sensor 9 is disposed at a position to detect the contraction pressure and relaxation pressure at various locations inside the esophagus from the upper esophageal sphincter to the lower esophageal sphincter.

[0063] After step S3, the user, such as a doctor, operates switch 224 to start supplying air from the air supply device 6 to the stomach through the air supply conduit 223 (step S4). In Fig. 6, the state in which intragastric pressure increases due to air supply is represented by a hollow arrow.

[0064] When gas is supplied to the stomach (pneumoperitoneum), if the subject is healthy, the dynamic changes shown below occur in Phases 1 to 3. Before explaining Phases 1 to 3, the structure of the stomach and esophagus will be described with reference to Figures 7 and 8.

[0065] The Intramural Anti-Reflux Barrier Complex (IM-ARB complex), which is part of the anti-reflux mechanism at the gastroesophageal junction, is composed of three main components: Collar Sling Muscle fibers (Figure 8), Clasp Muscle fibers (Figure 8), and the lower esophageal sphincter (Figures 7 and 8).

[0066] The Collar Sling Muscle Fibers are obliquely arranged muscles along the greater curvature of the stomach (Figure 7) and are arranged in a sling-like fashion to surround the upper part of the stomach (Figure 8). These Collar Sling Muscle Fibers tighten the gastric cardia to prevent reflux of gastric contents into the esophagus. From inside the stomach, the Collar Sling Muscle Fibers can be seen as a gastroesophageal flap valve (Figure 7). The gastroesophageal flap valve (GEFV) is a protrusion within the gastroesophageal junction formed by the acute angle between the esophagus and the gastric cardia and is a type of mucosal flap valve (MFV). The MFV located at the gastroesophageal junction is called the gastroesophageal flap valve. The MFV is a flap-shaped portion of the mucosa. The gastroesophageal flap valve changes shape depending on the contraction and relaxation of muscles such as the Collar Sling Muscle Fibers and Clasp Muscle Fibers.

[0067] Clasp muscle fibers are located on the lesser curvature side of the stomach (Figure 7) and consist of a circular muscle layer. These clasp muscle fibers tighten the gastric cardia to prevent the reflux of stomach contents into the esophagus.

[0068] The lower esophageal sphincter (LES) is a ring of muscle located at the junction of the esophagus and stomach that normally contracts to close the esophagus and prevent stomach contents from refluxing.

[0069] Phase 1 begins with the amount of gas sent into the stomach being 0. In Phase 1, the gastroesophageal flap valve and the longitudinal folds of the lesser curvature are observed, as shown in Figures 9(a) and 9(b). The longitudinal folds of the lesser curvature are mucosal folds that extend vertically along the inner wall of the stomach. Their shape changes with the expansion and contraction and relaxation of muscles such as clasp muscle fibers, and they gradually stretch and flatten as intragastric pressure increases. Note that the esophageal mucosa is not observed in Phase 1. Furthermore, in Phase 1, as the amount of gas sent into the stomach increases, the longitudinal folds of the lesser curvature are stretched, the gastroesophageal flap valve gradually flattens, and the crura open, as shown in Figure 9(b). However, the esophageal mucosa is not observed.

[0070] In other words, in Phase 1, it is possible to evaluate the valve function (anti-reflux mechanism) of the stomach structure formed by the gastroesophageal flap valve and the longitudinal folds of the lesser curvature.

[0071] Phase 2 occurs after Phase 1. In Phase 2, as shown in (c) of Figure 9 , the esophageal mucosa is observed beyond the squamocolumnar junction (SCJ) in Figure 7 . The SCJ is the intersection of the esophageal squamous epithelium at the gastroesophageal junction (GEJ) and the gastric columnar epithelium, marking the boundary between different epithelial cells in the digestive tract. The GEJ is located near the border between the esophagus and the stomach and is composed of various anatomical components that form a barrier to prevent reflux of gastric contents. The GEJ also includes collar sling muscle fibers, clasp muscle fibers, the lower esophageal sphincter, the gastroesophageal flap valve, and the SCJ. If the subject is healthy, the scope holding sign (SHS) is observed. The SHS refers to the phenomenon in which the insertion tube 21 is held in place by contraction of the lower esophageal sphincter when intragastric pressure increases. The state shown in FIG. 9(c) is the SHS.

[0072] That is, in Phase 2, it becomes possible to evaluate the valve function (anti-reflux mechanism) of the lower esophageal sphincter.

[0073] Phase 3 occurs after Phase 2. In Phase 3, intragastric pressure exceeds the contractile force of the lower esophageal sphincter, causing the lower esophageal sphincter to relax. In Figure 9(d), the arrow indicates that gas leaks into the esophagus (producing a belching sound) due to the relaxation of the lower esophageal sphincter. If the subject is healthy, peristaltic waves will subsequently descend from the upper esophagus, and SHS will be observed again.

[0074] That is, in Phase 3, it becomes possible to evaluate the acid clearance function by esophageal peristalsis.

[0075] After step S4, a user such as a doctor determines whether the state corresponds to the above-mentioned Phase 1 based on the diagnostic support information displayed on the display device 5, and evaluates the valve function (state of the gastric cardia) based on the structure of the stomach side (step S5).

[0076] After step S5, a user such as a doctor determines whether or not the state corresponds to the above-mentioned Phase 2 based on the diagnostic support information displayed on the display device 5, and evaluates the valve function of the lower esophageal sphincter (the state of the lower esophageal sphincter) (step S6).

[0077] After step S6, a user such as a doctor determines whether or not the state corresponds to the above-mentioned Phase 3 based on the diagnostic support information displayed on the display device 5, and evaluates the acid clearance function due to esophageal peristalsis (the state of peristaltic movement in the esophagus) (step S7).

[0078] Details of the diagnostic assistance information will be explained later in the section "Display Examples of Diagnostic Assistance Information."

[0079] After step S7, a user such as a doctor diagnoses gastroesophageal reflux disease based on the evaluation results of steps S5 to S7 (step S8).

[0080] [Display Examples of Diagnostic Support Information] Next, the diagnostic support information will be described. FIGS. 10 to 12 are diagrams illustrating display examples of the diagnostic support information I0. Specifically, FIG. 10 is a diagram illustrating an example of the diagnostic support information I0 displayed on the display device 5 when the subject is a healthy individual. FIG. 11 is a diagram illustrating an example of the diagnostic support information I0 displayed on the display device 5 when the subject is suspected of having gastroesophageal reflux disease. FIGS. 12(a) to 12(c) are diagrams illustrating examples of the first to third support information I2 to I4 constituting the diagnostic support information I0 when the subject is a healthy individual. FIGS. 12(d) to 12(f) are diagrams illustrating examples of the first to third support information I2 to I4 constituting the diagnostic support information I0 when the subject is suspected of having gastroesophageal reflux disease.

[0081] 10 to 12 , the processing device 4 generates an endoscopic image I1 by performing predetermined image processing on the captured image received from the endoscope 2 during gastric insufflation by the gas insufflation device 6. The processing device 4 also generates first support information I2 based on intragastric pressure information detected by the gastric pressure measuring device 7 during gastric insufflation. The processing device 4 also generates second support information I3 based on esophageal pressure information detected by the outer circumferential pressure sensor 9 during gastric insufflation. The processing device 4 also generates third support information I4 based on belching sound information collected by the microphone 8 during gastric insufflation. The processing device 4 then generates diagnostic support information I0 for diagnosing gastroesophageal reflux disease (GERD) based on the endoscopic image I1, the first to third support information I2 to I4, and time information I5 related to the time during which gas was insufflated into the stomach by the gas insufflation device 6 (hereinafter referred to as pneumoperitoneum time).

[0082] Here, the processing device 4 processes the captured image, intragastric pressure information, esophageal pressure information, and eructation information in a time-synchronized manner based on the synchronization signal generated by the synchronization signal generating unit 42. The processing device 4 also processes the esophageal pressure information detected by the multiple outer peripheral surface pressure sensors 9 in a time-synchronized manner based on the synchronization signal.

[0083] 10 to 12, the first support information I2 includes a pressure waveform I21 that indicates a change in pressure (intragastric pressure) due to an increase in intragastric pressure caused by air being supplied to the stomach by the air supply device 6. Furthermore, as shown in Figures 10 and 11, the first support information I2 also includes a current intragastric pressure (Current IGP) I22, a maximum intragastric pressure (Maximum IGP) I23 during the increase in intragastric pressure, and a basal intragastric pressure (Basal IGP) I24 before the increase in intragastric pressure. The first support information I2 is not limited to the pressure waveform I21, the current intragastric pressure I22, the maximum intragastric pressure I23, and the basal intragastric pressure I24, but may also include the pressure difference between the maximum intragastric pressure I23 and the basal intragastric pressure I24, or a pressure gradient obtained by dividing the pressure difference by the pneumoperitoneum time.

[0084] The current gastric pressure I22, the maximum gastric pressure I23 when the gastric pressure increases, and the basal gastric pressure I24 before the gastric pressure increase may be represented in any suitable manner, such as a bar graph, a line graph, a contour graph, or other representation, or an appropriate combination of these.

[0085] As shown in Figures 10 to 12, the second support information I3 includes a pressure waveform I31 that indicates changes in pressure detected by each of the multiple outer peripheral surface pressure sensors 9. In this embodiment, the pressure waveform I31 is configured as a pressure topography, as shown in Figures 12(c) and 12(f). Specifically, the pressure topography (pressure waveform I31) is a diagram in which the horizontal axis represents time and the changes in pressure detected by the outer peripheral surface pressure sensors 9, each arranged from the upper esophageal sphincter side to the lower esophageal sphincter side, are expressed as a color pattern from above to below the vertical axis.

[0086] The color patterns shown in Figures 12(c) and 12(f) may, for example, use blue to represent low pressure ranges and red to represent high pressure ranges. The time representation shown on the horizontal axis may move horizontally along the time dimension to indicate the passage of time during the time interval during which the displayed pressure values ​​were measured. For example, the rightmost end of the time representation may correspond to the most recent time during the time interval displayed in the time representation, while the leftmost end may represent the earliest time. To indicate the passage of time during the time interval, the time representation may continuously move left on the screen, allowing the user to observe the representation of the pressure measured in the esophagus over time. This continuous leftward movement allows the user to see the change in pressure (if any) at the displayed location over time and the occurrence of an event that caused the change in pressure (e.g., relaxation of the lower esophageal sphincter or peristalsis in the esophagus).

[0087] Furthermore, information indicating the positions within the esophageal tube at which the multiple outer peripheral pressure sensors 9 are arranged may be added to the pressure topography (pressure waveform I31). For example, markers indicating the positions of the upper esophageal sphincter and the lower esophageal sphincter may be added to the vertical axis.

[0088] Furthermore, even if the pressure data detected may be detected at discrete positions due to the number of outer peripheral pressure sensors 9 arranged, the pressure data can be made quasi-continuous in the spatial dimension. The pressure data can be made quasi-continuous by including interpolated pressure values ​​in the pressure data. Based on the quasi-continuous pressure data, a quasi-continuous visual representation (e.g., having a smooth transition) can be provided. Any of the appropriate visual representations described below may be quasi-continuous.

[0089] 10 and 11, the second support information I3 includes a basal LES pressure I32 detected by an outer peripheral surface pressure sensor 9 arranged on the lower esophageal sphincter side before the increase in intragastric pressure. Note that the second support information I3 is not limited to the pressure waveform I31 and the basal pressure I32, and may include maximum and minimum pressure values ​​detected by each of the multiple outer peripheral surface pressure sensors 9.

[0090] 10 to 12, the third support information I4 includes a waveform I41 of an aspiration sound based on the aspiration sound information detected by the microphone 8. Also, the third support information I4 includes a maximum sound pressure of the aspiration sound detected by the microphone 8, as shown in FIGS.

[0091] A user such as a doctor then checks the diagnostic support information I0 displayed on the display device 5 and diagnoses gastroesophageal reflux disease in the subject.

[0092] In step S5, a user such as a doctor checks the diagnostic support information I0 displayed on the display device 5, determines whether the state corresponds to Phase 1, and evaluates the valve function due to the structure of the stomach. For example, if the subject is healthy, the pressure waveform I21 will be a sloped pressure waveform as shown in (a) of Figure 12. On the other hand, if the subject is suspected of having an abnormality, the pressure waveform I21 will be a flat pressure waveform as shown in (d) of Figure 12. For this reason, a user such as a doctor checks, for example, the endoscopic image I1 and the first support information I2 that constitute the diagnostic support information I0, and evaluates the valve function due to the structure of the stomach.

[0093] In step S6, a user such as a doctor checks the diagnostic support information I0 displayed on the display device 5, determines whether or not the state corresponds to Phase 2, and evaluates the valve function of the lower esophageal sphincter. For example, if the subject is healthy, the SHS is observed and the contraction pressure of the lower esophageal sphincter is also observed. Therefore, a user such as a doctor checks, for example, the endoscopic image I1 and the first and second support information I2 and I3 constituting the diagnostic support information I0 to evaluate the valve function of the lower esophageal sphincter.

[0094] In step S7, a user such as a doctor checks the diagnostic support information I0 displayed on the display device 5, determines whether the state corresponds to Phase 3, and evaluates the acid clearance function due to esophageal peristalsis. For example, if the subject is healthy, belching occurs and the lower esophageal sphincter relaxes when the intragastric pressure is approximately 19 mmHg (see (a) to (c) of FIG. 12). On the other hand, if the subject is suspected of having an abnormality, the lower esophageal sphincter relaxes when the intragastric pressure is approximately 14 mmHg. Furthermore, if the subject is healthy, peristalsis in the esophagus is observed (see (c) of FIG. 12), and the SHS is again observed. For example, on the pressure topography (pressure waveform I31), it can be seen that as time progresses (the peristaltic movement moves to the right on the time axis, and the representation itself moves to the left), the contraction pressure in each part of the esophagus progresses from the upper to the lower part (from the upper esophageal sphincter to the lower esophageal sphincter).

[0095] On the other hand, if the subject is suspected of having an abnormality, no peristaltic movement is observed in the esophagus ((f) in FIG. 12). Therefore, a user such as a doctor checks, for example, the endoscopic image I1 and the first to third pieces of support information I2 to I4 that constitute the diagnostic support information I0, to evaluate the acid clearance function due to esophageal peristalsis.

[0096] Then, in step S8, if a user such as a doctor suspects an abnormality in any of the valve function due to the stomach structure, the valve function due to the lower esophageal sphincter, and the acid clearance function due to esophageal peristalsis evaluated in steps S5 to S7, the user diagnoses the patient as having suspected gastroesophageal reflux disease.

[0097] 10 and 11 includes diagnostic result information I6. The diagnostic result information I6 is information related to the diagnostic result obtained by automatically diagnosing whether or not the subject has gastroesophageal reflux disease (whether or not there is a suspicion of gastroesophageal reflux disease) by the processing device 4 based on the captured image, intragastric pressure information, esophageal pressure information, and belching sound information. The diagnostic result information I6 includes an evaluation result I61 which evaluates which phase the current phase is, an evaluation result I62 which evaluates whether or not the subject is in a state corresponding to each of Phases 1 to 3, and a diagnostic result I63 which evaluates whether or not there is gastroesophageal reflux disease (whether or not there is a suspicion of gastroesophageal reflux disease) based on the evaluation result I62.

[0098] The processing device 4 stores in memory the captured image, gastric pressure information, esophageal pressure information, and eructation information, which are processed in a time-synchronized manner based on the synchronization signal. That is, the captured image and various support information can be referenced in a time-synchronized state. Therefore, for example, a user such as a doctor can view the endoscopic image at a time specified by the user, as well as the various support information at that time.

[0099] The present embodiment described above has the following advantages. According to the present embodiment, it is possible to evaluate the valve function due to the stomach structure, the valve function due to the lower esophageal sphincter, and the acid clearance function due to esophageal peristalsis using only the endoscope system 1, without using any other examination equipment, and to diagnose whether or not the subject has gastroesophageal reflux disease. Therefore, according to the present embodiment, it is possible to reduce the burden on the subject when examining whether or not the subject has gastroesophageal reflux disease.

[0100] In the above-described embodiment, an example of generating a mapping image using SLAM has been described. However, from the viewpoint of improving the positional accuracy of the detection position in the mapping image, it is also possible to generate a trained model for constructing a three-dimensional model of the lumen with high accuracy, and create a three-dimensional image using this trained model.

[0101] This trained model is generated, for example, as follows. Figure 13 is a flowchart showing the flow of trained model generation processing for generating a three-dimensional model of a lumen. Here, the processing is described as being executed in a learning device different from the processing device 4, but the processing device 4 may also have the function of executing this learning processing. The learning device is configured using a dedicated processor, such as a general-purpose processor such as a CPU or various arithmetic circuits that execute specific functions, such as an ASIC, and a memory that stores various programs executed by the learning device.

[0102] The learning device first acquires an endoscopic image of a human body model serving as a body cavity phantom and the distance to the subject depicted in the endoscopic image (step S11). For example, a user operating an endoscope inserts the endoscope into a human body model for endoscopic training and acquires a captured image (endoscopic image). At this time, a measurement mechanism such as an infrared sensor or stereo camera measures the distance from the tip of the endoscope (e.g., the imaging surface) to the subject, and the measurement result is associated with the endoscopic image. The endoscopic image and distance information thus acquired are input into the learning device, which then acquires the endoscopic image and distance information. At this time, the human body model for endoscopic training is a known three-dimensional model. The endoscopic image and distance information may be 3D-CG (3-dimensional computer graphics) images.

[0103] Fig. 14 is a diagram showing an example of an endoscopic image acquired in a human body model for endoscopic training. In step S11, for example, an endoscopic image as shown in Fig. 14 is acquired. A plurality of such endoscopic images are acquired along a lumen. At this time, each endoscopic image is associated with information about the distance between the endoscope (tip) and the subject.

[0104] The learning device then converts the endoscopic images into organ images (step S12). At this time, the learning device converts the endoscopic images based on a human body model for endoscopic training into organ images that represent actual human organs. For this conversion, for example, an image conversion model is used, and known conversion models such as simulation models and trained models can be adopted. The organ images are associated with the distances associated with the corresponding endoscopic images.

[0105] Fig. 15 is a diagram showing an example of an organ image obtained by converting the subject shown in the endoscopic image shown in Fig. 14 into an organ. In step S12, for example, an organ image as shown in Fig. 15 is acquired. This organ image is associated with distance information related to the corresponding endoscopic image.

[0106] The learning device then performs learning using the organ images and the distances associated with the organ images (step S13). This learning generates a trained model that outputs a three-dimensional model of the organ. This trained model is, for example, a neural network composed of an input layer, an intermediate layer, and an output layer, generated by supervised learning with the organ images and distance information as explanatory variables and a three-dimensional model of the lumen (e.g., the three-dimensional model of the human body model used in endoscopic training) as the objective variable. Note that, in addition to neural networks, models generated by known machine learning or deep learning can also be used as long as they can generate a three-dimensional model based on input images and distances.

[0107] The learning device then outputs the generated trained model (step S14). The trained model is output to a memory provided in the learning device, an external server, or the processing device 4.

[0108] The processing device 4 generates a three-dimensional model of the lumen by inputting endoscopic images acquired of the subject into the trained model generated as described above, and sets the detection points and elastic property information detected by the contact pressure sensor in this three-dimensional model.

[0109] While the embodiments of the present invention have been described above, the present invention should not be limited to the above-described embodiments. In the above-described embodiments, in order to diagnose gastroesophageal reflux disease, at least two pieces of information, namely, an endoscopic image and esophageal pressure information, are sufficient, and it is not necessary to use all of the information, namely, the endoscopic image, the esophageal pressure information, the gastric pressure information, and the eructation information.

[0110] Although the above-described embodiment has been described as a technology capable of diagnosing gastroesophageal reflux disease, the present invention is not limited to this. For example, it can also evaluate the relaxation failure of the lower esophageal sphincter, which can be useful for diagnosing esophageal achalasia. It can also evaluate the function of the major sphincters present in the human digestive tract. Specifically, it can evaluate the external anal sphincter, i.e., physiological anorectal function associated with aging.

[0111] REFERENCE SIGNS LIST 1 Endoscope system 2 Endoscope 3 Light source device 4 Processing device 5 Display device 6 Air supply device 7 Gastric pressure measuring device 8 Microphone 9 Outer circumferential surface pressure sensor 21 Insertion section 22 Operation section 23 Universal cord 24 Tip section 25 Bending section 26 Flexible tube section 31 Light source section 32 Lighting control section 33 Light source driver 41 Image processing section 42 Synchronization signal generation section 43 Input section 44 Control section 45 Memory section 61 Air supply section 62 Flow rate measurement section 63 Control section 71 Pressure measurement probe 221 Bending knob 222 Treatment tool insertion section 223 Air supply conduit 224 Switch 231, 232 Connector 241 Light guide 242 Illumination lens 243 Optical system 244 Image pickup element 245 Collective cable 411 Mapping image generation section I0 Diagnostic support information I1 Endoscopic image I2 First support information I21 Pressure waveform I22 Current intragastric pressure I23 Maximum intragastric pressure I24 Basal intragastric pressure I3 Second support information I31 Pressure waveform I32 Basal pressure I4 Third support information I41 Waveform I5 Time information I6 Diagnostic result information I61, I62 Evaluation result I63 Diagnostic result TU Air supply tube

Claims

an endoscope having an insertion section with an imaging element disposed at the tip; a contact pressure sensor provided in the insertion portion to detect a contact pressure with a body cavity of the subject; a processor; The processor: An endoscopic system that generates a mapping image in which the acquisition position of the elastic property information estimated on a three-dimensional model representing the body cavity is associated with the three-dimensional model, based on the captured image captured by the imaging element and the elastic property information detected by the contact pressure sensor. 　 a plurality of pieces of elastic property information are acquired along a movement of the body cavity of the subject; Each piece of elastic property information is associated with a contact position with the body cavity on the three-dimensional model. The endoscope system according to claim 1 . 　 a plurality of the captured images are acquired along the movement of the body cavity of the subject; The three-dimensional model is generated by integrating a plurality of the captured images. The endoscope system according to claim 1 . 　 the captured image is composed of a plurality of images forming a set acquired by a plurality of image pickup elements having optical axes at angles different from each other, and the captured image is acquired along the movement of the body cavity of the subject; the processor integrates the plurality of captured images to generate the three-dimensional model. The endoscope system according to claim 3 . 　 the processor generates the three-dimensional model using SLAM; The endoscope system according to claim 1 . 　 the processor updates the mapping image using the captured image and / or the elasticity property information at a timing when the captured image and / or the elasticity property information is acquired. The endoscope system according to claim 3 . 　 The processor generates the elasticity property information by evaluating the elasticity value calculated from the elasticity property information based on a preset threshold value. The endoscope system according to claim 1 . 　 the processor generates elasticity property information that designates an elasticity value as an abnormal value when the elasticity value calculated from the elasticity property information is outside a predetermined normal range. The endoscope system according to claim 7 . 　 The processor generates the three-dimensional model using a trained model. The endoscope system according to claim 1 . 　 the processor generates a mapping image on the three-dimensional model, in which the elastic property information is color-coded according to the elastic property information and displayed in correspondence with the acquisition position. The endoscope system according to claim 1 . 　 the processor generates a mapping image on the three-dimensional model, the mapping image displaying the elastic property information indicating an abnormal value in correspondence with the acquisition position. The endoscope system according to claim 1 . 　 the processor determines whether the value is an abnormal value by comparing the elasticity property information obtained in the past; The endoscope system according to claim 11. 　 the processor determines the abnormal value using a trained model generated by learning the elastic property information previously acquired. The endoscope system according to claim 11. 　 the processor generates the three-dimensional model using the trained model trained using a converted image obtained by converting an image obtained by imaging a body cavity phantom into an organ image and a three-dimensional model of the body cavity phantom; The endoscope system according to claim 9 . 　 the processor generates the mapping image in which evaluation information evaluating the presence or absence of inflammation or stenosis of the esophagus determined based on the elasticity property information is associated with the three-dimensional model showing the esophagus. The endoscope system according to claim 1 . 　 an outer peripheral surface pressure sensor that detects a pressure applied to an outer peripheral surface of the insertion portion; an air supply device that supplies gas through an air supply conduit provided in the insertion portion; Furthermore, The processor: The endoscope system according to claim 1, wherein diagnostic support information for diagnosing gastroesophageal reflux disease is generated based on an image captured by the imaging element and esophageal pressure information relating to the pressure detected by the outer peripheral pressure sensor. 　 The computer An image processing method for generating a mapping image in which the acquisition position of the elastic property information estimated on a three-dimensional model representing the body cavity is associated with the three-dimensional model, based on an image captured by an imaging element provided at the tip of an insertion portion of an endoscope and elastic property information detected by a contact pressure sensor provided in the insertion portion that detects the contact pressure with the body cavity of an examinee. 　 An image processing program that causes a computer to execute a process of generating a mapping image in which the acquisition position of the elastic property information estimated on a three-dimensional model representing the body cavity is associated with the three-dimensional model, based on an image captured by an imaging element provided at the tip of an insertion portion of an endoscope and elastic property information detected by a contact pressure sensor provided in the insertion portion that detects the contact pressure with the body cavity of a subject.

Images