Auxiliary device, endoscope system, auxiliary method, and storage medium

Abstract

An assisting device, an endoscope system, an assisting method, and a program capable of objectively grasping an indwelling period when a medical instrument is to be indwelled in a lumen are provided. The assisting device has a generation section (406a) that generates a fluorescence image based on an imaging signal generated by imaging fluorescence generated by exciting light that is irradiated to biological tissue, a calculation section (406c) that calculates a fluorescence intensity based on the fluorescence image, an estimation section (406d) that estimates an indwelling period of a medical instrument to be indwelled in the lumen based on the fluorescence intensity, and an output section (406e) that outputs indwelling period information related to the indwelling period and an observation image obtained by imaging the biological tissue.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an assisting device that judges a period of indwelling of a stent to be indwelled in a urinary tract

[0002] An assisting device, an endoscope system, an assisting method, and a program that assist in a period of indwelling of a medical instrument to be indwelled in a lumen for a certain period of time. BACKGROUND

[0003] In the past, in an endoscope, a technique of performing fragmentation by irradiating a laser beam toward a stone generated in a urinary tract is known (for example, refer to Patent Literature 1). In this technique, in a case where it is possible to confirm that a collimating light beam is incident on a target mass such as a stone, an energy source is operated to irradiate an energy pulse on the target mass via an energy guide. In this case, since there is a possibility of damaging a urinary canal, a stent is indwelled after the stone is fragmented in order to protect the urinary canal (for example, refer to Patent Literature 2).

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Laid-Open No. 2017-500172

[0007] Patent Literature 2: Japanese Patent Application Laid-Open No. 2017-510371 SUMMARY

[0008] PROBLEMS TO BE SOLVED BY THE INVENTION

[0009] However, in the above-described Patent Literatures 1 and 2, it is difficult to visually confirm the degree of invasion of thermal damage of a living body tissue caused by a laser beam on a display monitor, and the period of indwelling of a medical instrument such as a stent in a lumen such as a urinary tract is determined according to an experience value of a surgeon, so it is difficult to grasp an optimal period of indwelling of a medical instrument.

[0010] The present disclosure was completed in view of the above-described circumstances, and aims to provide an assisting device, an endoscope system, an assisting method, and a program that can objectively grasp a period of indwelling when a medical instrument is to be indwelled in a lumen.

[0011] MEANS FOR SOLVING THE PROBLEMS

[0012] In order to solve the above-described problems and achieve the object, the assisting device of the present disclosure has: a generation section that generates a fluorescence image based on an imaging signal generated by imaging fluorescence generated by excitation light irradiated to a living body tissue; a calculation section that calculates a fluorescence intensity based on the fluorescence image; an estimation section that estimates a period of indwelling of a medical instrument to be indwelled in a lumen based on the fluorescence intensity; and an output section that outputs indwelling period information related to the period of indwelling and an observation image obtained by imaging the living body tissue.

[0013] Further, the assisting apparatus of the present disclosure in the above disclosure, the estimation section estimates the invasiveness of the energy device to the biological tissue based on the fluorescence intensity, and estimates the indwelling period based on the invasiveness.

[0014] Further, the assisting apparatus of the present disclosure in the above disclosure, the estimation section estimates the invasiveness based on correlation information representing a correlation between the invasiveness and the fluorescence intensity, which is measured in advance, and the fluorescence intensity.

[0015] Further, the assisting apparatus of the present disclosure in the above disclosure, the estimation section estimates whether or not the medical instrument is indwelled in the lumen based on the fluorescence intensity.

[0016] Further, the assisting apparatus of the present disclosure in the above disclosure, the output section outputs the indwelling period information superimposed on the observation image.

[0017] Further, the assisting apparatus of the present disclosure in the above disclosure, the assisting apparatus further has an extraction section that extracts a fluorescence region from the fluorescence image, and in a case where a plurality of the fluorescence regions are extracted by the extraction section, the estimation section estimates the indwelling period based on the fluorescence intensity of the fluorescence region located closest to a near point side of an imaging optical system.

[0018] Further, the assisting apparatus of the present disclosure in the above disclosure, the imaging signal is a signal obtained by imaging a urinary tract extending in a depth direction, and in a case where a plurality of the fluorescence regions are extracted by the extraction section, the estimation section estimates the indwelling period based on the fluorescence intensity of the fluorescence region located closest to a near point side of an imaging optical system.

[0019] Further, the assisting apparatus of the present disclosure in the above disclosure, in a case where a plurality of the fluorescence regions are extracted by the extraction section, the output section outputs each of the plurality of the fluorescence regions in an identifiable manner based on the fluorescence intensity of each of the plurality of the fluorescence regions.

[0020] Further, the assisting apparatus of the present disclosure in the above disclosure, the medical instrument is any one of a stent, a catheter, and an indwelling needle.

[0021] Further, the assisting apparatus of the present disclosure in the above disclosure, the lumen is a urinary tract.

[0022] Further, the assisting apparatus of the present disclosure in the above disclosure, the wavelength band of the excitation light is 390 nm to 430 nm, the wavelength band of the fluorescence is 500 nm to 640 nm, and the imaging signal is a signal obtained by imaging transmitted light after a transmission filter that shields light on a shorter wavelength side than the 430 nm.

[0023] Further, the endoscope system of the present disclosure includes an endoscope that is capable of being inserted into a lumen of an object, a light source device that is capable of irradiating excitation light that excites a glycosylation end product generated by performing thermal treatment on biological tissue, and a control device with which the endoscope is detachably attached, the endoscope having an imaging element that is capable of generating an imaging signal by imaging fluorescence emitted using the excitation light and a cut filter that is provided on the light-receiving surface side of the imaging element and that blocks light on the short-wavelength side of a part of a wavelength band including the excitation light, the control device having an assisting device that assists a surgeon, the assisting device having a generation section that generates a fluorescence image based on the imaging signal, a calculation section that calculates a fluorescence intensity based on the fluorescence image, an estimation section that estimates a period of indwelling of a medical instrument to be indwelled in the lumen based on the fluorescence intensity, and an output section that outputs indwelling period information related to the period of indwelling and an observation image obtained by imaging the biological tissue.

[0024] Further, the assisting method of the present disclosure is an assisting method that an assisting device executes, and includes a generation step of generating a fluorescence image based on an imaging signal that is generated by imaging fluorescence generated using excitation light that is irradiated to biological tissue, a calculation step of calculating a fluorescence intensity based on the fluorescence image, an estimation step of estimating a period of indwelling of a medical instrument to be indwelled in a lumen based on the fluorescence intensity, and an output step of outputting indwelling period information related to the period of indwelling and an observation image obtained by imaging the biological tissue.

[0025] Further, the program of the present disclosure is a program that causes an assisting device to execute, the program causing execution of a generation step of generating a fluorescence image based on an imaging signal that is generated by imaging fluorescence generated using excitation light that is irradiated to biological tissue, a calculation step of calculating a fluorescence intensity based on the fluorescence image, an estimation step of estimating a period of indwelling of a medical instrument to be indwelled in a lumen based on the fluorescence intensity, and an output step of outputting indwelling period information related to the period of indwelling and an observation image obtained by imaging the biological tissue.

[0026] Effects of Invention

[0027] According to the present disclosure, the effect of being able to objectively grasp the period of indwelling when a medical instrument is to be indwelled in a lumen is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a diagram that schematically shows the overall structure of an endoscope system of one embodiment.

[0029] Figure 2is a block diagram showing a functional configuration of main parts of an endoscope system according to an embodiment.

[0030] Figure 3 is a graph schematically showing a wavelength characteristic of excitation light emitted from a second light source section according to an embodiment.

[0031] Figure 4 is a graph schematically showing a transmittance characteristic of a cut filter according to an embodiment.

[0032] Figure 5 is a graph showing an example of relevant information recorded by a relevant information recording section according to an embodiment.

[0033] Figure 6 is a graph schematically showing an observation principle of a fluorescence observation mode of an endoscope system according to an embodiment.

[0034] Figure 7 is a graph schematically showing an observation principle of a general light observation mode of an endoscope system according to an embodiment.

[0035] Figure 8 is a flowchart showing a surgical procedure in which a surgeon performs a transurethral ureteral stone removal procedure using an endoscope system.

[0036] Figure 9A is a graph showing an example of a change in an image displayed by a display device in a transurethral ureteral stone removal procedure.

[0037] Figure 9B is a graph showing an example of a change in an image displayed by a display device in a transurethral ureteral stone removal procedure.

[0038] Figure 9C is a graph showing an example of a change in an image displayed by a display device in a transurethral ureteral stone removal procedure.

[0039] Figure 9D is a graph showing an example of a change in an image displayed by a display device in a transurethral ureteral stone removal procedure.

[0040] Figure 9E is a graph showing an example of a change in an image displayed by a display device in a transurethral ureteral stone removal procedure.

[0041] Figure 9F is a graph showing an example of a change in an image displayed by a display device in a transurethral ureteral stone removal procedure.

[0042] Figure 10 is a flowchart showing an outline of a process performed by an endoscope system according to an embodiment.

[0043] Figure 11FIG. 1 is a diagram schematically illustrating an estimation method of an estimation unit of an endoscope system of one embodiment regarding estimation of whether or not to indwell a stent.

[0044] Figure 12 FIG. 2 is a diagram schematically illustrating an estimation method of an estimation unit of an endoscope system of one embodiment regarding estimation of a stent indwelling period.

[0045] Figure 13 FIG. 3 is a diagram showing an example of an output screen outputted by an output unit of an endoscope system of one embodiment to a display device.

[0046] Figure 14 FIG. 4 is a diagram showing another example of an output screen outputted by an output unit of an endoscope system of one embodiment to a display device.

[0047] Figure 15 FIG. 5 is a diagram showing another example of an output screen outputted by an output unit of an endoscope system of one embodiment to a display device.

[0048] Figure 16 FIG. 6 is a diagram showing another example of an output screen outputted by an output unit of an endoscope system of one embodiment to a display device.

[0049] Figure 17 FIG. 7 is a diagram schematically showing a transmission characteristic of a cutoff filter of a modification example of one embodiment. DETAILED DESCRIPTION

[0050] Hereinafter, as a mode for carrying out the present disclosure (hereinafter, referred to as "embodiment"), a flexible-scope used in transurethral ureterolithotripsy (hereinafter, referred to as "TUL") is used to describe an endoscope system, but is not limited thereto, and can be applied to a rigid scope, a surgical robot, and the like, for example. In addition, the present disclosure is not limited to this embodiment. In addition, in the drawings, the same parts are given the same reference numerals and described. Note that the drawings are schematic, and the relationship between the thickness and the width of each component, the ratio of each component, and the like are different from actual ones. In addition, the drawings also include parts different from each other in size and ratio.

[0051] Structure of Endoscope System

[0052] Figure 1 FIG. 1 is a diagram schematically illustrating an estimation method of an estimation unit of an endoscope system of one embodiment regarding estimation of whether or not to indwell a stent. Figure 1The illustrated endoscope system 1 captures the inside of a subject such as a patient by inserting an insertion portion of an endoscope into a body cavity or a lumen of a subject, for example, a urinary tract, and displays a display image based on a captured image signal on a display device. Here, the urinary tract refers to the urethra, the bladder, the ureter, the kidney, and the like, which are tubular in shape and extend in the depth direction. An operator such as a doctor performs observation of the display image displayed on the display device while breaking up stones in the inside of the subject by a laser irradiation device that irradiates a high-output infrared laser such as a holmium YAG laser through the endoscope, and removes the broken stones by a treatment instrument such as a basket catheter, and leaves a medical instrument in the urinary tract for a prescribed period. Here, the medical instrument refers to any one of a stent, a catheter, and an indwelling needle. The endoscope system 1 includes an endoscope 2, a display device 3, a control device 4, and a laser irradiation device 5.

[0053] Structure of endoscope

[0054] First, the structure of the endoscope 2 will be described.

[0055] The endoscope 2 generates a captured image signal (RAW data) obtained by capturing the inside of a subject, and outputs the generated captured image signal to the control device 4. The endoscope 2 includes an insertion portion 21, an operation portion 22, and a general-purpose cord 23.

[0056] The insertion portion 21 is inserted into the inside of a subject. The insertion portion 21 has an elongated shape with flexibility. The insertion portion 21 has a front end portion 24 in which a captured image element described later is built in, a bendable bend portion 25 composed of a plurality of bend blocks, and a long strip-shaped flexible tube portion 26 connected to the proximal end side of the bend portion 25 and having flexibility.

[0057] The front end portion 24 is composed of glass fiber or the like. The front end portion 24 forms a light guide path of an illumination light supplied from the control device 4 through the general-purpose cord 23 and the operation portion 22, and generates a captured image signal obtained by capturing a return light of the illumination light and outputs the captured image signal to the control device 4.

[0058] The operation portion 22 has a bend knob 221 that bends the bend portion 25 in the up-down direction and the left-right direction, a treatment instrument insertion portion 222 that inserts a treatment instrument into the inside of a subject, and a plurality of switches 223 as operation input portions that input an operation instruction signal to the control device 4 and a surrounding device such as a gas supply unit, a water supply unit, a gas body supply unit, a pre-freezing signal that instructs still image capturing of the endoscope system 1, or a switching signal that switches the observation mode of the endoscope system 1. A treatment instrument inserted from the treatment instrument insertion portion 222 is exposed from an opening portion (not shown) through a treatment instrument channel (not shown) of the front end portion 24. Here, as the treatment instrument, there are the laser irradiation device 5 and a basket catheter, and the like.

[0059] The general-purpose cord 23 has at least a light guide and a collection cable that collects one or a plurality of cables. The collection cable is a signal line that transmits and receives signals between the endoscope 2 and the control apparatus 4, and includes a signal line for transmitting and receiving a camera signal (RAW data) and a signal line for transmitting and receiving a timing signal (synchronization signal and clock signal) for driving a camera element described later. The general-purpose cord 23 has a connector portion 27 that is detachable with respect to the control apparatus 4, and a connector portion 28 in which a spiral cable 27a is extended and is detachable with respect to the control apparatus 4 at an extension end of the spiral cable 27a.

[0060] Structure of display apparatus

[0061] Next, the structure of the display apparatus 3 will be described.

[0062] The display apparatus 3 displays a display image based on an image signal input from the control apparatus 4 under the control of the control apparatus 4. The display apparatus 3 is realized using a display panel such as an organic EL (Electro Luminescence) or a liquid crystal.

[0063] Structure of control apparatus

[0064] Next, the structure of the control apparatus 4 will be described.

[0065] The control apparatus 4 controls each part of the endoscope system 1. The control apparatus 4 supplies illumination light for the endoscope 2 to irradiate a subject. In addition, the control apparatus 4 performs various image processing on a camera signal input from the endoscope 2 and outputs to the display apparatus 3.

[0066] Structure of laser irradiation apparatus

[0067] Next, the structure of the laser irradiation apparatus will be described.

[0068] The laser irradiation apparatus 5 is inserted into the body of a subject, for example, a urinary tract (for example, a kidney, a ureter, a bladder, and a urethra), via the treatment instrument insertion portion 222 of the endoscope 2, and under the operation of an operator, irradiates a high-output infrared laser such as a holmium YAG laser toward a stone generated in the body of the subject, thereby breaking the stone.

[0069] Functional structure of main part of endoscope system

[0070] Next, the functional structure of the main part of the endoscope system 1 described above will be described. Figure 2 is a block diagram showing the functional structure of the main part of the endoscope system 1.

[0071] Structure of endoscope

[0072] First, the structure of the endoscope 2 will be described.

[0073] The endoscope 2 includes an illumination optical system 201, an imaging optical system 202, a cut filter 203, an imaging element 204, an A / D conversion section 205, a P / S conversion section 206, an imaging recording section 207, and an imaging control section 208. Further, the illumination optical system 201, the imaging optical system 202, the cut filter 203, the imaging element 204, the A / D conversion section 205, the P / S conversion section 206, the imaging recording section 207, and the imaging control section 208 are respectively arranged in the distal end portion 24.

[0074] The illumination optical system 201 irradiates the subject (biological tissue) with illumination light supplied from a light guide 231 formed of an optical fiber or the like. The illumination optical system 201 is implemented using one or a plurality of lenses or the like.

[0075] The imaging optical system 202 forms an object image (light rays) on a light-receiving surface of the imaging element 204 by condensing light such as reflected light from the subject, return light from the subject, and fluorescent light emitted from the subject. The imaging optical system 202 is implemented using one or a plurality of lenses or the like.

[0076] The cut filter 203 is arranged on an optical axis O1 of the imaging optical system 202 and the imaging element 204. The cut filter 203 blocks light of a wavelength band of reflected light or return light of excitation light supplied from the control device 4 described later, and transmits light of a wavelength band on the long wavelength side than the wavelength band of the excitation light. The transmission characteristics of the cut filter 203 are described later.

[0077] The imaging element 204 receives the object image (light rays) imaged by the imaging optical system 202 and transmitted through the cut filter 203 under the control of the imaging control section 208, performs photoelectric conversion to generate an imaging signal (RAW data) and outputs to the A / D conversion section 205. The imaging element 204 is implemented using a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, which is configured with any one of color filters constituting a Bayer arrangement (RGGB) arranged in each of a plurality of pixels arranged in a two-dimensional matrix.

[0078] The A / D conversion section 205 performs A / D conversion processing on the analog imaging signal input from the imaging element 204 under the control of the imaging control section 208 and outputs to the P / S conversion section 206. The A / D conversion section 205 is implemented using an A / D conversion circuit or the like.

[0079] Under the control of the camera control unit 208, the P / S conversion unit 206 performs parallel / serial conversion on the digital camera signal input from the A / D conversion unit 205, and outputs the converted camera signal to the control device 4 via the first transmission cable 232. The P / S conversion unit 206 is implemented using a P / S conversion circuit or the like. Alternatively, in Embodiment 1, an E / O conversion unit that converts the camera signal into an optical signal may be provided instead of the P / S conversion unit 206, outputting the camera signal to the control device 4 via the optical signal. For example, the camera signal can also be transmitted to the control device 4 via wireless communication such as Wi-Fi (Wireless Fidelity).

[0080] The camera recording unit 207 records various information related to the endoscope 2 (such as pixel information of the imaging element 204 and characteristics of the cutoff filter 203). Additionally, the camera recording unit 207 records various setting data and control parameters transmitted from the control device 4 via the second transmission cable 233. The camera recording unit 207 is constructed using either non-volatile memory or volatile memory.

[0081] The camera control unit 208 controls the operation of the camera element 204, the A / D converter 205, and the P / S converter 206 based on the setting data received from the control device 4 via the second transmission cable 233. The camera control unit 208 is implemented using a TG (Timing Generator), a processor as a processing device with hardware such as a CPU, and a memory as a temporary storage area used by the processor.

[0082] Structure of the control device

[0083] Next, the structure of control device 4 will be explained.

[0084] The control device 4 includes a condenser lens 401, a first light source unit 402, a second light source unit 403, a light source control unit 404, an S / P conversion unit 405, an image processing unit 406, an input unit 407, a recording unit 408, and a control unit 409.

[0085] The condenser lens 401 converges the light emitted by the first light source 402 and the second light source 403 and directs it toward the light guide 231. The condenser lens 401 is constructed using one or more lenses.

[0086] The first light source section 402 supplies white light, which is visible light, to the light guide 231 by emitting white light (general light) as visible light under the control of the light source control section 404. The first light source section 402 is configured using a collimator lens, a white LED (Light Emitting Diode) lamp, and a driver, and the like. In addition, the first light source section 402 can supply white light, which is visible light, by simultaneously emitting a red LED lamp, a green LED lamp, and a blue LED lamp. Of course, the first light source section 402 can be configured using a halogen lamp, a xenon lamp, and the like.

[0087] The second light source section 403 supplies narrow-band light as illumination light to the light guide 231 by emitting excitation light having a prescribed wavelength band under the control of the light source control section 404. Here, the wavelength band of the excitation light is 400 nm to 430 nm (center wavelength: 415 nm). The second light source section 403 is implemented using a collimator lens, a semiconductor laser such as a violet LD (laser Diode), and a driver, and the like. In addition, the wavelength characteristics of the white light emitted by the first light source section 402 and the excitation light emitted by the second light source section 403 are described later.

[0088] The light source control section 404 is configured using a processor, which is a processing device, and a memory, which is a temporary storage area used by the processor, and has hardware such as an FPGA (Field-Programmable Gate Array) or a CPU (Central Processing Unit). The light source control section 404 controls the light emission timing, the light emission intensity, and the light emission time, and the like of each of the first light source section 402 and the second light source section 403 in accordance with control data input from the control section 409.

[0089] The S / P conversion section 405 outputs an image pickup signal received from the endoscope 2 via the first transmission cable 232 to the image processing section 406 under the control of the control section 409 by performing serial / parallel conversion. In addition, in the case where the endoscope 2 outputs an image pickup signal as an optical signal, an O / E conversion section that converts an optical signal into an electrical signal can be provided instead of the S / P conversion section 405. Also, in the case where the endoscope 2 transmits an image pickup signal by wireless communication, a communication module that can receive a wireless signal can be provided instead of the S / P conversion section 405.

[0090] The image processing section 406 is realized using a processor having a CPU, a GPU (Graphics Processing Unit), or an FPGA, or the like, and a memory used as a temporary storage area for the processor. The image processing section 406, under the control of the control section 409, performs prescribed image processing on the imaging signal input from the S / P conversion section 405 and outputs to the display device 3. Further, in one embodiment, the image processing section 406 functions as an auxiliary device. The image processing section 406 has a generation section 406a, a calculation section 406c, an extraction section 406b, an estimation section 406d, and an output section 406e.

[0091] The generation section 406a generates a fluorescence image based on an imaging signal generated by imaging fluorescence generated by irradiation of excitation light to biological tissue. Specifically, the generation section 406a acquires an imaging signal from the imaging element 204 of the endoscope 2 (hereinafter, only described as "acquired from the imaging element 204 of the endoscope 2") via the A / D conversion section 205, the P / S conversion section 206, the first transmission cable 232, and the S / P conversion section 405. Then, the generation section 406a generates a fluorescence image based on the imaging signal acquired from the imaging element 204 of the endoscope 2, which is generated by imaging fluorescence generated by irradiation of excitation light to biological tissue. In addition, the generation section 406a generates an observation image (white light image) as a display image based on an imaging signal generated by imaging reflected light and return light from biological tissue due to irradiation of white light to biological tissue.

[0092] The extraction section 406b extracts a fluorescence region from the fluorescence image. Specifically, the extraction section 406b extracts a fluorescence region by performing a 2-value processing on each pixel of the fluorescence image. For example, the extraction section 406b extracts a fluorescence region by extracting a pixel of a prescribed value or more from a pixel value of each pixel of the fluorescence image.

[0093] The calculation section 406c calculates a fluorescence intensity from the fluorescence image generated by the generation section 406a. Specifically, the calculation section 406c calculates a fluorescence intensity of the fluorescence region extracted by the extraction section 406b in the fluorescence image generated by the generation section 406a.

[0094] The estimation unit 406d estimates the indwelling period of the medical instrument to be indwelled in the lumen based on the fluorescence intensity calculated by the calculation unit 406c. Here, the lumen refers to the urinary tract. In addition, the urinary tract includes the urethra, the bladder, the urinary tube, and the kidney. In addition, the medical instrument refers to any one of a stent, a catheter, and an indwelling needle. In addition, the estimation unit 406d estimates the invasiveness of the energy device to the biological tissue based on the fluorescence intensity calculated by the calculation unit 406c, and estimates the indwelling period of the medical instrument based on the invasiveness. Specifically, the estimation unit 406d estimates the invasiveness of the energy device to the biological tissue based on the correlation information indicating the correlation between the invasiveness and the fluorescence intensity recorded by the correlation information recording unit 408b described later and the fluorescence intensity calculated by the calculation unit 406c.

[0095] The output unit 406e outputs, to the display device 3, the indwelling period information related to the indwelling period of the medical instrument estimated by the estimation unit 406d and the observation image of the display image obtained by imaging the biological tissue.

[0096] The input unit 407 receives inputs of various operations related to the endoscope system 1 and outputs the received operations to the control unit 409. The input unit 407 is configured using a mouse, a foot switch, a keyboard, a button, a switch, a touch panel, and the like.

[0097] The recording unit 408 is realized using a volatile memory, a non-volatile memory, an SSD (Solid State Drive), an HDD (Hard Disk Drive), and the like, a recording medium such as a memory card. The recording unit 408 records data including various parameters required for the operation of the endoscope system 1 and the like. Also, the recording unit 408 has a correlation information recording unit 408b and a program recording unit 408a that records various programs for operating the endoscope system 1.

[0098] The correlation information recording unit 408b records correlation information indicating the correlation between the invasiveness of the laser irradiation device 5 to the biological tissue of the subject and the intensity of the fluorescence emitted when the excitation light is irradiated to the biological tissue that has been thermally treated by the laser irradiation device 5. In addition, the details of the correlation information will be described later.

[0099] The control unit 409 is realized using a processor having hardware such as an FPGA or a CPU and a memory that is a temporary storage area used by the processor. The control unit 409 uniformly controls each unit constituting the endoscope system 1.

[0100] Wavelength characteristics of excitation light

[0101] Next, the wavelength characteristics of the excitation light emitted by the second light source unit 403 will be described.

[0102] Figure 3 This is a diagram schematically illustrating the wavelength characteristics of the excitation light emitted by the second light source unit 403. Figure 3 In the graph, the horizontal axis represents wavelength (nm), and the vertical axis represents wavelength characteristics. Additionally, in... Figure 3 In the middle, the broken line L V This indicates the wavelength characteristics of the excitation light emitted by the second light source unit 403. Additionally, in Figure 3 In the middle, curve L B The blue band is represented by curve L. G The green band is represented by curve L. R The red band is indicated.

[0103] like Figure 3 As shown, the second light source 403 emits excitation light with a center wavelength (peak wavelength) of 415nm and a band of 400nm to 430nm.

[0104] Transmission characteristics of cut-off filters

[0105] Next, the transmission characteristics of the cutoff filter 203 will be explained.

[0106] Figure 4 This is a schematic diagram illustrating the transmission characteristics of the cutoff filter 203. Figure 4 In the diagram, the horizontal axis represents wavelength (nm), and the vertical axis represents transmission characteristics. Additionally, in... Figure 4 In the middle, the broken line L F The broken line L represents the transmission characteristics of the cutoff filter 203. V The broken line L represents the wavelength characteristics of the excitation light. NG The wavelength characteristics of fluorescence generated by irradiating the glycosylation end product with excitation light, which is produced by heat treatment of biological tissue by an energy device, such as a laser irradiation device 5.

[0107] like Figure 4 The broken line L V and broken line L NG As shown, the cutoff filter 203 blocks a portion of the excitation light reflected from the biological tissue in the observation area, allowing light of other wavelengths containing fluorescent components to pass through. Specifically, the cutoff filter 203 blocks a portion of the short-wavelength band containing the excitation light, which is greater than or equal to 400 nm and less than 430 nm, and allows light of a longer wavelength band containing fluorescence, which is generated by irradiating the glycosylation end product produced by heat treatment, with the excitation light.

[0108] Related information

[0109] Next, an example of the relevant information recorded by the relevant information recording unit 408b will be explained.

[0110] Figure 5 is a graph showing an example of the relevant information recorded by the relevant information recording section 408b. In Figure 5 , the vertical axis indicates the fluorescence intensity, and the horizontal axis indicates the invasiveness (depth and area) of the thermal treatment to the biological tissue. In Figure 5 , the straight line Ly indicates the correlation between the luminescence intensity and the invasiveness (depth and area) of the thermal treatment to the biological tissue.

[0111] As indicated by the straight line Ly of Figure 5 , the greater the invasiveness of the thermal treatment to the biological tissue, the stronger the luminescence intensity.

[0112] Outline of the fluorescence observation mode

[0113] Next, the fluorescence observation mode (thermal treatment observation mode) that can be performed in the endoscope system 1 will be described. Figure 6 is a graph schematically showing the observation principle at the time of the fluorescence observation mode.

[0114] As shown in the graph G11 of Figure 6 , first, the control device 4 irradiates the biological tissue O10 (thermal treatment area) after the thermal treatment to the subject is performed by the laser irradiation device 5 with excitation light (center wavelength 415 nm) by causing the second light source section 403 to emit light. In this case, as shown in the graph G12 of Figure 6 , at least the reflected light (hereinafter, simply referred to as "reflected light W10") including the component of the excitation light reflected by the biological tissue O10 (thermal treatment area) and the returning light is blocked by the cut filter 203 and the intensity is reduced, on the other hand, a part of the component of the waveband on the long wavelength side from the waveband of which the majority is blocked is not reduced in intensity and is incident to the imaging element 204.

[0115] More specifically, as shown in the graph G12 of Figure 6 , the cut filter 203 blocks the majority of the reflected light W10 of the waveband of the short wavelength of the waveband including the excitation light that is incident to the G pixel and transmits the waveband on the long wavelength side from the waveband of which the majority is blocked. Further, as shown in the graph G12 of Figure 6 , the cut filter 203 transmits the fluorescence (WF10) emitted from the AGEs themselves in the biological tissue O10 (thermal treatment area). Thus, the reflected light W10 and the fluorescence (WF10) of which the intensity is reduced are incident to the R pixel, the G pixel, and the B pixel, respectively.

[0116] Further, as shown by the broken line L NG of the fluorescence characteristics in the graph G12 of Figure 6 , the G pixel has sensitivity to the fluorescence, but the output value becomes a small value due to the slight reaction of the fluorescence.

[0117] Thereafter, the image processing section 406 acquires an imaging signal (RAW data) from the imaging element 204 of the endoscope 2, and generates a fluorescence image by performing image processing on the signal values of the G pixels and the B pixels included in the acquired imaging signal, respectively. In this case, the signal value of the G pixel includes fluorescence information emitted from the thermal treatment region. In addition, the B pixel includes background information from the biological tissue of the subject including the thermal treatment region. In this case, the image processing section 406 performs demosaicing processing, processing of calculating the intensity ratio of each pixel, processing of determining a fluorescence region and a background region, and image processing of the color component signals (pixel values) of the pixels located in the fluorescence region and the color component signals (pixel values) of the pixels located in the background region, respectively, with different parameters, and generates a fluorescence image. Then, the image processing section 406 outputs the fluorescence image to the display device 3. Here, the fluorescence region refers to a region in which fluorescence information is dominant compared to background information. In addition, the background region refers to a region in which background information is dominant compared to fluorescence information. Specifically, the extraction section 406b in the image processing section 93 determines that it is a fluorescence region when the intensity ratio of the reflected light component signal corresponding to the background information and the fluorescence component signal corresponding to the fluorescence information included in the pixel is equal to or greater than a predetermined threshold value (for example, 0.5 or more), and determines that it is a background region when the intensity ratio is less than the predetermined threshold value, thereby extracting the fluorescence region and the background region.

[0118] Thus, the fluorescence observation mode (thermal treatment observation mode) enables easy observation of the biological tissue (thermal treatment region) that has been thermally treated by the laser irradiation device 5.

[0119] Outline of the general light observation mode

[0120] Next, the general light observation mode that the endoscope system 1 can perform will be described. Figure 7 is a view that schematically shows the observation principle when the general light observation mode is performed.

[0121] As shown in Figure 7 , first, the control device 4 irradiates the biological tissue O10 of the subject with white light W3 by causing the first light source section 402 to emit light. In this case, part of the reflected light and the return light (hereinafter, simply referred to as "reflected light WR30, reflected light WG30, reflected light WB30") reflected in the biological tissue O10 is blocked by the cut filter 203, and the remaining part is incident on the imaging element 204. Specifically, as shown in Figure 7 , the cut filter 203 blocks the reflected light of the wavelength band including the narrow-band light. Therefore, as shown in Figure 7 , the component of the light of the blue wavelength band incident on the B pixel is smaller than in the state where the cut filter 203 is not disposed.

[0122] Next, the image processing unit 406 acquires an imaging signal (RAW data) from the imaging element 204, performs image processing on the signal values of the R pixels, G pixels, and B pixels included in the acquired imaging signal, respectively, and generates an observation image (white light image) as a display image. In this case, since the blue component included in the imaging signal is smaller than in the past at the time of white light observation, the image processing unit 406 performs white balance adjustment processing that adjusts the white balance so that the ratio of the red component, the green component, and the blue component is constant.

[0123] Thus, in the normal light observation mode, even in the case where the cut filter 203 is provided, a natural observation image (white image) can be observed.

[0124] Transurethral urinary stone removal procedure using an endoscope system

[0125] Next, a surgical procedure in which the operator performs a transurethral urinary stone removal procedure (f-TUL) using the endoscope system 1 will be described. Figure 8 is a flowchart illustrating a surgical procedure in which the operator performs a transurethral urinary stone removal procedure (f-TUL) using the endoscope system 1. Figures 9A-9F is a view illustrating an example of the transition of the image displayed by the display device 3 in the transurethral urinary stone removal procedure (f-TUL).

[0126] As shown in Figure 8 , first, the operator inserts the insertion section 21 of the endoscope 2 into the urinary tract (ureter) of the subject while irradiating white light (normal light) to the endoscope 2 (step S1). In this case, as shown in Figure 9A , the operator inserts the insertion section 21 of the endoscope 2 into the urinary tract of the subject while observing the observation image P1 based on white light displayed by the display device 3.

[0127] Next, the operator performs confirmation of the stone generated in the subject while observing the observation image P1 displayed by the display device 3 (step S2). In this case, as shown in Figure 9B , the operator inserts the insertion section 21 of the endoscope 2 into the urinary tract of the subject while observing the observation image P2 displayed by the display device 3, explores the stone K1 while observing the observation image P2, and confirms the size and position of the stone K1.

[0128] After that, the operator irradiates laser light toward the stone by inserting the laser irradiation device 5 into the ureter of the subject via the treatment instrument insertion section 222 of the endoscope 2 while observing the observation image displayed by the display device 3 (step S3). In this case, as shown in Figure 9CAs shown, the operator observes the observation image P3 displayed by the display device 3, and irradiates laser light toward the stone K1 by the laser irradiation device 5, thereby breaking the stone K1.

[0129] Next, the operator takes out the broken stone from the subject using the basket via the treatment instrument insertion portion 222 of the endoscope 2 while observing the observation image displayed by the display device 3 (step S4). In this case, as shown in Figure 9D and Figure 9E As shown, the operator takes out the broken stone K1 from the subject using the graspable treatment instrument, such as the basket catheter K2, via the treatment instrument insertion portion 222 of the endoscope 2 while observing the observation image P4 or the observation image P5 displayed by the display device 3.

[0130] Then, the operator operates the operation portion of the endoscope 2 to switch the observation mode of the endoscope 2 from the normal light observation mode to the fluorescence observation mode (heat treatment observation mode) (step S5). In this case, the control device 4 irradiates excitation light toward the subject by causing the second light source portion 403 to emit light. At this time, as shown in Figure 9F the operator grasps the invasiveness of the heat damage caused by the laser irradiation device 5 by observing the fluorescence region Q1 included in the fluorescence image P6 displayed in the display device 3.

[0131] Next, the operator alternately switches the observation mode of the endoscope 2 between the fluorescence observation mode and the normal light observation mode while operating the operation portion 22 of the endoscope 2, and grasps the invasiveness of the heat damage of the surrounding tissue by the laser irradiation device 5 (step S6).

[0132] Thereafter, the operator decides the indwelling of the stent and the indwelling period while referring to whether the stent is indwelled and the indwelling period displayed by the display device 3 (step S7). In this case, the control device 4 outputs whether the stent is indwelled in the urinary tract and the indwelling period information related to the indwelling period on the observation image displayed by the display device 3 in accordance with the emission intensity of the emission region included in the fluorescence image P6. Thereby, the operator decides whether the stent is indwelled in the urinary tract and the indwelling period with reference to the indwelling period information displayed by the display device 3. In addition, the estimation method of the control device 4 on whether the stent is indwelled in the urinary tract and the indwelling period displayed in the display device 3 is described later.

[0133] Next, the operator indwells the stent in the urinary tract in the case where the stent is to be indwelled in the urinary tract (step S8). Then, the operator pulls out the endoscope 2 from the urinary tract of the subject to end the surgery.

[0134] Thus, the operator breaks up a calculus located in a urinary tract of an object by laser light, removes the broken-up calculus from the object by a treatment instrument or the like, and then switches the observation mode of the endoscope system 1 from the normal light observation mode to the fluorescence observation mode, and determines whether or not to leave a stent and a period of leaving by grasping the degree of invasion of the laser light into the biological tissue of the object.

[0135] Processing of endoscope system

[0136] Next, the processing performed by the endoscope system 1 will be described.

[0137] Figure 10 is a flowchart showing an outline of the processing performed by the endoscope system 1.

[0138] As shown in Figure 10 first, the control section 409 causes the first light source section 402 to emit light by controlling the light source control section 404, and thereby irradiates white light to the object (Step S101).

[0139] Next, the image processing section 406 acquires the imaging signal from the imaging element 204 of the endoscope 2, generates an observation image as a display image, and outputs it to the display device 3 (Step S102).

[0140] Then, the control section 409 determines whether or not a change signal to change the observation mode to the fluorescence observation mode is input from the input section 407 or the operation section 22 of the endoscope 2 (Step S103). In the case where it is determined by the control section 409 that the change signal to change the observation mode to the fluorescence observation mode is input from the input section 407 or the operation section 22 of the endoscope 2 (Step S103: Yes), the endoscope system 1 shifts to Step S104 described later. In contrast, in the case where it is determined by the control section 409 that the change signal to change the observation mode to the fluorescence observation mode is not input from the input section 407 or the operation section 22 of the endoscope 2 (Step S103: No), the endoscope system 1 shifts to Step S120 described later.

[0141] In Step S104, the control section 409 causes the second light source section 403 to irradiate excitation light by controlling the light source control section 404.

[0142] Next, the image processing section 406 generates a fluorescence image from the imaging signal generated by the imaging element 204 of the endoscope 2 (Step S105).

[0143] Then, the extraction section 406b extracts a fluorescence region included in the fluorescence image generated by the generation section 406a (Step S106). Specifically, the extraction section 406b extracts the fluorescence region by performing a 2-value processing or the like on the fluorescence image. In addition, in the case where a plurality of fluorescence regions are included in the fluorescence image, the extraction section 406b extracts the plurality of fluorescence regions.

[0144] Next, the calculation section 406c calculates the fluorescence intensity of the fluorescence region extracted by the extraction section 406b (step S107). In this case, the calculation section 406c calculates the fluorescence intensity of each of the plurality of fluorescence regions in the case where the extraction section 406b extracts a plurality of fluorescence regions.

[0145] Next, the estimation section 406d determines whether there are a plurality of fluorescence regions (step S108). In the case where the estimation section 406d determines that there are a plurality of fluorescence regions (step S108: YES), the endoscope system 1 shifts to step S109 described later. In contrast, in the case where the estimation section 406d determines that the fluorescence region is not a plurality (step S108: NO), the endoscope system 1 shifts to step S110 described later.

[0146] In step S109, the estimation section 406d estimates whether a stent is indwelled in the urinary tract on the basis of the correlation information recorded by the correlation information recording section 408b and the emission intensity of the strongest of the plurality of fluorescence regions calculated by the calculation section 406c. Specifically, as shown in FIG. 10, the estimation section 406d estimates whether a stent is indwelled in the urinary tract on the basis of the correlation information recorded by the correlation information recording section 408b and the emission intensity calculated by the calculation section 406c. Figure 11 Figure 11 As shown in FIG. 10, the estimation section 406d determines whether the emission intensity is less than a threshold value TL1 that represents a value indicating that the stent does not need to be indwelled, on the basis of the correlation information recorded by the correlation information recording section 408b and the emission intensity calculated by the calculation section 406c, and estimates that the stent does not need to be indwelled in the urinary tract in the case where the emission intensity is less than the threshold value TL1. In contrast, the estimation section 406d estimates on the basis of the correlation information recorded by the correlation information recording section 408b and the emission intensity calculated by the calculation section 406c, and estimates that the stent needs to be indwelled in the urinary tract in the case where the emission intensity is the threshold value TL1 or more. After step S109, the endoscope system 1 shifts to step S111 described later.

[0147] In step S110, the estimation section 406d estimates whether a stent is indwelled on the basis of the emission intensity calculated by the calculation section 406c and the correlation information recorded by the correlation information recording section 408b. After step S110, the endoscope system 1 shifts to step S111 described later.

[0148] In step S111, in the case where the estimation section 406d estimates that a stent is indwelled in the urinary tract (step S111: YES), the endoscope system 1 shifts to step S112 described later. In contrast, in the case where the estimation section 406d estimates that a stent is not indwelled in the urinary tract (step S111: NO), the endoscope system 1 shifts to step S114 described later.

[0149] ​In step S112, the estimation unit 406d estimates the indwelling period of the stent in the urinary tract based on the luminescence intensity calculated by the calculation unit 406c and the relevant information recorded by the relevant information recording unit 408b. Specifically, as Figure 12 As shown, the estimation unit 406d estimates the indwelling period of the stent in the urinary tract based on the luminous intensity calculated by the calculation unit 406c and the relevant information recorded by the relevant information recording unit 408b. For example, as Figure 12 As shown, the estimation unit 406d estimates the luminescence intensity calculated by the calculation unit 406c and the relevant information recorded by the relevant information recording unit 408b. When the correlation between luminescence intensity and invasiveness is in the first region Z1 (low), the stent's retention period is estimated as short; when the correlation is in the second region Z2 (medium), the stent's retention period is estimated as normal; and when the correlation is in the third region Z3 (high), the stent's retention period is estimated as long. Here, "short" means about a few days, "normal" means about one week, and "long" means more than 10 days.

[0150] Next, the output unit 406e outputs dwell time information to the display device 3 (step S113). Specifically, as... Figure 13 As shown, the output unit 406e outputs retention period information M1 and observation image P10 as a display image to the display device 3. The retention period information M1 includes maximum invasiveness m1 (maximum invasive depth based on fluorescence intensity), whether a stent is placed m2, and the recommended retention period of the stent m3.

[0151] Then, the control unit 409 determines whether a change signal to change the observation mode to the normal light observation mode has been input from the input unit 407 or the operation unit 22 of the endoscope 2 (step S114). If the control unit 409 determines that a change signal to change the observation mode to the normal light observation mode has been input from the input unit 407 or the operation unit 22 of the endoscope 2 (step S114: Yes), the endoscope system 1 proceeds to step S115, which will be described later. Conversely, if the control unit 409 determines that no change signal to change the observation mode to the normal light observation mode has been input from the input unit 407 or the operation unit 22 of the endoscope 2 (step S114: No), the endoscope system 1 returns to step S104 as described above.

[0152] In step S115, the control unit 409 controls the light source control unit 404 to make the first light source unit 402 emit light, thereby illuminating white light.

[0153] Next, the image processing section 406 generates an observation image from the imaging signal acquired from the imaging element 204 of the endoscope 2 and outputs the observation image to the display device 3 (step S116). Specifically, the generating section 406a acquires the imaging signal from the imaging element 204 and generates an observation image from the imaging signal.

[0154] Then, the control section 409 determines whether the laser has invaded the biological tissue by the estimation section 406d (step S117). Specifically, the control section 409 determines whether the stent is estimated to be left in the urinary tract by the estimation section 406d, and determines that the laser has invaded the biological tissue in a case where the estimation section 406d estimates that the stent is left in the urinary tract. In a case where the control section 409 determines that the laser has invaded the biological tissue (step S117: YES), the endoscope system 1 shifts to step S118 described later. In contrast, in a case where the control section 409 determines that there is no invasion of the biological tissue by the laser (step S117: NO), the endoscope system 1 shifts to step S119 described later.

[0155] In step S118, the output section 406e outputs the observation image generated by the generating section 406a to the display device 3 with the stent leaving information indicating whether the stent is estimated to be left and the leaving period information indicating the leaving period superimposed thereon. After step S118, the endoscope system 1 shifts to step S120 described later.

[0156] In step S119, the output section 406e outputs the observation image generated by the generating section 406a to the display device 3. After step S119, the endoscope system 1 shifts to step S120 described later.

[0157] In step S120, the control section 409 determines whether an end signal to end the observation of the subject is input from the input section 407 or the operation section 22 of the endoscope 2. In a case where the control section 409 determines that the end signal to end the observation of the subject is input from the input section 407 or the operation section 22 of the endoscope 2 (step S120: YES), the endoscope system 1 ends the present processing. In contrast, in a case where the control section 409 determines that the end signal to end the observation of the subject is not input from the input section 407 or the operation section 22 of the endoscope 2 (step S120: NO), the endoscope system 1 returns to step S101 described above.

[0158] According to one embodiment described above, the estimation section 406d estimates the indwelling period of the medical instrument to be indwelled in the lumen based on the fluorescence intensity calculated by the calculation section 406c, and the output section 406e outputs the observation image obtained by imaging the biological tissue and the indwelling period information on the indwelling period estimated by the estimation section 406d to the display device 3, so that the indwelling period when the medical instrument is to be indwelled in the lumen can be objectively grasped.

[0159] In addition, according to one embodiment, the estimation section 406d estimates the invasiveness of the energy device to the biological tissue based on the fluorescence intensity calculated by the calculation section 406c, estimates the indwelling period based on the invasiveness, so that the indwelling period when the medical instrument is to be indwelled in the lumen can be objectively grasped.

[0160] In addition, according to one embodiment, the estimation section 406d estimates the invasiveness based on the correlation information recorded by the correlation information recording section 408b and the fluorescence intensity calculated by the calculation section 406c, so that the actual invasiveness to the biological tissue can be estimated.

[0161] In addition, according to one embodiment, the estimation section 406d estimates whether or not the medical instrument is to be indwelled in the lumen based on the fluorescence intensity calculated by the calculation section 406c, so that the user such as the surgeon can objectively grasp whether or not the medical instrument needs to be indwelled in the lumen.

[0162] In addition, according to one embodiment, in a case where a plurality of fluorescence regions are extracted by the extraction section 406b, the estimation section 406d estimates the indwelling period of the medical instrument to be indwelled in the lumen based on the strongest fluorescence intensity among the plurality of fluorescence regions, so that assistance can be performed with the most appropriate information in the situation of a series of treatments.

[0163] In addition, in one embodiment, the output section 406e outputs the observation image and the indwelling period information to the display device 3, but for example, the indwelling period information can be superimposed on the observation image and output to the display device 3.

[0164] In addition, in one embodiment, the output section 406e outputs the indwelling period information to the display device 3, but is not limited thereto, and for example, in a case where the estimation section 406d estimates the invasiveness of the laser irradiation device 5 to the biological tissue, a case where the invasiveness to the biological tissue is detected can also be output. Specifically, as shown in Figure 14 in a case where the estimation section 406d estimates the invasiveness of the laser irradiation device 5 to the biological tissue, the output section 406e can also output the invasiveness information M10 indicating that the invasiveness to the biological tissue is detected to the display device 3. Thereby, the surgeon can grasp the invasiveness of the laser irradiation device 5 to the biological tissue.

[0165] Further, in one embodiment, the output section 406e outputs the indwelling period information to the display device 3, but is not limited thereto, and for example, in a case where the calculation section 406c calculates the emission intensity of each of the plurality of fluorescent regions, the invasiveness of each of the fluorescent regions can be output to the display device 3 in a recognizable manner. Specifically, as shown in FIG. 15, the output section 406e can superimpose the observation image P30 in a display manner corresponding to the emission intensity of each of the plurality of fluorescent regions Q11 to Q13 calculated by the calculation section 406c, and output the depth information M20 related to the depth of invasion and the indwelling period information M1 to the display device 3. In this case, the estimation section 406d estimates whether or not the stent is indwelled and the indwelling period based on the fact that the emission intensity of the fluorescent region Q12 closer to the point is higher than the emission intensity of the fluorescent regions Q11, Q13 farther from the point among the plurality of fluorescent regions Q11 to Q13 in the region H1 shown in FIG. 15. Here, as a method of determining the far point side and the near point side, the estimation section 406d determines whether or not the luminance information of each pixel of the observation image (white light image) is equal to or greater than a predetermined threshold value, estimates the pixel equal to or greater than the predetermined threshold value as the near point side, and estimates the pixel less than the predetermined threshold value as the far point side, thereby estimating the far point side and the near point side. Further, the output section 406e can output only the meaning that the invasiveness of the energy device to the biological tissue has occurred to the display device 3 in a case where the extraction section 406b detects the fluorescent region only on the far point side. Furthermore, the calculation section 406c can calculate the fluorescent intensity by performing amplification processing based on a gain or the like on the signal value of the pixel located in the fluorescent region in a case where the extraction section 406b detects the fluorescent region only on the far point side. Figure 15 Figure 16 Further, in one embodiment, the output section 406e outputs the indwelling period information to the display device 3, but is not limited thereto, and for example, in a case where the calculation section 406c calculates the emission intensity of each of the plurality of fluorescent regions, the invasiveness of each of the fluorescent regions can be output to the display device 3 in a recognizable manner. Specifically, as shown in FIG. 15, the output section 406e can superimpose the observation image P30 in a display manner corresponding to the emission intensity of each of the plurality of fluorescent regions Q11 to Q13 calculated by the calculation section 406c, and output the depth information M20 related to the depth of invasion and the indwelling period information M1 to the display device 3. In this case, the estimation section 406d estimates whether or not the stent is indwelled and the indwelling period based on the fact that the emission intensity of the fluorescent region Q12 closer to the point is higher than the emission intensity of the fluorescent regions Q11, Q13 farther from the point among the plurality of fluorescent regions Q11 to Q13 in the region H1 shown in FIG. 15. Here, as a method of determining the far point side and the near point side, the estimation section 406d determines whether or not the luminance information of each pixel of the observation image (white light image) is equal to or greater than a predetermined threshold value, estimates the pixel equal to or greater than the predetermined threshold value as the near point side, and estimates the pixel less than the predetermined threshold value as the far point side, thereby estimating the far point side and the near point side. Further, the output section 406e can output only the meaning that the invasiveness of the energy device to the biological tissue has occurred to the display device 3 in a case where the extraction section 406b detects the fluorescent region only on the far point side. Furthermore, the calculation section 406c can calculate the fluorescent intensity by performing amplification processing based on a gain or the like on the signal value of the pixel located in the fluorescent region in a case where the extraction section 406b detects the fluorescent region only on the far point side.

[0166] Further, in one embodiment, the operator switches the observation mode by operating the input section 407 or the operation section 22, and switches the observation image and the fluorescent image, but can acquire the fluorescent image by automatically switching the irradiation excitation light every predetermined frame (for example, 10 fps) at a predetermined frame rate (for example, 60 fps). In this case, the output section 406e can output the observation image to the display device 3, and output the invasiveness information M10 indicating that the invasiveness to the biological tissue is detected to the display device 3 only in a case where the invasiveness of the energy device is estimated by the estimation section 406d.

[0167] Further, in one embodiment, the estimation section 406d performs the estimation of whether or not the stent is indwelled and the estimation of the indwelling period of the stent in the urinary tract, but is not limited thereto, and for example, the estimation section 406d can perform only either of the estimation of whether or not the stent is indwelled and the estimation of the indwelling period of the stent in the urinary tract. Of course, the estimation section 406d can perform only either of the estimation of whether or not the stent is indwelled and the estimation of the indwelling period of the stent in the urinary tract according to the operation of the operator to the input section 407 or the operation section 22.​

[0168] (Variation Example 1)

[0169] In another embodiment, the transmission characteristics of the cutoff filter 203 can also be modified. Figure 17 This is a diagram schematically illustrating the transmission characteristics of a cutoff filter according to a modified embodiment. Figure 17 In the diagram, the horizontal axis represents wavelength (nm), and the vertical axis represents transmission characteristics. Additionally, in... Figure 17 In the middle, the broken line L FF The broken line L represents the transmission characteristics of cutoff filter 203A. V The broken line L represents the wavelength characteristics of the excitation light. NG The wavelength characteristics of fluorescence generated by irradiating the glycosylation end product with excitation light, which is produced by heat treatment of biological tissue by an energy device, such as a laser irradiation device 5.

[0170] like Figure 17 The broken line L V and broken line L NG As shown, the cutoff filter 203A allows a portion of the excitation light reflected from the biological tissue in the observation area to pass through, and allows only the fluorescent component to pass through. Specifically, the cutoff filter 203A blocks light in the short wavelength band (greater than or equal to 400 nm and less than 430 nm) containing the excitation light, and allows light in the long wavelength band (greater than 430 nm) containing fluorescence to pass through, the fluorescence being generated by irradiating the glycosylation end product produced by heat treatment with the excitation light.

[0171] (Other implementation methods)

[0172] Various inventions can be formed by appropriately combining the multiple structural elements disclosed in the endoscope system of the above-described embodiment. For example, several structural elements may be deleted from all the structural elements described in the endoscope system of the above-described embodiment. Furthermore, the structural elements described in the endoscope system of the above-described embodiment may be appropriately combined.

[0173] In another embodiment, the first light source unit, the second light source unit, and the light source control unit are integrally provided in the control device, but it is not limited to this. For example, the light source device and the control device may be provided separately, and the light source device includes the first light source unit, the second light source unit, and the light source control unit.

[0174] In another embodiment, an endoscope system using a flexible endoscope is used, but it is not limited to this. It can also be applied to an endoscope system using a rigid endoscope, a medical surgical robot using multiple rigid endoscopes and a laser irradiation device, or a medical observation system.

[0175] Also, in the endoscope system of one embodiment, the "section" described above can be replaced with "unit" or "circuit" or the like. For example, the control section can be replaced with a control unit or a control circuit.

[0176] In the description of the flowchart in this specification, expressions such as "first", "then", "next", and the like are used to clearly indicate the order of processing between steps, but the order of processing required to implement the present application is not uniquely determined by these expressions. That is, the order of processing in the flowchart described in this specification can be changed within a range where no contradiction arises.

[0177] The above describes several embodiments of the present application in detail based on the drawings, but these are examples, and the present application can be implemented in other ways in which modifications and the like are made based on the knowledge of a person skilled in the art, starting from the manner described in the present disclosure.

[0178] Further, the present disclosure can also take the following surgical method. (1)

[0180] A surgical method of transurethral removal of urinary tract stones using an endoscope system includes the steps of:

[0181] An insertion step of inserting an endoscope into a urinary tract of an object by white light observation;

[0182] A fragmentation step of fragmenting a stone located in the urinary tract by irradiating laser light;

[0183] A removal step of removing the stone fragmented by the laser light from the urinary tract;

[0184] A switching step of switching the observation method of the endoscope to fluorescence observation;

[0185] An observation step of observing the object by fluorescence observation; and

[0186] A leaving step of leaving a stent after observation by the fluorescence observation.

[0187] Label Explanation

[0188] 1 Endoscope system

[0189] 2 Endoscope

[0190] 3 Display device

[0191] 4 Control device

[0192] 5 Laser irradiation device

[0193] 7 Display device

[0194] 21 Insertion section

[0195] 22 operation section

[0196] 23 general-purpose cord

[0197] 24 front end portion

[0198] 25 bending portion

[0199] 26 flexible tube portion

[0200] 27 connector portion

[0201] 27a spiral cable

[0202] 28 connector portion

[0203] 201 illumination optical system

[0204] 202 imaging optical system

[0205] 203, 203A cut filter

[0206] 204 imaging element

[0207] 205 A / D conversion section

[0208] 206 P / S conversion section

[0209] 207 imaging recording section

[0210] 208 imaging control section

[0211] 221 bending knob

[0212] 222 treatment instrument insertion portion

[0213] 223 switch

[0214] 231 light guide

[0215] 232 first transmission cable

[0216] 233 second transmission cable

[0217] 401 condenser lens

[0218] 402 first light source portion

[0219] 403 second light source portion

[0220] 404 light source control section

[0221] 405 S / P conversion section

[0222] 406 image processing section

[0223] 406a generation section

[0224] 406b extraction section

[0225] 406c calculation section

[0226] 406d estimation section

[0227] 406e output section

[0228] 407 input section

[0229] 408 recording section

[0230] 408a program recording section

[0231] 408b related information recording section

[0232] 409 control section

[0233] K1 calculus

[0234] K2 basket catheter

[0235] M1 indwelling period information

[0236] P1 to P5, P10 observation image

[0237] P6 fluorescent image

Claims

1. An auxiliary device comprising: The generation unit generates fluorescence images based on camera signals, which are generated by photographing fluorescence produced by excitation light irradiated onto biological tissues; The computing unit calculates the fluorescence intensity based on the fluorescence image; The estimation unit estimates the indwelling period of the medical device to be placed in the lumen based on the fluorescence intensity. as well as The output unit outputs retention period information related to the retention period and observation images obtained by photographing the biological tissue. The estimation unit estimates the degree of invasiveness of the energy device to the biological tissue based on the fluorescence intensity, and estimates the retention period based on the degree of invasiveness.

2. The auxiliary device according to claim 1, wherein, The estimation unit estimates the degree of invasion based on pre-determined relevant information representing the correlation between the degree of invasion and the fluorescence intensity, and the fluorescence intensity.

3. The auxiliary device according to claim 1, wherein, The estimation unit estimates, based on the fluorescence intensity, whether the medical device should be left inside the lumen.

4. The auxiliary device according to any one of claims 1 to 3, wherein, The output unit superimposes the retention period information onto the observed image and outputs it.

5. The auxiliary device according to claim 1, wherein, The auxiliary device also includes an extraction unit for extracting fluorescent regions from the fluorescent image. When the extraction unit extracts multiple fluorescent regions, the estimation unit estimates the retention period based on the strongest fluorescence intensity.

6. The auxiliary device according to claim 5, wherein, The imaging signal is obtained by imaging the urinary tract extending in the depth direction. When the extraction unit extracts multiple fluorescent regions, the estimation unit estimates the retention period based on the fluorescence intensity of the fluorescent region located on the near-point side closest to the imaging optical system.

7. The auxiliary device according to claim 5 or 6, wherein, When the extraction unit extracts multiple fluorescent regions, the output unit outputs each of the multiple fluorescent regions in a recognizable manner based on the fluorescence intensity of each of the multiple fluorescent regions.

8. The auxiliary device according to any one of claims 1 to 3, wherein, The medical device is any one of a stent, catheter, or indwelling needle.

9. The auxiliary device according to any one of claims 1 to 3, wherein, The lumen is the urinary tract.

10. The auxiliary device according to any one of claims 1 to 3, wherein, The wavelength range of the excitation light is 390nm to 430nm. The fluorescence wavelength range is 500nm to 640nm. The image signal is obtained by capturing the transmitted light after it passes through the cutoff filter, which blocks the light on the shorter wavelength side of 430nm.

11. An endoscope system comprising: An endoscope is an instrument that can be inserted into the lumen of a patient being examined. A light source device capable of irradiating excitation light that excites glycosylation end products produced by heat treatment of biological tissues; as well as The endoscope is detachable from the control device. The endoscope has the following features: An imaging element capable of generating an imaging signal by capturing an image of fluorescence emitted using the excitation light; and A cutoff filter is disposed on the light-receiving surface side of the imaging element to block light from the shorter wavelength side, which includes a portion of the excitation light's wavelength range. The control device includes auxiliary devices to assist the surgeon. The auxiliary device has: The generation unit generates a fluorescence image based on the camera signal; The computing unit calculates the fluorescence intensity based on the fluorescence image; The estimation unit estimates the indwelling period of the medical device to be placed in the lumen based on the fluorescence intensity. as well as The output unit outputs retention period information related to the retention period and observation images obtained by photographing the biological tissue. The estimation unit estimates the degree of invasiveness of the energy device to the biological tissue based on the fluorescence intensity, and estimates the retention period based on the degree of invasiveness.

12. A storage medium storing a program that causes an auxiliary device to execute, wherein, This program causes the following steps to be performed: The generation step involves generating a fluorescence image based on a camera signal, which is generated by photographing the fluorescence produced using excitation light irradiated onto biological tissue. The calculation step involves calculating the fluorescence intensity based on the fluorescence image. The estimation step involves estimating the indwelling period of the medical device to be placed in the lumen based on the fluorescence intensity. as well as The output step outputs retention period information related to the retention period and observation images obtained by photographing the biological tissue. The estimation step estimates the degree of invasiveness of the energy device to the biological tissue based on the fluorescence intensity, and estimates the retention period based on the degree of invasiveness.

Images