Image processing device, light treatment system, image processing method, image processing program, and light treatment method

Abstract

This image processing device is provided with: a white light image generation unit that generates a white light image based on white light comprising light in the wavelength band of the visible light region; an absorption band light image generation unit that generates an absorption band light image based on absorption band light comprising light in the wavelength band of the Soley band absorbed by the drug collected in the treatment target; and a display image generation unit that generates a display image on the basis of the white light image and the absorption band light image.

Description

Technical Field

[0001] The present invention relates to an image processing device, a phototherapy system, an image processing method, an image processing program, and a phototherapy method. Background Art

[0002] In recent years, research on photoimmunotherapy (PIT) has been conducted, which allows antibody drugs to specifically bind to the proteins of cancer cells, and activates the antibody drugs by irradiating near-infrared light as therapeutic light to destroy cancer cells, thereby treating cancer (for example, refer to Patent Documents 1 to Patent Document 3 and Non-Patent Documents 1 and 2). The antibody drug irradiated with near-infrared light absorbs light energy and vibrates its molecules, generating heat. The heat destroys cancer cells. At this time, the antibody drug is excited and emits fluorescence. The intensity of the fluorescence is used as an indicator of the therapeutic effect. For example, Patent Documents 2 and 3 record the following: The antibody drug is illuminated by light in the same wavelength band as the therapeutic light, thereby confirming the aggregation of the drug and the course of treatment.

[0003] Prior art literature

[0004] Patent Literature

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2017-71654

[0006] [Patent Document 2] International Publication No. 2019 / 215905

[0007] [Patent Document 3] International Publication No. 2021 / 038913

[0008] Non-patent literature

[0009] [Non-patent document 1] K. Sato et al, ACS Cent. Sci. 2018, 4, 1559-1569.

[0010] [Non-patent document 2] F. Inagaki, et al, Cancer Science 2021;112:1326-1330. Summary of the invention

[0011] Problems to be solved by the invention

[0012] In PIT, excessive irradiation of therapeutic light may cause cell damage. On the other hand, in PIT, even a small amount of therapeutic light will cause a reaction, so it is required to control the suppression of excessive irradiation of therapeutic light (for example, refer to non-patent document 2). However, in Patent Documents 2 and 3, in order to confirm the progress of treatment by irradiating light of the same wavelength band as the therapeutic light, there is a possibility that the light used for treatment confirmation will further cause a reaction.

[0013] The present invention is completed in view of the above situation, and its purpose is to provide an image processing device, phototherapy system, image processing method, image processing program and phototherapy method that can inhibit the progress of the reaction in photoimmunotherapy and confirm the aggregation of drugs and the course of treatment.

[0014] Means for solving problems

[0015] In order to solve the above-mentioned problems and achieve the purpose, the image processing device of the present invention comprises: a white light image generating unit, which generates a white light image based on white light, wherein the white light is composed of light in a wavelength band of the visible light domain; an absorption band light image generating unit, which generates an absorption band light image based on the absorption band light, wherein the absorption band light is composed of light in a wavelength band of the Soret band absorbed by a drug concentrated on a treatment object; and a display image generating unit, which generates a display image based on the white light image and the absorption band light image.

[0016] In addition, the image processing device of the present invention in the above invention further includes a standardization unit, which divides a blue signal based on light in a blue band by a green signal based on light in a green band to standardize the signal value of the blue signal, and the absorption band light image generation unit uses the standardized signal value of the blue signal to generate the absorption band light image.

[0017] In addition, the image processing device of the present invention in the above invention also has a standardization unit, which divides the blue signal based on the blue band light by the sum of the signal values ​​of the blue signal, the red signal based on the red band light and the green signal based on the green band light, and standardizes the signal value of the blue signal, and the absorption band light image generation unit uses the standardized signal value of the blue signal to generate the absorption band light image.

[0018] In addition, the image processing device of the present invention in the above invention also has a standardization unit, which divides the blue signal based on the blue band light by the sum of the signal values ​​of the red signal based on the red band light and the green signal based on the green band light, and standardizes the signal value of the blue signal. The absorption band light image generating unit uses the standardized signal value of the blue signal to generate the absorption band light image.

[0019] In addition, the image processing device of the present invention in the above invention also has a control unit, which switches between a white light observation mode and an absorption band light observation mode. The white light observation mode is a mode in which the display device displays the white light image, and the absorption band light observation mode is a mode in which the display device displays a display image including the absorption band light image.

[0020] Furthermore, in the image processing device of the present invention in the above invention, the wavelength band of the Soret band is a wavelength band below 450 nm, and the wavelength band for exciting the drug is a wavelength band above 680 nm.

[0021] In addition, the phototherapy system of the present invention comprises: a white light emitting unit, which emits white light composed of light in a wavelength band in the visible light domain; an absorption band light emitting unit, which emits absorption band light, wherein the absorption band light is composed of light in a wavelength band of the Soret band absorbed by a drug focused on a treatment object; an image processing device, which generates an image based on the white light or the absorption band light; and a display device, which displays the image generated by the image processing device, the image processing device comprising: a white light image generating unit, which generates a white light image based on the white light; an absorption band light image generating unit, which generates an absorption band light image based on the absorption band light; and a display image generating unit, which generates a display image based on the white light image and the absorption band light image.

[0022] In addition, the image processing method of the present invention includes: a white light image generating step, in which a white light image generating unit generates a white light image based on white light, wherein the white light is composed of light in a wavelength band of the visible light domain; an absorption band light image generating step, in which an absorption band light image generating unit generates an absorption band light image based on the absorption band light, wherein the absorption band light is composed of light in a wavelength band of the Soret band absorbed by a drug focused on a treatment object; and a display image generating step, in which a display image generating unit generates a display image based on the white light image and the absorption band light image.

[0023] In addition, the image processing program of the present invention enables the computer to perform the following steps: a white light image generating step, generating a white light image based on white light, wherein the white light is composed of light in a wavelength band of the visible light domain; an absorption band light image generating step, generating an absorption band light image based on the absorption band light, wherein the absorption band light is composed of light in a wavelength band of the Soret band absorbed by the agent concentrated on the treatment object; and a display image generating step, generating a display image based on the white light image and the absorption band light image.

[0024] In addition, the phototherapy method of the present invention includes the following steps: administering a drug for phototherapy to a treatment target area; irradiating the treatment target area with absorption band light to obtain an absorption band light image, wherein the absorption band light is light in a band that includes a Soret band and is smaller than an excitation wavelength of the drug; generating a display image including the absorption band light image; displaying the display image; observing the absorption band light image to confirm the aggregation amount of the drug; and irradiating the treatment target area with therapeutic light to cause the drug bound to the treatment target area to react.

[0025] Effects of the Invention

[0026] According to the present invention, the effect of being able to suppress the progress of the reaction in photoimmunotherapy while confirming the accumulation of the drug and the progress of the treatment is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to an embodiment of the present invention.

[0028] Figure 2 This is a block diagram showing a schematic configuration of an endoscope system according to an embodiment of the present invention.

[0029] Figure 3 It is a diagram for explaining the distal end structure of an endoscope according to one embodiment of the present invention.

[0030] Figure 4 It is a diagram for explaining the structure of the optical system of the endoscope.

[0031] Figure 5 A diagram schematically showing a structure of a pixel portion according to an embodiment.

[0032] Figure 6 It is a diagram schematically showing the structure of a color filter according to an embodiment.

[0033] Figure 7 It is a diagram schematically showing the sensitivity characteristics of each filter.

[0034] Fig. 8A FIG. 1 is a diagram schematically showing the signal value of the R pixel of the image sensor according to the embodiment.

[0035] Figure 8B FIG. 1 is a diagram schematically showing the signal value of the G pixel of the image sensor according to the embodiment.

[0036] Figure 8C FIG. 1 is a diagram schematically showing the signal value of the B pixel of the image sensor according to the embodiment.

[0037] Fig. 9 This is a diagram for explaining an example of wavelength bands of light used as therapeutic light and absorption band light.

[0038] Fig.10 This is a diagram showing an example of a treatment flow using an endoscope according to an embodiment of the present invention.

[0039] Fig.11 This is a flowchart showing an example of processing of the endoscope system according to one embodiment of the present invention.

[0040] Fig.12 This is a diagram for explaining an example of a display image during treatment.

[0041] Fig.13 This is a diagram for explaining an example of a display image according to Modification Example 1.

[0042] Fig.14 This is a diagram for explaining an example of a display image according to Modification Example 2.

[0043] Fig.15 This is a block diagram showing a schematic configuration of an endoscope system according to Modification 3. DETAILED DESCRIPTION

[0044] Hereinafter, a method for implementing the present invention (hereinafter referred to as an "embodiment") is described. In the embodiment, as an example of a system including the phototherapy device of the present invention, a medical endoscope system for capturing and displaying images inside a subject such as a patient is described. In addition, the present invention is not limited to this embodiment. Furthermore, in the description of the drawings, the same parts are marked with the same figure numerals for description.

[0045] (Implementation Method)

[0046] Figure 1 This is a diagram showing a schematic configuration of an endoscope system according to an embodiment of the present invention. Figure 2 This is a block diagram showing a schematic configuration of an endoscope system according to the present embodiment. Figure 3 It is a diagram for explaining the distal end structure of the endoscope according to the first embodiment.

[0047] Figure 1 and Figure 2 The endoscope system 1 shown comprises: an endoscope 2, which captures an in-vivo image of a subject by inserting the front end portion into the subject; a light source device 3, which generates illumination light emitted from the front end of the endoscope 2; a processing device 4, which performs prescribed signal processing on the video signal captured by the endoscope 2 and uniformly controls the overall operation of the endoscope system 1; a display device 5, which displays the in-vivo image generated by the signal processing of the processing device 4; and a treatment instrument device 6.

[0048] The endoscope 2 comprises: an insertion portion 21 which is flexible and has an elongated shape; an operating portion 22 which is connected to the base end side of the insertion portion 21 and receives input of various operating signals; and a universal cable 23 which extends from the operating portion 22 in a direction different from the direction in which the insertion portion 21 extends and has various cables connected to the light source device 3 and the processing device 4 built therein.

[0049] The insertion section 21 includes a distal end portion 24 having an imaging element 244 built therein, in which pixels are arranged two-dimensionally for generating signals by receiving light and performing photoelectric conversion, a bendable bending portion 25 composed of a plurality of bending blocks, and a long flexible tube portion 26 connected to the base end side of the bending portion 25 and having flexibility. The insertion section 21 is inserted into the body cavity of the subject, and uses the imaging element 244 to capture images of a subject such as a living tissue located at a position that cannot be reached by external light.

[0050] The operation part 22 has: a bending knob 221, which bends the bending part 25 in the vertical direction and the horizontal direction; a treatment instrument insertion part 222, which inserts treatment instruments such as a therapeutic light irradiation device, a biopsy forceps, an electric scalpel, and an inspection probe into the body cavity of the subject; and a plurality of switches 223 as an operation input part, which input operation instruction signals of peripheral devices such as an air supply unit, a water supply unit, and a screen display control in addition to the operation instruction signals of the processing device 4. The treatment instrument inserted from the treatment instrument insertion part 222 is exposed from the opening through the treatment instrument channel (not shown) of the front end part 24 (refer to Figure 3 ).

[0051] The universal cable 23 has at least a light guide 241 and a collection cable 245 in which one or more signal lines are gathered. The universal cable 23 branches at the end on the side opposite to the side connected to the operation unit 22. A connector 231 that is detachably mounted on the light source device 3 and a connector 232 that is detachably mounted on the processing unit 4 are provided at the branched end of the universal cable 23. A portion of the light guide 241 extends from the end of the connector 231. The universal cable 23 transmits the illumination light emitted from the light source device 3 to the front end portion 24 via the connector 231 (light guide 241), the operation unit 22, and the flexible tube portion 26. In addition, the universal cable 23 transmits the image signal captured by the imaging element 244 provided at the front end portion 24 to the processing unit 4 via the connector 232. The collection cable 245 includes a signal line for transmitting an imaging signal, a signal line for transmitting a drive signal for driving the imaging element 244, and a signal line for transmitting and receiving information including unique information related to the endoscope 2 (imaging element 244). In the present embodiment, an electric signal is transmitted using a signal line for explanation, but an optical signal may be transmitted, or a signal may be transmitted between the endoscope 2 and the processing device 4 by wireless communication.

[0052] The front end portion 24 includes: a light guide 241, which is made of glass fiber or the like and constitutes a light guide path for light emitted by the light source device 3; an illumination lens 242, which is arranged at the front end of the light guide 241; an optical system 243 for focusing light; and an imaging element 244, which is arranged at the imaging position of the optical system 243, receives the light focused by the optical system 243 and converts it into an electrical signal to implement prescribed signal processing.

[0053] The optical system 243 is formed using one or more lenses. The optical system 243 forms an observation image on the light receiving surface of the imaging element 244. The optical system 243 may have an optical zoom function for changing the viewing angle and a focusing function for changing the focus.

[0054] Figure 4 This is a diagram for explaining the structure of the optical system of the endoscope. The optical system 243 has two lenses 243a and 243b and a cutoff filter 243c disposed between the lenses 243a and 243b. The cutoff filter 243c cuts off light in a wavelength band above the wavelength band of the therapeutic light. That is, the cutoff filter 243c allows light in a wavelength band smaller than the wavelength band of the therapeutic light to pass. For example, in the case of photoimmunotherapy (PIT), light in a wavelength band above 680nm is cut off, and light in a wavelength band smaller than 680nm passes through the cutoff filter 243c. According to the filter characteristics, a portion of the light in the wavelength band above 680nm passes through the cutoff filter 243c (refer to Figure 4 ). For example, the red light Q R The blue light Q B and green band light Q G Pass through cutoff filter 243c.

[0055] The cut filter 243c is preferably a filter with a high OD value.

[0056] in addition, Figure 4 The structure of the optical system shown is an example, and various design changes can be made.

[0057] The image sensor 244 performs photoelectric conversion on the light from the optical system 243 to generate an electric signal (image signal). The imaging element 244 has: a pixel unit, which is composed of a plurality of pixels arranged in a matrix, each of which has a photodiode that accumulates charges corresponding to the amount of light, a capacitor that converts the charges transmitted from the photodiode into a voltage level, and a color filter that is provided in each pixel. Each pixel performs photoelectric conversion on the light incident through the optical system 243 to generate an electric signal, and the imaging element 244 sequentially reads out the electric signals generated by the pixels arbitrarily set as the readout object among the plurality of pixels, and outputs them as image signals. The imaging element 244 is implemented using, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0058] Figure 5 2 is a diagram schematically showing the structure of a pixel unit of the image pickup element 244. The pixel unit is composed of a plurality of pixels P such as a photodiode that accumulates charges corresponding to the amount of light. nm (n, m are integers greater than 1) are arranged in a two-dimensional matrix. The pixel unit is controlled by the control unit 44 from a plurality of pixels P nm The pixel P in the readout area arbitrarily set as the readout target nm The image signal is read out as image data and output to the processing device 4 .

[0059] Figure 6 2 is a diagram schematically showing the structure of the color filter of the imaging element 244. The color filter is composed of a Bayer array with 2×2 as one unit. The color filter is composed of a filter R that transmits light in the red band, two filters G that transmit light in the green band, and a filter B that transmits light in the blue band. Figure 5 In FIG. 1 , the reference numerals (e.g., G11) for each filter are associated with the pixel P. nm Corresponding means configured at corresponding pixel positions.

[0060] Figure 7 is a diagram schematically showing the sensitivity characteristics of each filter. Figure 7 In the figure, the horizontal axis represents wavelength (nm) and the vertical axis represents transmission characteristics (sensitivity characteristics). Figure 7 In the curve L B Represents the transmission characteristics of filter B, curve L G Represents the transmission characteristics of filter G, curve L R Represents the transmission characteristics of filter R.

[0061] Filter B allows light in the blue band to pass through (see Figure 7The curve L B ). In addition, filter G allows light in the green band to pass through (see Figure 7 The curve L G ). In addition, filter R allows light in the red band to pass through (see Figure 7 The curve L R ). In addition, the following is a pixel P formed by configuring a filter R on the light receiving surface. nm The pixel P is marked as R pixel, and the pixel P is formed by configuring the filter G on the light receiving surface. nm The pixel P is marked as G pixel, and the pixel P is formed by configuring the filter B on the light receiving surface. nm Marked as B pixel for explanation.

[0062] According to the imaging element 244 configured in this way, when receiving the subject image formed by the optical system 243, a color signal (G signal, R signal, and B signal) of each of the G pixel, the R pixel, and the B pixel is generated (see Figure 8A to Figure 8C ).

[0063] In addition, the endoscope 2 has a memory (not shown) that stores an execution program and a control program for causing the imaging element 244 to execute various actions, and data including identification information of the endoscope 2. The identification information includes unique information (ID), year model, specification information, and transmission method of the endoscope 2. In addition, the memory can also temporarily store image data generated by the imaging element 244.

[0064] The structure of the light source device 3 will be described. The light source device 3 includes a light source unit 31, an illumination control unit 32, and a light source driver 33. The light source unit 31 sequentially switches illumination light under the control of the illumination control unit 32 and emits the illumination light toward the subject (object).

[0065] The light source unit 31 is composed of a light source, one or more lenses, etc., and emits light (illumination light) by driving the light source. The light generated by the light source unit 31 is emitted from the front end of the front end portion 24 toward the subject via the light guide 241. The light source unit 31 includes a white light source 311 and an absorption band light source 312. The light source units, the light guide 241, and the illumination lens 242 constitute an emission unit. For example, the absorption band light source 312, the light guide 241, and the illumination lens 242 constitute an absorption band light emission unit.

[0066] The white light source 311 emits light having a wavelength band in the visible light region (white light). The white light source 311 is realized by using any light source such as an LED light source, a laser light source, a xenon lamp, or a halogen lamp.

[0067] The absorption band light source 312 emits light of the absorption band of the antibody drug used in PIT (absorption band light), which is composed of a part of the wavelength or band in the visible light region. The absorption band is the band of the Soret band of the antibody drug, for example, below 450nm. In the antibody drug used in PIT, the Soret band has a band of 350nm or more and 400nm or less. Fig. 9 This is a diagram for explaining an example of a wavelength band of light used as therapeutic light and absorption band light. Fig. 9 The vertical axis is the relative intensity, which shows the same example, but the absorption band light is, for example, the light L in the wavelength range of 350 nm to 400 nm. S In this embodiment, the light L S The absorption band light source 312 is described as an example of absorption band light. The absorption band light source 312 is realized by using an LED light source, a laser light source, or the like.

[0068] In the case of an antibody drug that stimulates PIT, for example, near-infrared light with a central wavelength of 690 nm (e.g. Fig. 9 The wavelength range of light L shown is between 660nm and 710nm. P ).

[0069] At this time, it is also possible to use narrowband light of a different wavelength from the absorption band light. For example, it is possible to use any light or a combination of narrowband light in the wavelength band of 390 nm to 445 nm and the wavelength band of 530 nm to 550 nm. By irradiating light of the wavelength band of 390 nm to 445 nm and obtaining its scattered light or return light, the blood vessels in the mucosal surface layer can be depicted with high contrast. In addition, by irradiating light of the wavelength band of 530 nm to 550 nm and obtaining its scattered light or return light, the blood vessels in the deeper part can be depicted in the mucosal surface layer with high contrast. In addition, by irradiating light of the wavelength band of 590 nm to 620 nm or the wavelength band of 620 nm to 780 nm and obtaining its scattered light or return light, the blood vessels in the deeper part can also be depicted in the mucosal surface layer with high contrast.

[0070] The lighting control unit 32 controls the amount of power supplied to the light source unit 31 based on a control signal (dimming signal) from the processing device 4 , and controls the light source to be emitted and the driving timing of the light source.

[0071] The light source driver 33 supplies current to the light source of the light emission target under the control of the illumination control unit 32 , thereby causing the light source unit 31 to emit light.

[0072] The configuration of the processing device 4 will be described. The processing device 4 includes an image processing unit 41 , a synchronization signal generating unit 42 , an input unit 43 , a control unit 44 , and a storage unit 45 .

[0073] The image processing unit 41 receives image data of illumination light of various colors captured by the imaging element 244 from the endoscope 2. When receiving analog image data from the endoscope 2, the image processing unit 41 performs A / D conversion to generate a digital imaging signal. Furthermore, when receiving image data from the endoscope 2 as an optical signal, the image processing unit 41 performs photoelectric conversion to generate digital image data.

[0074] The image processing unit 41 performs predetermined image processing on the image data received from the endoscope 2 to generate an image and output it to the display device 5, or sets an enhanced region determined from the image, or calculates the time change of the fluorescence intensity. The image processing unit 41 includes a white light image generating unit 411, an absorption band light image generating unit 412, and a display image generating unit 413.

[0075] The white-light image generating unit 411 generates a white-light image based on an image formed by white light. The white-light image generating unit 411 generates a white-light image based on color signals of each of the R pixel, the G pixel, and the B pixel.

[0076] In the present embodiment, the absorption band light image generation unit 412 generates an absorption band light image based on an image formed by the absorption band light, and the absorption band light image is an image for overlapping with the white light image. The absorption band light image generation unit 412 uses the color signal of the B pixel to generate an absorption band light image, such as a grayscale, that expresses the absorption of the absorption band light by the antibody agent. In the absorption band light image, the closer the absorption band light is to a part where the absorption band light is absorbed, the smaller the brightness value. The absorption band light image generation unit 412 generates an absorption band light image in which the shading is thicker as the brightness value is smaller. Therefore, in the case of grayscale, the absorption band light image becomes an image in which the greater the absorption of the absorption band light, the thicker the black.

[0077] The display image generation unit 413 generates a superimposed image in which a white light image, an absorption band light image, a narrow-band light image, or an absorption band light image for superposition is superimposed on a predetermined image to be displayed on the display device 5. The superimposed image is an image in which the absorption band light image is superimposed on an image based on white light or narrow-band light. At this time, the display image generation unit 413 extracts a region having a brightness value equal to or greater than the absorption region from the absorption band light image, for example, and superimposes the extracted region on the white light image.

[0078] Here, the optical system 243, the imaging element 244 and the image generation unit constitute an image acquisition unit. For example, when acquiring an image formed by illumination with absorption band light, the optical system 243, the imaging element 244 and the absorption band light image generation unit 412 constitute an absorption band light image acquisition unit.

[0079] The white light image generating unit 411, the absorption band light image generating unit 412, and the display image generating unit 413 generate images by performing prescribed image processing. Here, the prescribed image processing is synchronization processing, grayscale correction processing, and color correction processing. Synchronization processing is a process of synchronizing the image data of each color component of RGB. Grayscale correction processing is a process of performing grayscale correction on image data. Color correction processing is a process of performing hue correction on image data. In addition, the white light image generating unit 411, the absorption band light image generating unit 412, and the display image generating unit 413 may also perform gain adjustment according to the brightness of the image.

[0080] The image processing unit 41 is configured using a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor such as an ASIC (Application Specific Integrated Circuit) or various arithmetic circuits that perform specific functions. Alternatively, the image processing unit 41 may include a frame memory that stores R image data, G image data, and B image data.

[0081] The synchronization signal generating unit 42 generates a clock signal (synchronization signal) as a reference for the operation of the processing device 4, and outputs the generated synchronization signal to the light source device 3, the image processing unit 41, the control unit 44, and the endoscope 2. Here, the synchronization signal generated by the synchronization signal generating unit 42 includes a horizontal synchronization signal and a vertical synchronization signal.

[0082] Therefore, the light source device 3 , the image processing unit 41 , the control unit 44 , and the endoscope 2 operate in synchronization with each other based on the generated synchronization signal.

[0083] The input unit 43 is implemented using a keyboard, a mouse, a switch, or a touch panel, and receives input of various signals such as an operation instruction signal for instructing the operation of the endoscope system 1. Alternatively, the input unit 43 may include a switch provided in the operation unit 22 or an external portable terminal such as a tablet computer.

[0084] The control unit 44 performs drive control of each component including the imaging element 244 and the light source device 3, and input and output control of information for each component. The control unit 44 refers to the control information data for imaging control (for example, read timing, etc.) stored in the storage unit 45, and sends it as a drive signal to the imaging element 244 via a predetermined signal line included in the collective cable 245, or switches between the normal observation mode (white light observation mode) in which an image obtained by illumination with white light is observed and the absorption band light observation mode in which an image obtained by illumination with absorption band light is observed. The control unit 44 is configured using a general-purpose processor such as a CPU, a dedicated processor such as various operation circuits that perform specific functions such as an ASIC, etc.

[0085] The storage unit 45 stores various programs for operating the endoscope system 1 and data including various parameters required for the operation of the endoscope system 1. In addition, the storage unit 45 stores identification information of the processing device 4. Here, the identification information includes unique information (ID) of the processing device 4, year model, specification information, and the like.

[0086] In addition, the storage unit 45 stores various programs including an image acquisition processing program for executing the image acquisition processing method of the processing device 4. Various programs can also be recorded in a computer-readable recording medium such as a hard disk, a flash memory, a CD-ROM, a DVD-ROM, a floppy disk, etc. and widely circulated. In addition, the above-mentioned various programs can also be obtained by downloading via a communication network. The communication network mentioned here is realized by, for example, an existing public line network, a LAN (Local Area Network), a WAN (Wide Area Network), etc., and can be wired or wireless.

[0087] The storage unit 45 having the above configuration is realized by using a ROM (Read Only Memory) in which various programs and the like are pre-installed, a RAM for storing calculation parameters and data and the like for each process, a hard disk, and the like.

[0088] The display device 5 displays a display image corresponding to the image signal received from the processing device 4 (image processing unit 41) via the video cable. The display device 5 is configured using a monitor such as a liquid crystal or an organic EL (Electro Luminescence).

[0089] The treatment instrument device 6 has a treatment instrument operating part 61 and a flexible treatment instrument 62 extending from the treatment instrument operating part 61. The treatment instrument 62 used in PIT is a treatment light emitting part that emits light for treatment (hereinafter referred to as treatment light). The treatment instrument operating part 61 controls the emission of treatment light by the treatment instrument 62. The treatment instrument operating part 61 has an operation input part 611. The operation input part 611 is composed of, for example, a switch. The treatment instrument operating part 61 causes the treatment instrument 62 to emit treatment light by inputting to the operation input part 611 (for example, pressing a switch). In addition, in the treatment instrument device 6, a light source that emits treatment light may be provided in the treatment instrument 62 or in the treatment instrument operating part 61. The light source is implemented using a semiconductor laser, an LED, or the like. For example, in the case of PIT, the treatment light is light in a wavelength band above 680 nm, for example, light with a center wavelength of 690 nm (for example Fig. 9 The light L shown P ).

[0090] Here, the illumination optical system of the treatment device 62 may also be configured to be able to change the irradiation range of the treatment light. For example, it may be configured by an optical system or DMD (Digital Micromirror Device) that can change the focal length under the control of the treatment device operation unit 61, and can change the spot diameter of the light irradiated to the subject or the shape of the irradiation range.

[0091] Next, refer to Fig.10 The flow of treatment using the endoscope 2 will be described. Fig.10 This is a diagram showing an example of a treatment flow using the endoscope according to the first embodiment of the present invention. Fig.10 1 is a diagram showing an example of implementation of PIT, in which the insertion portion 21 is inserted into the stomach ST to perform treatment.

[0092] First, the operator inserts the insertion portion 21 into the stomach ST (see Fig.10 (a)). At this time, the operator causes the light source device 3 to emit white light, and searches for the treatment position while observing the white light image in the stomach ST displayed by the display device 5. Here, the tumor B to be treated is performed. 1 , B 2 In this case, the tumor B that is the target of treatment 1 , B 2 The antibody drug can be administered using the endoscope 2 or other equipment, or the patient can swallow the drug.

[0093] The surgeon observes the white light image and places the tumor B 1 , B 2The area is determined as the irradiation area. In addition, the irradiation area can be irradiated with narrow-band light as needed to obtain a narrow-band light image. The operator can confirm the surface blood vessels of the living body tissue through the narrow-band light image.

[0094] The operator directs the distal end portion 24 toward the tumor B. 1 The treatment instrument 62 is protruded from the front end of the endoscope 2 to treat the tumor B. 1 Irradiation with therapeutic light (see Fig.10 (b)). Through the irradiation of therapeutic light, the tumor B 1 The bound antibody agent reacts to the tumor B 1 Implement treatment.

[0095] Then, the operator directs the distal end portion 24 toward the tumor B. 2 The treatment instrument 62 is protruded from the front end of the endoscope 2 to treat the tumor B. 2 Irradiation with therapeutic light (see Fig.10 (c)). Through the irradiation of therapeutic light, the tumor B 2 The bound antibody agent reacts to the tumor B 2 Implement treatment.

[0096] The operator repeatedly performs additional irradiation of therapeutic light and confirmation of the treatment effect as needed.

[0097] At this time, the operator irradiates each tumor with absorption band light and obtains an absorption band light image to observe the amount of antibody drug in the tumor. For example, the operator irradiates the absorption band light before or after the treatment light irradiation to confirm the accumulation amount of the antibody drug at that time. Fig.11 This is a flowchart showing an example of processing of the endoscope system according to the first embodiment.

[0098] First, the insertion unit 21 is inserted into the subject through the operation of the operator, and the biological tissue in the subject is irradiated with white light (step S101). In the processing device 4, the white light image generation unit 411 generates a white light image based on the white light (step S102). Thereafter, the display image generation unit 413 generates a white light image for display and displays it on the display device 5 (step S103). At this time, the control unit 44 sets the observation mode to the normal observation mode, so that the display device 5 displays the white light image. The control unit 44 sets the observation mode, for example, using the emission instruction of white light as a trigger.

[0099] The operator observes the displayed white light image and searches for tumors, etc. At this time, the operator administers the antibody drug to the searched treatment object.

[0100] Then, under the operation of the operator, the control unit 44 emits the absorption band light, thereby irradiating the treatment position with the absorption band light from the distal end portion 24 (step S104). Then, the absorption band light image generation unit 412 generates an absorption band light image based on the absorption band light (step S105). The absorption band light image generation unit 412 uses the signal of the B pixel to generate an absorption band light image for superposition with the white light image, which is displayed darker as the brightness value is smaller (the degree of absorption is greater).

[0101] At this time, the control unit 44 sets the observation mode to the absorption band light observation mode. The control unit 44 switches the observation mode, for example, using the emission instruction of the absorption band light as a trigger.

[0102] Thereafter, the display image generator 413 generates a superimposed image for display in which the absorption band image is superimposed on the white light image (step S106 ). The controller 44 causes the display device 5 to display the generated superimposed image (step S107 ).

[0103] In addition, when narrow-band light is used, the display image generation unit 413 may generate a display image in which the superimposed image is superimposed on the narrow-band light image. When a narrow-band light image is used, the vascular contrast image becomes the background of the image due to the narrow-band light, so that, for example, an image can be obtained in which the boundary area of ​​the antibody drug accumulated in the cancer tissue area can be visually recognized.

[0104] After displaying the superimposed image, the control unit 44 determines whether to irradiate the therapeutic light by the operator's operation (step S108). The control unit 44 determines whether to irradiate the therapeutic light by, for example, determining whether there is an operation input from the operation input unit 611. If the control unit 44 determines that the therapeutic light is not to be irradiated (step S108: No), the control unit 44 moves to step S110. On the other hand, if the control unit 44 determines that the therapeutic light is to be irradiated (step S108: Yes), the control unit 44 moves to step S109.

[0105] In step S109, the treatment instrument 62 is inserted into the endoscope 2 by the operator's operation, and therapeutic light is irradiated from the treatment instrument 62 to the antibody drug bound to the cancer cells to cause the drug to react (drug reaction step). In this drug reaction step, treatment is performed to activate the antibody drug by irradiating near-infrared light as therapeutic light to destroy the cancer cells.

[0106] In step S110, the control unit 44 determines whether the absorption band light is irradiated by the operation of the operator. The control unit 44 determines whether the absorption band light is irradiated by determining whether there is an operation input to the input unit 43. When the control unit 44 determines that the absorption band light is not irradiated (step S110: No), the processing ends. On the other hand, when the control unit 44 determines that the absorption band light is irradiated (step S110: Yes), it moves to step S104 and repeats the processing. When the absorption band light is irradiated after the treatment light is irradiated, the density of the obtained absorption band light image expresses the amount of the antibody drug reduced by the irradiation of the treatment light.

[0107] Here, refer to Fig.12 A display image displayed in absorption band light observation will be described. Fig.12 1 is a diagram for explaining an example of a display image during treatment. First, a white light image F is displayed by irradiation with white light. 1 (Refer to Fig.12 (a)). In this white light image F 1 Tumor B 3 The operator observes the white light image F 1 , master tumor B 3 .

[0108] Then, by irradiating the absorption band light, a superimposed image F is displayed in which the superimposition image is superimposed on the white light image. 2 , where the overlay image represents the absorption of light in the absorption band (refer to Fig.12 (b)). In this overlapping image F 2 The Soret band region B corresponding to the region absorbed by the antibody agent is shown in FIG. 4 . Soret band region B 4 The superimposed image is generated based on the brightness value of the image based on the absorption band light, and is drawn by the superimposed image given a preset color tone and shading.

[0109] Moreover, through the white light image F 3 The display shows that the treatment device 62 is treating the tumor B 3 When using therapeutic light (see Fig.12 (c)). In this white light image F 3 Tumor B 3 and treatment instrument 62. The operator observes the white light image F 3 , while confirming that tumor B 3 and the position of the treatment instrument 62 while irradiating the treatment light.

[0110] The operator can understand the amount of antibody drug and the therapeutic effect by displaying the image on the display device 5. Specifically, the operator confirms the amount of antibody drug based on the density of the superimposed image, and determines whether to irradiate additional therapeutic light or the portion to irradiate therapeutic light.

[0111] In the above-described embodiment, by observing the superimposed image obtained by irradiating the absorption band light corresponding to the Soret band, the operator can confirm the amount of the antibody drug accumulated on the treatment object, and the absorption band light is light of a different wavelength band from the treatment light, so the progress of the treatment during confirmation can be suppressed. According to this embodiment, the progress of the PIT reaction can be suppressed, and the treatment process including the accumulation amount of the antibody drug can be confirmed. In addition, according to this embodiment, in order to suppress the progress of the treatment during confirmation, it is possible to ensure that the observation time of the antibody drug is long compared to the case where the antibody drug is excited and confirmed based on fluorescence.

[0112] Furthermore, in the above embodiment, by using the imaging element 244 using a color filter, a color signal (B pixel) indicating absorption of absorption band light can be obtained without adding a new structure, thereby generating an absorption band light image (superimposed image) indicating absorption of the antibody drug.

[0113] (Variant 1)

[0114] Next, refer to Fig.13 Modification 1 of the embodiment will be described. Fig.13 This is a diagram for explaining an example of a display image of Modification 1. The endoscope system of Modification 1 is the same as the endoscope system 1 of the embodiment, so the description is omitted. In the embodiment, an example is described in which the absorption band light image expresses the absorption degree of the absorption band light with a single hue, but Modification 1 uses multiple hues to express the absorption degree.

[0115] In the first modification, the absorption band light image generation unit 412 generates an absorption band image for superposition by using the signal of the B pixel and changing the color tone according to the brightness value (degree of absorption). Fig.13 The absorption band light image F 4 In the figure, the Soret band region B corresponding to the region absorbed by the antibody agent is shown. 5 . Soret band region B 5 A hue corresponding to the brightness value of an image based on light in the absorption band is given.

[0116] The display image generation unit 413 generates a superimposed image for display by superimposing the absorption band light image on the white light image. The generated superimposed image is displayed on the display device 5 under the control of the control unit 44 .

[0117] In the above-described modification 1, similarly to the embodiment, by observing the superimposed image obtained by irradiating the absorption band light corresponding to the Soret band, the operator can confirm the amount of the antibody drug accumulated on the treatment object, and since the absorption band light is light of a different wavelength band from the treatment light, the progress of the treatment during the confirmation can be suppressed. According to this modification 1, the progress of the PIT reaction can be suppressed, and the treatment process including the accumulation amount of the antibody drug can be confirmed, and further, the observation time of the antibody drug can be ensured to be long.

[0118] (Variant 2)

[0119] Next, refer to Fig.14 Modification 2 of the embodiment will be described. Fig.14 This is a diagram for explaining an example of a display image of Modification 2. The endoscope system of Modification 2 is the same as the endoscope system 1 of the embodiment, so the description is omitted. In the embodiment, an example is described in which the display image is an image in which the absorption band light image is superimposed on the white light image, but Modification 2 is set to display the white light image and the absorption band light image side by side.

[0120] In the second modification, the display image generation unit 413 generates a display image including the tumor B. 3 White light image F 11 and region B containing the Soret band 4 Absorption band light image F 21 Display images side by side F 5 .

[0121] In the above-described variation 2, by separately displaying a white light image and an absorption band light image obtained by irradiating an absorption band light corresponding to the Soret band, the operator can confirm the amount of the antibody drug accumulated on the treatment object, as in the embodiment, and since the absorption band light is light of a different wavelength band from the treatment light, the progress of the treatment during the confirmation can be suppressed. According to this variation 2, the progress of the PIT reaction can be suppressed, and the treatment process including the accumulation amount of the antibody drug can be confirmed, and further, the observation time of the antibody drug can be ensured to be long.

[0122] (Variant 3)

[0123] Next, refer to Fig.15 Modification 3 of the embodiment will be described. Fig.151 is a block diagram showing a schematic structure of an endoscope system of Modification 3. The endoscope system of Modification 3 includes a processing device 4A instead of the processing device 4 of the endoscope system 1 of the embodiment. The other structures are the same as those of the embodiment, so the description is omitted. In the embodiment, an example is described in which the display image is an image in which the absorption band light image is superimposed on the white light image, but Modification 2 is set to be a display image in which the white light image and the absorption band light image are arranged.

[0124] The processing device 4A includes an image processing unit 41A, a synchronization signal generating unit 42, an input unit 43, a control unit 44, and a storage unit 45. The synchronization signal generating unit 42, the input unit 43, the control unit 44, and the storage unit 45 are the same as those in the embodiment, and thus description thereof is omitted.

[0125] The image processing unit 41A receives image data of illumination light of each color captured by the imaging element 244 from the endoscope 2, performs predetermined image processing to generate an image and outputs it to the display device 5, or sets an enhanced region determined from the image, or calculates a temporal change in fluorescence intensity. The image processing unit 41A includes a white light image generating unit 411, an absorption band light image generating unit 412, a display image generating unit 413, and a normalizing unit 414.

[0126] In Modification 3, the color signal of the B pixel is processed as a signal including a region corresponding to the absorption region of the antibody drug, which is a region where the reflectance decreases due to absorption. In addition, the color signal of the G pixel is a color signal corresponding to scattered light from the biological tissue during PIT, and is processed as a background that is not affected by the absorption of the antibody drug and as a reference signal for standardizing the color signal.

[0127] The normalization unit 414 normalizes the signal of the B pixel using the signal of the G pixel, for example. Through the normalization process, the signal value (normalization value) of the B pixel after normalization becomes smaller as the absorption band light is absorbed.

[0128] The smaller the normalization value, the darker the absorption band light image is generated by the absorption band light image generation unit 412. At this time, a background image may be generated based on the signal of the G pixel.

[0129] The display image generation unit 413 generates a superimposed image by superimposing the absorption band light image generated by the absorption band light image generation unit 412 on the white light image.

[0130] In this modification example 3, an image with a darker contrast is generated as the normalization value is smaller. Therefore, the absorption band light image becomes an image with a darker color as the absorption of the absorption band light is greater. Alternatively, the absorption band light image of modification example 1 and the display image of modification example 2 may be used.

[0131] In the above-described modification example 3, by separately displaying a white light image and an absorption band light image obtained by irradiating an absorption band light corresponding to the Soret band, the operator can confirm the amount of the antibody drug accumulated on the treatment object, as in the embodiment, and since the absorption band light is light of a different wavelength band from the treatment light, the progress of the treatment during the confirmation can be suppressed. According to this modification example 3, the progress of the PIT reaction can be suppressed, and the treatment process including the accumulation amount of the antibody drug can be confirmed, and further, the observation time of the antibody drug can be ensured to be long.

[0132] Furthermore, according to Modification 3, by normalizing the signal values ​​of the B pixels, it is possible to generate a superimposed image in which the unevenness of illumination intensity is suppressed and the contribution of light absorption is emphasized.

[0133] Furthermore, according to variant example 3, by using an imaging element 244 that uses a color filter, it is possible to obtain color signals (B pixels) indicating the absorption of absorption band light and color signals (G pixels) for background and reference, respectively, without adding a new structure, thereby generating an absorption band light image (overlapping image) showing the absorption of the antibody agent.

[0134] In addition, in variant example 3, an example is described in which the standardization unit 414 uses the signal of the G pixel for standardization, but the present invention is not limited to this. The signal of the B pixel may be divided by the sum of the signals of the R pixel, the G pixel, and the B pixel for standardization, or the signal of the B pixel may be divided by the sum of the signals of the R pixel and the G pixel for standardization.

[0135] At this time, normalization using the sum of signals from pixels of different categories (eg, R pixels and G pixels) can more reliably eliminate uneven illumination intensity and improve image contrast, compared with normalization using only signals from G pixels.

[0136] In the above-described embodiment, an example in which the light source device 3 and the treatment device 4 are separate bodies is described, but it is also possible to have a structure in which the light source device 3 and the treatment device 4 are integrated. In addition, in the embodiment, an example in which the treatment light is irradiated by the treatment device is described, but it is also possible to have a structure in which the light source device 3 emits the treatment light.

[0137] In the above-described embodiment, the imaging element 244 may be configured using a multi-band image sensor to separately acquire light of a plurality of mutually different wavelength bands.

[0138] In addition, in the above-mentioned embodiment, in order to confirm the presence of the antibody drug, it can also be configured to emit an excitation light that excites the antibody drug. In this case, the excitation light and the therapeutic light can be of the same band (same center wavelength) or different bands (center wavelengths). In addition, in the case where the excitation light and the therapeutic light are used together, it is sufficient to irradiate the therapeutic light (excitation light) through the treatment device 62 or the excitation light source provided in the light source device, and it can be configured without the excitation light source and the treatment device 62. In the case of an antibody drug that excites PIT, for example, near-infrared light with a center wavelength of 690nm is used.

[0139] In the above-mentioned embodiment, an example of application to photoimmunotherapy (PIT) is described, but the invention is not limited to PIT and can also be applied to a therapy using a photosensitive substance such as photodynamic therapy (PDT).

[0140] In addition, in the above-mentioned embodiment, it is described that the endoscope system of the present invention is an endoscope system 1 that uses a soft endoscope 2 whose observation object is a biological tissue in a subject, etc., but it can also be applied to an endoscope system that uses a rigid endoscope, an industrial endoscope for observing the characteristics of a material, a fiberscope, an optical sight tube, etc., in which a camera is connected to the eyepiece of the optical endoscope.

[0141] [Industrial Application Possibility]

[0142] As described above, the image processing device, phototherapy system, image processing method, image processing program, and phototherapy method of the present invention are useful for confirming drug accumulation and the course of treatment while suppressing the progress of the reaction in photoimmunotherapy.

[0143] Description of Reference Numerals

[0144] 1 Endoscopic system

[0145] 2 Endoscope

[0146] 3 Light source device

[0147] 4 Processing device

[0148] 5 Display device

[0149] 6 Disposal equipment

[0150] 21 Insertion

[0151] 22 Operation Department

[0152] 23 Universal Cable

[0153] 24 Front end

[0154] 25 Bend

[0155] 26 Flexible pipe

[0156] 31 Light source

[0157] 32 Lighting Control Department

[0158] 33 Light source driver

[0159] 41, 41A Image processing unit

[0160] 42. Synchronous signal generation unit

[0161] 43 Input section

[0162] 44 Control Department

[0163] 45 Storage

[0164] 61 Disposal Equipment Operation Department

[0165] 62 Disposal Equipment

[0166] 241 Light Guide

[0167] 242 Lighting lens

[0168] 243 Optical System

[0169] 244 Camera Components

[0170] 311 White light source

[0171] 312 Absorption band light source

[0172] 411 White light image generation unit

[0173] 412 Absorption band light image generation unit

[0174] 413 Display image generation unit

[0175] 414 Standardization Department

Claims

1. An image processing device, wherein: The image processing device comprises: A white light image generating unit that generates a white light image based on white light, wherein the white light is composed of light in a wavelength band of a visible light region; an absorption band light image generating unit that generates an absorption band light image based on absorption band light composed of light in a wavelength band of a Soret band absorbed by a drug focused on a treatment object; as well as A display image generating unit generates a display image based on the white light image and the absorption band light image.

2. The image processing device according to claim 1, wherein: The image processing device further includes a normalizing unit configured to divide a blue signal based on light in a blue wavelength band by a green signal based on light in a green wavelength band to normalize a signal value of the blue signal. The absorption band light image generation unit generates the absorption band light image using the normalized signal value of the blue signal.

3. The image processing device according to claim 1, wherein: The image processing device further includes a normalizing unit that normalizes the signal value of the blue signal by dividing a blue signal based on light in a blue wavelength band by a sum of signal values ​​of the blue signal, a red signal based on light in a red wavelength band, and a green signal based on light in a green wavelength band. The absorption band light image generation unit generates the absorption band light image using the normalized signal value of the blue signal.

4. The image processing device according to claim 1, wherein: The image processing device further includes a normalizing unit that normalizes the signal value of the blue signal by dividing the blue signal based on the light in the blue wavelength band by the sum of the signal values ​​of the red signal based on the light in the red wavelength band and the green signal based on the light in the green wavelength band. The absorption band light image generation unit generates the absorption band light image using the normalized signal value of the blue signal.

5. The image processing device according to claim 1, wherein: The image processing device further includes a control unit that switches between a white light observation mode and an absorption band light observation mode. The white light observation mode is a mode in which a display device displays the white light image. The absorption band light observation mode is a mode in which the display device displays a display image including the absorption band light image.

6. The image processing device according to claim 1, wherein: The wavelength of the Soret band is below 450 nm. The wavelength band for exciting the drug is a wavelength band above 680 nm.

7. A light therapy system, wherein: The light therapy system comprises: A white light emitting unit that emits white light consisting of light in a wavelength band of a visible light region; an absorption band light emitting unit that emits absorption band light, the absorption band light being light of a wavelength band of a Soret band absorbed by a drug focused on a treatment object; an image processing device that generates an image based on the white light or the absorption band light; as well as a display device for displaying the image generated by the image processing device, The image processing device comprises: a white light image generating unit that generates a white light image based on the white light; an absorption band light image generating unit that generates an absorption band light image based on the absorption band light; as well as A display image generating unit generates a display image based on the white light image and the absorption band light image.

8. An image processing method, wherein: The image processing method comprises: A white light image generating step, wherein a white light image generating unit generates a white light image based on white light, wherein the white light is composed of light in a wavelength band of a visible light region; an absorption band light image generating step, wherein the absorption band light image generating unit generates an absorption band light image based on absorption band light composed of light in a wavelength band of a Soret band absorbed by a drug focused on a treatment object; and In the display image generating step, a display image generating unit generates a display image based on the white light image and the absorption band light image.

9. An image processing program, wherein: The image processing program enables the computer to execute the following steps: A white light image generating step of generating a white light image based on white light, wherein the white light is composed of light in a wavelength band of a visible light region; an absorption band light image generating step of generating an absorption band light image based on the absorption band light, wherein the absorption band light is composed of light of a wavelength band of a Soret band absorbed by a drug focused on a treatment object; as well as The display image generating step generates a display image based on the white light image and the absorption band light image.

10. A light therapy method, wherein: The light therapy method comprises the following steps: Administering a phototherapy agent to the treatment target area; irradiating the treatment target site with absorption band light to obtain an absorption band light image, wherein the absorption band light is light in a wavelength band including a Soret band and smaller than an excitation wavelength of the drug; generating a display image including the absorption band light image; displaying the display image; observing the absorption band light image to confirm the amount of the drug accumulated; and The treatment target site is irradiated with therapeutic light to cause the drug bound to the treatment target site to react.

Images