Image processing apparatus and method of operation thereof

Abstract

Provided is an image processing apparatus capable of accurately and favorably performing a determination relating to ulcerative colitis using a medical image, and a method of operation thereof. The image processing apparatus (16) includes a processor that determines whether or not to perform image processing on a medical image based on an image analysis result obtained by analyzing a photographic condition and / or the medical image. Also, the medical image on which it has been determined to perform image processing is subjected to at least one of a step of calculating an index value relating to a stage of ulcerative colitis based on a density of superficial blood vessels, intramucosal bleeding, and extramucosal bleeding obtained from the medical image, a step of determining the stage of ulcerative colitis, or a step of determining remission or non-remission of ulcerative colitis.

Description

Technical Field

[0001] This invention relates to an image processing apparatus for performing disease-related processing and its operating method. Background Technology

[0002] In the medical field, diagnostic procedures using medical images are widely performed. For example, as a device for using medical images, there is an endoscopic system that includes a light source, an endoscope, and a processor. In an endoscopic system, an endoscopic image, serving as a medical image, is acquired by illuminating the object being observed and photographing the illuminated object. The endoscopic image is then displayed on a monitor and used for diagnosis.

[0003] In recent years, by processing endoscopic images, diagnostic information has been provided to users, supporting the identification of abnormal areas such as lesions. Furthermore, appropriate image processing techniques have been developed for endoscopic images, such as Computer-Aided Diagnosis (CAD) technology for determining the stage of a disease. In CAD technology, image processing based on high-resolution endoscopic images is preferred for high-precision stage determination.

[0004] For example, Patent Document 1 discloses an image analysis device that, when analyzing the state of epithelium using an endoscope with super-magnification function, uses the halo region contained in the image to determine whether it is a super-magnified image of the object being analyzed.

[0005] Previous technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2019-111040 Summary of the Invention

[0008] The technical problem to be solved by the invention

[0009] Ulcerative colitis (UC) is a known colorectal disease. When diagnosing UC endoscopically, the resolution of endoscopic images is low when the distance from the endoscope tip to the colonic mucosa or other observable objects is large. This makes it difficult to accurately determine the vascular shape and presence of bleeding, which are fundamental to stage determination. For example, to achieve accurate stage determination using endoscopic images, if the distance to the observable objects in the endoscopic image is large, the physician needs to perform a second endoscopy to obtain new images at an appropriate distance.

[0010] The purpose of this invention is to provide an image processing apparatus and its working method that can accurately and reliably determine ulcerative colitis using medical images.

[0011] means for solving technical problems

[0012] The image processing apparatus of the present invention includes a processor that acquires medical images obtained by photographing an observed object, determines whether to perform image processing on the medical images based on image analysis results obtained by analyzing the photographic conditions and / or the medical images, and performs at least one of the following steps on the medical images for which image processing has been determined: a step of calculating index values ​​related to the stage of ulcerative colitis based on the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage obtained from the medical images; a step of determining the stage of ulcerative colitis; and a step of determining whether the ulcerative colitis is relieved or not relieved.

[0013] The preferred photographic conditions for medical images are those with a magnification index.

[0014] The image analysis result is preferably at least one of the following: halo distribution, frequency characteristics, brightness value, and shadow distribution obtained from the medical image.

[0015] The preferred medical image is obtained by photographing the object being irradiated with measurement-aid light used for measuring the object, and the preferred image analysis result is the location of the measurement-aid light irradiated area formed on the object in the medical image.

[0016] The preferred processor performs at least one of the following steps on the medical image that has been determined to undergo image processing: calculating an index value related to the stage of ulcerative colitis based on the image analysis results obtained by analyzing the photographic conditions and / or the medical image; determining the stage of ulcerative colitis; and determining whether the ulcerative colitis is in remission or not.

[0017] When determining whether ulcerative colitis is in remission or not, the preferred processor classifies the density of superficial blood vessels, intramucosal bleeding, and extramucosal bleeding by frequency characteristics or brightness values ​​obtained from medical images, and determines whether ulcerative colitis is in remission or not based on the classification.

[0018] Preferably, when the processor determines whether ulcerative colitis is relieved or not relieved, the processor calculates the spatial frequency component distribution from the medical image, extracts a first frequency characteristic region with a first frequency characteristic based on the spatial frequency component distribution, extracts a second frequency characteristic region with a second frequency characteristic, and extracts a third frequency characteristic region with a third frequency characteristic, wherein the second frequency characteristic has a higher frequency than the first frequency characteristic, and the third frequency characteristic has a higher frequency than the second frequency characteristic. Based on the first frequency characteristic region that has undergone first region discrimination processing using brightness values, the second frequency characteristic region that has undergone second region discrimination processing using brightness values, and the third frequency characteristic region, the processor detects the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage, and determines whether ulcerative colitis is relieved or not relieved based on the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage.

[0019] Preferred medical images are obtained by photographing the object being observed, which is illuminated by light including short wavelengths. Furthermore, the illumination light is preferably violet light with a center wavelength or peak wavelength including 410 nm.

[0020] Furthermore, the image processing apparatus of the present invention operates as an image processing apparatus equipped with a processor, comprising: an image acquisition step in which the processor acquires a medical image obtained by photographing an observed object; an image processing determination step in which it is determined whether to perform image processing on the medical image based on the image analysis results obtained by analyzing the photographic conditions of the medical image or the medical image; and a determination step in which at least one of the following steps is performed on the medical image that has been determined to undergo image processing: a step of calculating an index value related to the stage of ulcerative colitis based on the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage obtained from the medical image; a step of determining the stage of ulcerative colitis; and a step of determining whether the ulcerative colitis is relieved or not relieved.

[0021] Preferably, in the determination step, when determining whether ulcerative colitis is relieved or not relieved, the determination step includes: a step whereby the processor calculates the spatial frequency component distribution from the medical image; a step whereby the processor extracts a first frequency characteristic region having a first frequency characteristic, a second frequency characteristic region having a second frequency characteristic, and a third frequency characteristic region having a third frequency characteristic, wherein the second frequency characteristic has a higher frequency than the first frequency characteristic, and the third frequency characteristic has a higher frequency than the second frequency characteristic; a step whereby the processor detects the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage based on the first frequency characteristic region that has undergone first region discrimination processing using brightness values, the second frequency characteristic region that has undergone second region discrimination processing using brightness values, and the third frequency characteristic region; and a step whereby the processor determines whether ulcerative colitis is relieved or not relieved based on the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage. Furthermore, it is preferred that the medical image is obtained by photographing the object being observed illuminated by a light source, wherein the light source is violet light with a center wavelength or peak wavelength including 410 nm.

[0022] Invention Effects

[0023] According to the present invention, it is possible to make a determination related to ulcerative colitis with good accuracy using medical images. Attached Figure Description

[0024] Figure 1 This is an external view of the endoscope system.

[0025] Figure 2 This is a block diagram illustrating the function of the endoscope system of the first embodiment.

[0026] Figure 3 It is a graph representing the spectra of violet light (V), blue light (B), green light (G), and red light (R).

[0027] Figure 4 This is a graph showing the spectrum of the special light in the first embodiment.

[0028] Figure 5 It is a graph representing the spectrum of a special light that includes only violet light V.

[0029] Figure 6 This is a block diagram representing the functions of the disease-related treatment department.

[0030] Figure 7 This is a block diagram representing the functions of the image processing determination unit.

[0031] Figure 8 It is an image diagram of a monitor displaying a special image. Figure 8 (A) is an image of special image 91a. Figure 8(B) is a special image 95a.

[0032] Figure 9 This is an explanatory diagram of the halo distribution image. Figure 9 (A) is an illustration excluding the halo region. Figure 9 (B) is an explanatory diagram when the halo region is included.

[0033] Figure 10 This is an explanatory diagram of the spatial frequency distribution image. Figure 10 (A) is an explanatory diagram that includes the low-frequency region. Figure 10 (B) is an illustration diagram when the low-frequency region is not included.

[0034] Figure 11 This is an explanatory diagram illustrating the average brightness value. Figure 11 (A) is an explanatory diagram for the case of special image 91a. Figure 11 (B) is an illustration of the case of special image 95a.

[0035] Figure 12 This is an explanatory diagram illustrating the shadow area of ​​the light shield. Figure 12 (A) is an explanatory diagram for the case of special image 91a. Figure 12 (B) is an illustration of the case of special image 95a.

[0036] Figure 13 It is an image diagram containing a special image 100 of speckled SP.

[0037] Figure 14 This is an explanatory diagram illustrating the relationship between the position of spot SP and the observation distance.

[0038] Figure 15 It is an explanatory diagram of a special image including speckled SP. Figure 15 (A) is an explanatory diagram for the case of special image 100a. Figure 15 (B) is an illustration of the case of special image 100b.

[0039] Figure 16 This is an explanatory diagram showing the changes in vascular structures according to the severity of ulcerative colitis.

[0040] Figure 17 It is a cross-sectional view showing the large intestine.

[0041] Figure 18 This is an explanatory diagram showing the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage, classified according to brightness value and spatial frequency.

[0042] Figure 19This is an explanatory diagram illustrating a series of processes for extracting the first, second, and third frequency characteristic regions.

[0043] Figure 20 It is an image of a monitor that displays information related to the determination.

[0044] Figure 21 It is a flowchart representing a series of processes related to disease management.

[0045] Figure 22 This is an explanatory diagram illustrating the case where the condition is determined to be NOT in step ST130.

[0046] Figure 23 This is an explanatory diagram illustrating the case where it is determined to be yes in step ST130.

[0047] Figure 24 It is a flowchart representing a series of processes including step ST150, which is a disease-related treatment pattern.

[0048] Figure 25 This is a block diagram illustrating the function of the endoscope system in the second embodiment.

[0049] Figure 26 This is a plan view of a rotating filter.

[0050] Figure 27 This is a block diagram illustrating the function of the endoscope system in the third embodiment.

[0051] Figure 28 This is a graph showing the spectrum of ordinary light in the third embodiment.

[0052] Figure 29 This is a graph showing the spectrum of the special light in the third embodiment.

[0053] Figure 30 This is a block diagram representing a diagnostic support device.

[0054] Figure 31 This is a block diagram representing a medical service support device. Detailed Implementation

[0055] [First Implementation]

[0056] exist Figure 1The endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18, and a control console 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 has an insertion part 12a that is inserted into the body of the object being observed, an operating part 12b provided at the base of the insertion part 12a, and a bending part 12c and a front end 12d provided at the front end of the insertion part 12a. The bending part 12c is bent by operating the bending knob 12e of the operating part 12b. The front end 12d moves in the desired direction as the bending part 12c is bent.

[0057] In addition to the angle knob 12e, the operation unit 12b is also equipped with a mode switch SW (mode switching switch) 12f for mode switching operation and a zoom operation unit 12g for indicating the magnification and reduction of the endoscope image.

[0058] Furthermore, the endoscope system 10 has three modes: normal light mode, special light mode, and disease-related treatment mode. In normal light mode, a normal image with natural colors is displayed on the monitor 18 by illuminating the object with normal light and taking a picture. In special light mode, a special image emphasizing specific structures is displayed on the monitor 18 by illuminating the object with both normal light and special light of different wavelength bands. In disease-related treatment mode, the remission or non-remission of ulcerative colitis, as a disease, is determined based on the normal or special image. Additionally, in disease-related treatment mode, steps such as calculating indicator values ​​related to the stage of ulcerative colitis or determining the stage of ulcerative colitis can also be performed.

[0059] In addition, in this embodiment, a special image (endoscopic image) is used in the disease-related processing mode, but ordinary images can also be used. Furthermore, the images used in the disease-related processing mode can be special images such as endoscopic images (which are medical images), as well as medical images such as radiographic images obtained by a radiography device, CT images obtained by CT (Computed Tomography), and MRI images obtained by MRI (Magnetic Resonance Imaging). Furthermore, the disease-related processing mode is executed within the processor device 16 connected to the endoscope 12, but it can also be executed by other methods. For example, the disease-related processing unit 66 can be provided in an external image processing device different from the endoscope system 10. Figure 2 The function is to input medical images into an external image processing device and execute disease-related processing modes, and display the execution results on an external monitor connected to the external image processing device.

[0060] The processor device 16 is electrically connected to the monitor 18 and the console 19. The monitor 18 outputs and displays an image of the observed object, information accompanying the image of the observed object, etc. The console 19 functions as a user interface for receiving input operations such as function settings. Additionally, an external recording unit (not shown) for recording images or image information can also be connected to the processor device 16. Furthermore, the processor device 16 corresponds to the image processing apparatus of the present invention.

[0061] exist Figure 2 In this embodiment, the light source device 14 includes a light source unit 20 and a light source control unit 21 for controlling the light source unit 20. The light source unit 20, for example, has multiple semiconductor light sources, which are individually turned on or off. When turned on, illumination light for illuminating the object being observed is emitted by controlling the emission amount of each semiconductor light source. In this embodiment, the light source unit 20 has LEDs of four colors: V-LED (Violet Light Emitting Diode) 20a, B-LED (Blue Light Emitting Diode) 20b, G-LED (Green Light Emitting Diode) 20c, and R-LED (Red Light Emitting Diode) 20d.

[0062] like Figure 3 As shown, V-LED20a generates violet light V with a center wavelength of 405±10nm and a wavelength range of 380–420nm. B-LED20b generates blue light B with a center wavelength of 460±10nm and a wavelength range of 420–500nm. G-LED20c generates green light G with a wavelength range of 480–600nm. R-LED20d generates red light R with a center wavelength of 620–630nm and a wavelength range of 600–650nm. Additionally, the violet light V is a short-wavelength light used for detecting dense superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage in disease-related treatment modalities, and preferably includes 410nm in its center or peak wavelength.

[0063] The light source control unit 21 controls V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. Furthermore, in normal light mode, the light source control unit 21 controls each LED 20a to 20d in a normal light mode in which the light intensity ratio between violet light V, blue light B, green light G, and red light R is Vc∶Bc∶Gc∶Rc.

[0064] Furthermore, when the light source control unit 21 is in a special light mode or a disease-related treatment mode, it controls each LED 20a to 20d in a special light mode that emits violet light V (as short wavelength) with a light intensity ratio of Vs:Bs:Gs:Rs to blue light B, green light G, and red light R. This special light with a light intensity ratio of Vs:Bs:Gs:Rs preferably emphasizes superficial blood vessels, etc. Therefore, the special light preferably has a light intensity of violet light V greater than that of blue light B. For example, as... Figure 4 As shown, the ratio of the light intensity Vs of violet light V to the light intensity Bs of blue light B is set to "4:1". Furthermore, as... Figure 5 As shown, regarding special light, the intensity ratio between violet light V, blue light B, green light G, and red light R can be set to 1:0:0:0, emitting only violet light V, which is a short wavelength light.

[0065] Furthermore, in this specification, the light intensity ratio includes the case where the ratio of at least one semiconductor light source is 0 (zero). Therefore, it includes the case where any one or more of the semiconductor light sources are not lit. For example, as in the case where the light intensity ratio between violet light V, blue light B, green light G, and red light R is 1∶0∶0∶0, the light intensity ratio is also considered to be present even when only one of the semiconductor light sources is lit and the other three are not lit.

[0066] The light emitted by each LED 20a to 20d is incident on the light guide 25 via the light path coupling section 23, which is composed of a reflector and a lens. The light guide 25 is built into the endoscope 12 and the universal plug (the plug connecting the endoscope 12 to the light source device 14 and the processor device 16). The light guide 25 propagates the light from the light path coupling section 23 to the front end portion 12d of the endoscope 12.

[0067] An illumination optical system 30a and a camera optical system 30b are provided at the anterior end portion 12d of the endoscope 12. The illumination optical system 30a has an illumination lens 32, through which illumination light propagating via the light guide 25 illuminates the object being observed. The camera optical system 30b has an objective lens 42 and a camera sensor 44. Light from the object illuminating the illumination light is incident on the camera sensor 44 via the objective lens 42. Thus, an image of the object being observed is formed in the camera sensor 44.

[0068] As the image sensor 44, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used. Furthermore, instead of the primary color image sensor 44, a complementary color image sensor equipped with complementary color filters for C (cyan), M (magenta), Y (yellow), and G (green) can be used. When using a complementary color image sensor, it outputs image signals of the four colors CMYG. Therefore, through complementary-primary color conversion, the image signals of the four colors CMYG are converted into image signals of the three colors RGB, thereby obtaining image signals of the same RGB colors as those of the image sensor 44.

[0069] The camera sensor 44 is driven and controlled by the camera control unit 45. The control in the camera control unit 45 varies depending on the mode. In normal light mode or disease-related processing mode, the camera control unit 45 controls the camera sensor 44 to capture images of the observed object illuminated by normal light. As a result, a Bc image signal is output from the B pixel of the camera sensor 44, a Gc image signal is output from the G pixel, and an Rc image signal is output from the R pixel.

[0070] In special light modes or disease-related processing modes, the camera control unit 45 controls the camera sensor 44 to capture images of the observed object illuminated by ordinary light. Consequently, Bs image signals are output from the B pixels of the camera sensor 44, Gs image signals are output from the G pixels, and Rs image signals are output from the R pixels.

[0071] The CDS / AGC (Correlated Double Sampling / Automatic Gain Control) circuit 46 performs correlation double sampling (CDS) or automatic gain control (AGC) on the analog image signal obtained by the camera sensor 44. The image signal after passing through the CDS / AGC circuit 46 is converted into a digital image signal by the A / D (Analog / Digital) converter 48. The digital image signal after A / D conversion is input to the processor device 16.

[0072] The processor device 16 includes an image acquisition unit 50, a DSP (Digital Signal Processor) 52, a noise reduction unit 54, an image processing switching unit 56, an image processing unit 58, and a video signal generation unit 60. The image processing unit 58 includes a general image generation unit 62, a special image generation unit 64, and a disease-related processing unit 66.

[0073] The image acquisition unit 50 acquires an image signal of an endoscopic image, one of the medical images input from the endoscope 12. The acquired image signal is sent to the DSP 52. The DSP 52 performs various signal processing on the received image signal, including defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing. In the defect correction processing, the signals of defective pixels of the camera sensor 44 are corrected. In the offset processing, dark current components are removed from the image signal after defect correction processing, and an accurate zero level is set. The gain correction processing adjusts the signal level of each image signal by multiplying the image signal of each color after offset processing by a specific gain. Linear matrix processing is performed on the image signal of each color after gain correction processing to improve color reproducibility.

[0074] Then, the brightness or chroma of each image signal is adjusted through gamma conversion. Demosaic processing (also known as isotropic processing or synchronization processing) is performed on the image signal after linear matrix processing, and signals for the missing colors of each pixel are generated through interpolation. Through demosaic processing, all pixels acquire signals for each of the RGB colors. DSP52 performs YC conversion processing on each image signal after demosaic processing and outputs the brightness signal Y, color difference signal Cb, and color difference signal Cr to the noise reduction unit 54.

[0075] The noise reduction unit 54 performs noise reduction processing on the image signal that has undergone de-mosaic processing by the DSP 52, such as using moving average and median filtering methods. The image signal with reduced noise is then input to the image processing switching unit 56.

[0076] The image processing switching unit 56 switches the destination of the image signal from the noise reduction unit 54 to any one of the following: the normal image generation unit 62, the special image generation unit 64, and the disease-related processing unit 66, according to a set mode. Specifically, for example, when set to normal light mode, the image signal from the noise reduction unit 54 is input to the normal image generation unit 62. When set to special light mode, the image signal from the noise reduction unit 54 is input to the special image generation unit 64. When set to disease-related processing mode, the image signal from the noise reduction unit 54 is input to the disease-related processing unit 66.

[0077] The general image generation unit 62 performs general image processing on the input one-frame Rc image signal, Gc image signal, and Bc image signal. General image processing includes 3×3 matrix processing, grayscale transformation processing, three-dimensional LUT (LookUp Table) processing, color conversion processing, color emphasis processing, and spatial frequency emphasis processing. The Rc image signal, Gc image signal, and Bc image signal that have undergone general image processing are input as general images to the video signal generation unit 60.

[0078] The special image generation unit 64 performs special image processing on the input one-frame Rs, Gs, and Bs image signals. This special image processing includes 3×3 matrix processing, grayscale transformation processing, 3D LUT (Look Up Table) processing, color conversion processing, color emphasis processing, and spatial frequency emphasis processing. The Rs, Gs, and Bs image signals that have undergone special image processing are input as special images to the video signal generation unit 60.

[0079] The disease-related processing unit 66 determines whether to perform image processing on a special image based on the image analysis results obtained by analyzing the photographic conditions of the special image or the special image itself. At this time, the disease-related processing unit 66 receives zoom-related information based on the operation of the zoom operation unit 12g via the zoom control unit 57. Furthermore, for the special image determined to require image processing, at least one of the following steps is performed: calculating index values ​​related to the stage of ulcerative colitis based on the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage obtained from the special image; determining the stage of ulcerative colitis; or determining whether ulcerative colitis is in remission or not. Information related to the determination result is input to the video signal generation unit 60. Details of the disease-related processing unit 66 will be described later. In addition, in embodiments 1 to 3, the determination of whether ulcerative colitis is in remission or not by the disease-related processing unit 66 will be explained.

[0080] The video signal generation unit 60 converts information related to ordinary images, special images, or determination results output from the image processing unit 58 into a video signal that can be displayed in full color on the monitor 18. The converted video signal is then input to the monitor 18. As a result, the monitor 18 displays information related to ordinary images, special images, or determination results.

[0081] The following is a detailed explanation of the contents of Disease-Related Management Section 66. For example... Figure 6 As shown, the disease-related processing unit 66 includes an image processing determination unit 70, a spatial frequency component distribution calculation unit 71, a frequency characteristic region extraction unit 72, a structure detection unit 74, and a determination unit 76. The image processing determination unit 70 determines whether to perform image processing on a special image based on the image analysis results obtained by analyzing the photographic conditions and / or the special image itself. The image processing determination unit 70 performs image processing on the special images determined to be subject to image processing. The spatial frequency component distribution calculation unit 71, the frequency characteristic region extraction unit 72, the structure detection unit 74, and the determination unit 76 perform image processing. Image processing will be described later.

[0082] The image processing determination unit 70 determines whether to perform image processing on the special image based on image analysis results obtained by analyzing the photographic conditions and / or special image as a medical image. For example... Figure 7 As shown, the image processing determination unit 70 includes an image processing implementation control unit 81, a magnification index analysis unit 82, a halo distribution analysis unit 83, a frequency characteristic analysis unit 84, a brightness value analysis unit 85, a shadow distribution analysis unit 86, and a measurement auxiliary light analysis unit 87.

[0083] The image processing implementation control unit 81 performs controls related to the image processing determination unit 70. These controls include determining which of the following should be used to perform image processing on a special image, based on the image analysis results obtained by analyzing the photographic conditions and / or the special image itself; and determining whether to perform image processing on a special image based on the integration of multiple image analysis results when multiple analysis results are obtained. The system pre-sets which of the following should be used to determine whether to perform image processing on a special image, based on the image analysis results obtained by analyzing the photographic conditions and / or the special image itself. For example, after obtaining all photographic conditions and image analysis results, the image processing implementation control unit 81 can determine whether to perform image processing based on the integration of these analysis results, or it can determine this using only the photographic conditions.

[0084] Furthermore, the image processing implementation control unit 81 controls the transmission of image analysis results obtained by analyzing the photographic conditions and / or special images of the special images to the disease-related processing unit 66. The disease-related processing unit performs at least one of the following steps on the special images determined by the image processing determination unit for image processing: calculating an indicator value related to the stage of ulcerative colitis using the image analysis results obtained by analyzing the photographic conditions and / or special images of the special images transmitted from the image processing implementation control unit 81; determining the stage of ulcerative colitis; and determining whether the ulcerative colitis is in remission or not in remission.

[0085] When photographic conditions are based on specific images, magnification indicators may be used. Besides magnification indicators, photographic conditions may also include the photographic distance (distance between the object being observed and the tip of the endoscope) or the amount of illumination light. When based on image analysis results obtained by analyzing specific images, indicators such as halo distribution, frequency characteristics, brightness values, shadow distribution, or the position of the measurement auxiliary light may be used. These can be used individually or in combination. When multiple indicators are used, the image processing implementation control unit 81 integrates them and determines whether image processing should be performed based on preset references.

[0086] The magnification index used based on photographic conditions is an index related to magnification. For example, in addition to photographic conditions such as the magnification of optical zoom and / or electronic zoom, focal distance, or depth of field during image acquisition, information related to magnification, such as the operation history of the zoom unit, the type of zoom, and the proportion of each zoom, can also be used. Furthermore, information related to magnification obtained through image analysis of specific images can be used. Additionally, magnification is a relative value; for example, it represents the proportion of the size of the observed object's image in an image acquired after magnification or zoom, compared to the size of the observed object's image in an image acquired without magnification or zoom, based on the size of the observed object's image in an image acquired without magnification or zoom.

[0087] In this embodiment, the types of zoom include optical zoom and electronic zoom. In optical zoom, the image of the observed object imaged on the camera sensor is magnified by moving the lens and adjusting the focal distance, thereby magnifying the observed object displayed on the monitor. In electronic zoom, a portion (e.g., the central portion) of the image signal obtained by photographing the observed object is trimmed, the trimmed area is stretched, and then displayed on the monitor to magnify the observed object. In this embodiment, the type of zoom can be optical zoom, electronic zoom, or a combination of both.

[0088] like Figure 8 As shown, in the case where the image of the observed object is displayed on a monitor, for example, as Figure 8 As shown in (A), in the special image 91a displayed on the monitor image 91 without magnification or other operations, the light shield 92 mounted on the front end 12d of the endoscope creates shadows and other dark areas. Furthermore, for example, as... Figure 8 As shown in (B), in the special image 95a displayed on the magnified monitor image 95, illumination light passes through the transparent light shield 92 to reach the observed object, forming a halo area 96. Additionally, in Figure 8 (A) and Figure 8 In (B), and in Figure 11 , Figure 12 , Figure 22 and Figure 23 In the image, the light shield 92 is indicated by a diagonal line. Furthermore, a magnification display 93, representing the magnification, is shown in monitor images 91 and 95 to allow the user to easily grasp the magnification.

[0089] The magnification index analysis unit 82 receives information related to the magnification index through the zoom control unit 57, etc., and calculates the magnification by analyzing this information. To determine whether to perform image processing on a specific image, the magnification is sent to the image processing implementation control unit 81. The image processing implementation control unit 81 determines to perform image processing if the calculated magnification is above a preset threshold, and determines not to perform image processing if the magnification is below the preset threshold.

[0090] Next, based on the image analysis results obtained by analyzing a specific image, the image analysis results can include, for example, halo distribution, frequency characteristics, brightness values, shadow distribution, or the position of the measurement auxiliary light. Halo distribution is the halo distribution that calculates the distribution of halo regions contained in the specific image. Frequency characteristics are the frequency characteristics that calculate the distribution of spatial frequency components in the specific image. Brightness values ​​are the brightness values ​​that calculate the brightness values ​​of the specific image. Shadow distribution is the shadow distribution that calculates the distribution of shadow regions when the specific image contains shadows formed on the object being observed by the light shield provided at the front end of the endoscope. The position of the reflected light from the measurement auxiliary light is the position of the bright spot formed on the object by the measurement auxiliary light illuminating the object during the measurement of the object.

[0091] The halo distribution analysis unit 83 performs image analysis on special images related to halo distribution. In these special images, halo regions with extremely high brightness values ​​compared to their surroundings may sometimes appear, depending on the distance between the endoscope tip 12d and the observed object. When a special image contains a halo region, the distance between the endoscope tip 12d and the observed object is relatively close; conversely, when a halo region is not present, the distance between the endoscope tip 12d and the observed object is relatively far. Therefore, by analyzing special images and calculating the distribution of halo regions, indicators such as the distance between the endoscope tip 12d and the observed object or the resolution of the special image can be obtained.

[0092] For example, such as Figure 8 As shown in (B), the special image 95a contains multiple halo regions 96. Additionally, to avoid overly complex diagrams, in Figure 8 In (B), only a portion of the halo area is marked with symbols. Furthermore, because... Figure 8 (B) is an image that has been magnified, and therefore the magnification is displayed as 93, indicating that this image is a special image that has been magnified.

[0093] The halo distribution analysis unit 83 defines areas with a pre-set brightness value or higher as halo regions. For example, ... Figure 9 As shown, a halo distribution image representing areas above a predetermined specific brightness value is created in a special image. In the halo distribution image 95b created based on the special image 95a, as... Figure 9As shown in (B), a high-brightness region 97 is formed, which is a region with a brightness value (S) or higher (signal ≥ S) based on the aforementioned specific brightness value (S) of the halo region. Furthermore, in the halo distribution image 91b created based on the special image 91a, as... Figure 9 As shown in (A), a high-brightness region 97 is not formed. The halo distribution analysis unit 83 calculates the area (area (signal ≥ S)) of the high-brightness region 97 based on the halo distribution images 91b and 95b respectively.

[0094] In addition, in including Figure 9 In the following diagrams, special images, etc., are schematically depicted as images displayed on the monitor. However, the actual shape of the special images, etc., can be appropriately set as needed, and may be circular or rectangular, etc.

[0095] To determine whether to perform image processing on special images 91a and 95a, the area of ​​each high-brightness region 97 in the halo distribution images 91b and 95b of special images 91a and 95a is sent to the image processing implementation control unit 81. The image processing implementation control unit 81 determines to perform image processing on the special image if the calculated area of ​​each high-brightness region 97 is above a preset threshold, and determines not to perform image processing on special images where the area of ​​the high-brightness region 97 is less than the preset threshold.

[0096] The frequency response analysis unit 84 performs image analysis on special images related to frequency characteristics. It calculates the spatial frequency distribution of the special image. For example, the deepest part of the lumen where the endoscope tip 12d is far from the object being observed and the illumination light cannot reach is a low-frequency region with a lower spatial frequency than the surrounding area. Conversely, the part where the endoscope tip 12d is close to the object being observed and the illumination light reaches all parts of the object does not contain a low-frequency region. Therefore, by analyzing the spatial frequency distribution of the special image, indicators such as the distance between the endoscope tip 12d and the object being observed, or the resolution of the special image, can be obtained.

[0097] like Figure 10 As shown in (A), the spatial frequency distribution image 91c created based on the special image 91a includes a low-frequency region (frequency ≤ F) 98 where the spatial frequency is lower than the surrounding area. Furthermore, as... Figure 10 As shown in (B), the spatial frequency distribution image 95c created based on the special image 95a does not include the low-frequency region 98. The frequency characteristic analysis unit 84 calculates the area (area (frequency ≤ F)) of the low-frequency region 98 based on the spatial frequency distribution images 91c and 95c respectively.

[0098] To determine whether to perform image processing on special images 91a and 95a, the area of ​​low-frequency region 98 in the spatial frequency distribution images 91c and 95c of special images 91a and 95a is sent to the image processing implementation control unit 81. The image processing implementation control unit 81 determines to perform image processing on the special image if the calculated area of ​​the low-frequency region 98 is below a preset threshold, and determines not to perform image processing on special images if the area of ​​the low-frequency region 98 is greater than the preset threshold.

[0099] The brightness value analysis unit 85 performs image analysis on a special image related to brightness values. The analysis calculates the brightness value distribution of the special image. Based on the brightness value distribution calculated from the special image, for example, the average brightness value of the entire special image is calculated. For example, when the distance between the endoscope tip 12d and the object being observed is large and the area of ​​the deepest part of the lumen, which the illumination light cannot reach, is large, the average brightness value decreases. Conversely, when the distance between the endoscope tip 12d and the object being observed is small and the illumination light reaches most of the object, the average brightness value increases. Therefore, by analyzing the brightness values ​​of the special image, indicators such as the distance between the endoscope tip 12d and the object being observed or the resolution of the special image can be obtained.

[0100] like Figure 11 As shown in (A), the luminance value analysis unit 85 calculates the luminance value distribution based on the special image 91a, and calculates the average luminance value BV1 of the special image 91a relative to the entire image based on the luminance value distribution. Furthermore, as... Figure 11 As shown in (B), the brightness value distribution is calculated based on the special image 95a, and the average brightness value BV2 of the special image 95a relative to the whole image is calculated based on the brightness value distribution.

[0101] To determine whether to perform image processing on special images 91a and 95a, the average brightness values ​​BV1 and BV2 of each special image 91a and 95a are sent to the image processing implementation control unit 81. The image processing implementation control unit 81 determines to perform image processing on the special image if the calculated average brightness value is above a preset threshold, and determines not to perform image processing on special images with an average brightness value below the preset threshold.

[0102] The shadow distribution analysis unit 86 performs image analysis on a special image related to the shadows (hereinafter referred to as shadows) generated in the observed object by means of a light shield installed on the endoscope tip 12d. The analysis calculates the distribution of the area containing the shadows (hereinafter referred to as shadow areas) within the special image. For example, when the endoscope tip 12d is far from the observed object, the angle of the illumination light relative to the observed object deviates from perpendicular, thus the shadow area becomes larger. Conversely, when the endoscope tip 12d is close to the observed object, the angle of the illumination light relative to the observed object is close to perpendicular, thus the shadow area becomes smaller. Therefore, by analyzing the distribution of the shadow areas in the special image, indices such as the distance between the endoscope tip 12d and the observed object or the resolution of the special image can be obtained.

[0103] like Figure 12 (A) and Figure 12 As shown in (B), special images 91a and 99 each include a shadow region 92a formed by the light shield 92. The shadow distribution analysis unit 86 calculates the area of ​​the shadow region 92a based on special images 91a and 99 respectively. Furthermore, regarding the calculation of the area of ​​the shadow region 92a, for example, it is possible to use image analysis to calculate the area where the brightness value is within a specified range as the shadow region 92a.

[0104] To determine whether to perform image processing on special images 91a and 99 respectively, the area SH1 of the shadow region 92a of the light shading region 92a in special image 91a and the area SH2 of the shadow region 92a of the light shading region 92a in special image 99 are sent to the image processing implementation control unit 81. The image processing implementation control unit 81 determines that no image processing will be performed on the special image if the calculated area of ​​the shadow region 92a is greater than a preset threshold, and determines that image processing will be performed on the special image if the area of ​​the shadow region 92a is less than or equal to the preset threshold. For example, since the area SH1 of the shadow region 92a of special image 91a is greater than the preset threshold, it is determined that no image processing will be performed on special image 91a; since the area SH2 of the shadow region 92a of special image 99 is less than the preset threshold, it is determined that image processing will be performed on special image 99.

[0105] The measurement auxiliary light analysis unit 87 performs image analysis on a special image related to the position of the measurement auxiliary light. The measurement auxiliary light is used to measure the observed object and illuminates the observed object in addition to the illumination light. The measurement auxiliary light is light of a color detectable by the pixels of the image sensor 44, such as visible light like red light with a wavelength of 600nm or more and 650nm or less, and uses a light-emitting element such as a laser light source or an LED.

[0106] like Figure 13 As shown, by illuminating the object with measurement auxiliary light, a measurement auxiliary light illumination area, such as a spot SP, is formed on the object. The object with the spot SP formed by the measurement auxiliary light is photographed, and a special image 100 including the spot SP, which is the measurement auxiliary light illumination area, is obtained. By specifying the position of the spot SP in the special image 100, the distance between the endoscope tip 12d and the object (hereinafter referred to as the observation distance) is determined.

[0107] If we were to explain the method for determining the distance between the anterior endoscope tip 12d and the object of observation based on the position of the spot SP in the special image 91a, then as follows: Figure 14 As shown, the endoscope's anterior endpiece 12d includes a lens 30c for measuring auxiliary light, which illuminates the object being observed by the measuring auxiliary light emitted from the measuring auxiliary light emitting section 30d. The measuring auxiliary light is emitted when the optical axis Lm of the measuring auxiliary light enters the photographic field of view of the imaging optical system (within the area enclosed by the two solid lines L1). If observation is possible within the observation distance range Rx, it is known that the positions of the spots SP formed on the object by the measuring auxiliary light within the imaging range (indicated by arrows Qx, Qy, and Qz) at the proximal end Px, near the center Py, and distal end Pz of the range Rx are different (the points where each arrow Qx, Qy, and Qz intersects the optical axis Lm). Let the position of the endoscope's anterior endpiece 12d be position P1. The observation distance is the distance between the endoscope's anterior endpiece 12d and the object being observed. Therefore, the observation distances are the distances between position P1 and the proximal end Px, near the center Py, and distal end Pz, respectively. In detail, the observation distance is the distance from the starting point of the optical axis Ax of the imaging optics 30b in the anterior endoscope 12d to the object being observed. The axis Dv represents the observation distance.

[0108] By capturing an image of the object being observed illuminated by the measurement auxiliary light using the camera sensor 44, a special image is obtained containing spots SP, which constitute the area illuminated by the measurement auxiliary light. In this special image, the position of the spots SP varies depending on the relationship between the optical axis Ax of the imaging optical system 30b and the optical axis Lm of the measurement auxiliary light, as well as the observation distance. Therefore, by pre-acquiring correspondence information that establishes a correlation between the position of the spots SP and the observation distance, the observation distance can be obtained based on the position of the spots SP. Furthermore, when measuring the length or size of the object being observed, the actual size can be measured by pre-acquiring correspondence information that establishes a correlation between the observation distance and, for example, a marker of a predetermined size, overlaying a marker representing the actual size onto the special image.

[0109] The measurement-aided light analysis unit 87 specifies the position of the spot SP in the special image by analyzing the special image including the spot SP. For example, when the distance between the endoscope tip 12d and the object being observed is far, the spot SP is located at the top of the special image. Conversely, when the distance between the endoscope tip 12d and the object being observed is close, the spot SP is located at the center of the special image. Therefore, by analyzing the special image to specify the position of the spot SP, indices such as the distance between the endoscope tip 12d and the object being observed or the resolution of the special image can be obtained.

[0110] like Figure 15 As shown in (A), the special image 100a includes a spot SP1 formed on the observed object by measuring auxiliary light. Furthermore, as... Figure 15 As shown in (B), the special image 100b includes a spot SP2 formed on the object of observation by measurement auxiliary light. The measurement auxiliary light analysis unit 87 specifies the positions of spot SP1 and spot SP2 in each of the special images 100a and 100b, respectively. Using correspondence information pre-acquired by the measurement auxiliary light analysis unit 87 that establishes a correspondence between the position of spot SP and the observation distance, the measurement auxiliary light analysis unit 87 calculates the observation distance in special image 100a from the position of spot SP1, and calculates the observation distance in special image 100b from the position of spot SP2. Furthermore, in this embodiment, the observation distance of special image 100a calculated based on the position of spot SP1 is farther, while the observation distance of special image 100b calculated based on the position of spot SP2 is closer.

[0111] To determine whether to perform image processing on special images 91a and 95a, the calculated viewing distance is sent to the image processing implementation control unit 81. The image processing implementation control unit 81 determines to perform image processing on special images whose calculated viewing distance is below a preset threshold, and determines not to perform image processing on special images whose viewing distance is greater than the preset threshold.

[0112] The image processing control unit 81 determines a special image for image processing, and the disease-related processing unit 66 performs at least one of the following steps: calculating index values ​​related to the stage of ulcerative colitis based on the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage obtained from the special image; determining the stage of ulcerative colitis; and determining whether the ulcerative colitis is in remission or not. The inventors discovered that, as Figure 16 As shown in (A) to (E), in the disease-related treatment section 66, where ulcerative colitis is the subject of judgment, the pattern of the vascular structure changes whenever the severity worsens. When ulcerative colitis is relieved or does not occur, the pattern of the superficial blood vessels is regular. Figure 16 (A) or the degree of disorder in the regularity of the pattern of superficial blood vessels ( Figure 16 (B)). On the other hand, when ulcerative colitis is unresolved and the severity is mild, the density of superficial blood vessels is sparse ( Figure 16 (C)). Furthermore, when ulcerative colitis is unresolved and of moderate severity, intramucosal bleeding occurs ( Figure 16 (D)). Furthermore, extramucosal bleeding occurs when ulcerative colitis is unresolved and of moderate to severe severity. Figure 16 (E)). In the disease-related treatment unit 66, the pattern changes of the aforementioned vascular structures are used to determine whether ulcerative colitis has been relieved or not, based on a special image of one of the medical images.

[0113] Here, "dense superficial blood vessels" refers to the meandering and clustered state of superficial blood vessels. In visual observation, it means that many superficial blood vessels surround the intestinal gland volume (crypt) (see reference). Figure 17 The area surrounding the mucosa. "Intramucosal hemorrhage" refers to bleeding within the mucosal tissue (see reference). Figure 17 Bleeding from the endoscope needs to be differentiated from bleeding within the lumen. "Intramucosal hemorrhage" refers to bleeding that is not in the mucosa but within the lumen (catheter, fold) as observed on imaging. "Extramucosal hemorrhage" refers to small amounts of blood that have entered the lumen, blood that has seeped out from the lumen or mucosa in front of the endoscope after cleaning the lumen and can be visually identified, or blood that has seeped out from within the lumen on bleeding mucosa.

[0114] The Disease-Related Treatment Department 66 classifies superficial vascular density, intramucosal hemorrhage, and extramucosal hemorrhage based on frequency characteristics or brightness values ​​obtained from specific images, and determines the remission or non-remission status of ulcerative colitis based on the classification. Specifically, it classifies superficial vascular density, intramucosal hemorrhage, and extramucosal hemorrhage as follows: Figure 18 The images are categorized as follows: For dense superficial blood vessels, low brightness represents the brightness value, and high frequency represents the frequency characteristics. For intramucosal hemorrhage, medium brightness represents the brightness value, and medium frequency represents the frequency characteristics. For extramucosal hemorrhage, low brightness represents the brightness value, and low frequency represents the frequency characteristics. Furthermore, when representing various structures of a special image using brightness values ​​and frequency characteristics, in addition to the aforementioned three categories of dense superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage, it also includes blurred dark areas of the special image or endoscopic shadows (shadows formed in the central part of the endoscopic image when the tip 12d of the endoscope is moved along the lumen). In this embodiment, using the above classification, the dense superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage required for determining the remission or non-remission of ulcerative colitis are extracted from the special image.

[0115] like Figure 6As shown, the disease-related processing unit 66 includes a spatial frequency component distribution calculation unit 71, a frequency characteristic region extraction unit 72, a structure detection unit 74, and a determination unit 76. The spatial frequency component distribution calculation unit 71 calculates the spatial frequency component distribution by applying a Laplacian filter to a specific image.

[0116] The frequency characteristic region extraction unit 72 extracts a first frequency characteristic region (low frequency region) having a first frequency characteristic (low frequency) based on the spatial frequency component distribution, extracts a second frequency characteristic region (intermediate frequency region) having a second frequency characteristic (intermediate frequency) having a frequency higher than the first frequency characteristic, and extracts a third frequency characteristic region (high frequency region) having a third frequency characteristic (high frequency) having a frequency higher than the second frequency characteristic.

[0117] Specifically, the frequency response region extraction unit 72 includes a first frequency response region extraction unit 72a, a third frequency response region extraction unit 72b, an analysis target region detection unit 72c, and a second frequency response region extraction unit 72d, and along... Figure 19 The process shown extracts the first to third frequency characteristic regions. The first frequency characteristic region extraction unit 72a, based on the spatial frequency component distribution, identifies a specific pixel as belonging to the first frequency characteristic when the standard deviation of the frequencies of nine pixels surrounding that pixel is below a constant value. The first frequency characteristic region is extracted by detecting that specific pixel across all pixels. The first frequency characteristic region corresponds to a low-frequency region. The third frequency characteristic region extraction unit 72b extracts the third frequency characteristic region through Hesian analysis of the spatial frequency component distribution. The third frequency characteristic region corresponds to a high-frequency region. Furthermore, while the first frequency characteristic region extraction unit 72a identifies a specific pixel as belonging to the first frequency characteristic when the standard deviation of the frequencies of nine pixels surrounding that pixel is below a constant value, it can also identify a specific pixel as belonging to the first frequency characteristic when other statistical measures, such as the maximum or minimum value or average value of the frequencies of the nine surrounding pixels, are below a constant value.

[0118] The object region detection unit 72c detects the object region from which the first frequency characteristic region has been removed from the special image. The second frequency characteristic region extraction unit 72d extracts the second frequency characteristic region by removing the third frequency characteristic region from the object region. The second frequency characteristic region corresponds to the mid-frequency region.

[0119] The structure detection unit 74 detects the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage based on a first frequency characteristic region that undergoes first region discrimination processing using luminance values, a second frequency characteristic region that undergoes second region discrimination processing using luminance values, and a third frequency characteristic region. Specifically, the structure detection unit 74 detects extramucosal hemorrhage by performing first region discrimination processing on the first frequency characteristic region, detects intramucosal hemorrhage by performing second region discrimination processing on the second frequency characteristic region, and detects the density of superficial blood vessels in the third frequency characteristic region.

[0120] In addition to low-brightness extramucosal hemorrhage, the first frequency characteristic region of the low-frequency image also includes medium-brightness blurred dark areas or endoscopic shadows. Therefore, a first region discrimination process is performed to distinguish these. In the first region discrimination process, areas with brightness values ​​below a threshold in the first frequency characteristic region of the special image are detected as extramucosal hemorrhage areas. To distinguish medium-brightness intramucosal hemorrhage, a second region discrimination process is performed. In the second region discrimination process, areas with brightness values ​​above a threshold in the second frequency characteristic region of the special image are detected as intramucosal hemorrhage areas. Furthermore, to distinguish low-brightness dense superficial blood vessels, a third region discrimination process can be performed. In the third region discrimination process, areas with brightness values ​​below a threshold in the third frequency characteristic region of the special image are detected as dense superficial blood vessel areas.

[0121] When any one of the following steps is met: dense superficial blood vessels are present; the amount of intramucosal bleeding detected is above the threshold for intramucosal bleeding; the amount of extramucosal bleeding detected is above the threshold for extramucosal bleeding; and the sum of the amounts of intramucosal and extramucosal bleeding detected is above the threshold for both intramucosal and extramucosal bleeding, the determination unit 76 determines that ulcerative colitis is not relieved. Conversely, when any one of the following steps is met: dense superficial blood vessels are present; the amount of intramucosal bleeding detected is above the threshold for intramucosal bleeding; the amount of extramucosal bleeding detected is above the threshold for extramucosal bleeding; and the sum of the amounts of intramucosal and extramucosal bleeding detected is above the threshold for both intramucosal and extramucosal bleeding, the determination unit 76 determines that ulcerative colitis is relieved. Information related to the determination made by the determination unit 76 is displayed on the monitor 18 for the user to determine whether ulcerative colitis is relieved or not relieved. When the determination unit 76 determines that ulcerative colitis is relieved, as... Figure 20 As shown, the relevant messages are displayed on the monitor 18. Furthermore, when displaying information related to the determination, it is preferable to overlay a special image used for the determination by the determination unit 76.

[0122] Furthermore, in the determination unit 76, the detection quantity of intramucosal bleeding is preferably calculated based on the proportion of the second frequency characteristic region in the special image. Also, in the determination unit 76, the detection quantity of extramucosal bleeding is preferably calculated based on the proportion of the low-brightness first frequency characteristic region (the first frequency characteristic region after first region discrimination processing) in the special image. Furthermore, in the determination unit 76, in addition to or instead of determining whether ulcerative colitis is relieved or not relieved, an index value that quantifies the severity of ulcerative colitis can be calculated, and the determination of whether ulcerative colitis is relieved or not relieved is made based on the index value. The index value is then displayed as the determination result on the monitor 18.

[0123] Furthermore, in the disease-related processing unit 66, if it has the same function as, for example, the frequency characteristic analysis unit 84 and the spatial frequency component distribution calculation unit 71, these units can also be combined. That is, sometimes one part can function as the frequency characteristic analysis unit 84, and in other cases, it can function as the spatial frequency component distribution calculation unit 71.

[0124] As described above, when using medical images to determine ulcerative colitis, the image processing determination unit 70 determines whether to perform image processing on the special image based on the image analysis results obtained by analyzing the photographic conditions and / or the special image, which is a medical image. For the special image determined by the image processing determination unit 70 to require image processing, the disease-related processing unit 66 performs image processing on the special image to obtain the determination result with good accuracy, based on at least one of the steps of calculating index values ​​related to the stage of ulcerative colitis, determining the stage of ulcerative colitis, and determining whether ulcerative colitis is relieved or not relieved, according to the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage obtained from the special image. Therefore, it is possible to perform ulcerative colitis-related determinations with good accuracy using special images.

[0125] Furthermore, depending on the situation, the image processing implementation control unit 81 determines whether to perform image processing based on the image analysis results obtained by analyzing the photographic conditions and / or the special image, according to pre-set parameters. Therefore, image processing is performed on automatically selected special images. Thus, it is possible to perform accurate and simple determinations related to ulcerative colitis using special images.

[0126] Next, along Figure 21The flowchart shown illustrates a series of procedures in a disease-related treatment mode. When switching to the disease-related treatment mode, special light is shone on the object of observation. Endoscope 12 acquires a special image, one of the endoscopic images (medical images), at a certain point in time by photographing the object illuminated by the special light (step ST110). Image acquisition unit 50 acquires the special image from endoscope 12 (step ST120).

[0127] The special image is sent to the disease-related processing unit 66, and the image processing determination unit 70 determines whether image processing should be performed, that is, determines whether image processing is appropriate (step ST130). In this embodiment, two analyses are performed: frequency characteristics and halo distribution. The image processing implementation control unit 81 integrates these analysis results to determine whether image processing is appropriate.

[0128] The frequency characteristic analysis unit 84 calculates the frequency characteristics of a special image and creates a spatial frequency distribution image. The spatial frequency distribution image can be created by applying filters of various colors according to the spatial frequency. For example, the low-frequency region 98a is set to light blue, and the high-frequency region 98b is set to red, etc. The frequency characteristic analysis unit 84, for example, calculates the area with a frequency below F in the spatial frequency distribution image and compares it with a pre-set threshold for the low-frequency region. For example, the comparison is made using the following formula (1). Here, the area with a frequency below F is set as the area (frequency ≤ F), and the threshold for the low-frequency region is set as threshold - low frequency. A special image satisfying the following formula (1) indicates that the observed object is far from the anterior endoscope tip 12d or exists at the deepest part of the lumen.

[0129] Area (frequency ≤ F) > threshold_low frequency (1)

[0130] Next, the halo distribution analysis unit 83 calculates the halo distribution of the special image and creates a halo distribution image. The halo distribution image can be created by calculating the area with a brightness value of a certain value or higher. For example, the halo distribution image can be created by applying various color filters to the area with a brightness value of a certain value or higher. For example, the halo area can be set to white. The halo distribution analysis unit 83 calculates the area with a brightness value of S or higher in the halo distribution image and compares it with a pre-set threshold for the halo area. For example, the comparison is made using the following formula (2). Here, the area with a brightness value of S or higher is set as area (signal ≥ S), and the threshold for the halo area is set as threshold_halo. A special image that satisfies the following formula (2) indicates that there is no halo area indicating that the observed object is close to the front end of the endoscope (12d).

[0131] Area (signal ≥ S) < threshold_halo (2)

[0132] The image processing implementation control unit 81 receives the area of ​​the low-frequency region 98a (frequency ≤ F) and the area of ​​the halo region (signal ≥ S) from the frequency characteristic analysis unit 84 and the halo distribution analysis unit 83, and compares them with each threshold using the above equations (1) and (2). Furthermore, special images satisfying both equation (1) and equation (2) are unsuitable for determining the stage of, for example, ulcerative colitis through image processing because of the long distance from the endoscope tip 12d to the observed object and the low resolution; therefore, image processing is not performed (no in step ST140). For example, such as... Figure 22 As shown, the spatial frequency distribution image 91c based on the special image 91a satisfies equation (1), and the halo distribution image 91b based on the special image 91a satisfies equation (2). Therefore, the image processing determination unit 70 determines not to perform image processing. When the image processing determination unit 70 determines not to perform image processing, the medical image is acquired again.

[0133] Furthermore, special images that do not satisfy equation (1) and equation (2) above are advantageous for determining, for example, the stage of ulcerative colitis through image processing because of their short distance from the endoscope tip 12d to the object being observed and their high resolution. Therefore, it is determined that image processing should be performed (yes in step ST140). For example, such as Figure 23 As shown, since the spatial frequency distribution image 95c based on the special image 95a does not satisfy equation (1) and the halo distribution image 95b based on the special image 95a does not satisfy equation (2), the image processing determination unit 70 determines to perform image processing. When the image processing determination unit 70 determines to perform image processing, it proceeds to the next step.

[0134] Furthermore, since the special image that satisfies either Equation (1) or Equation (2) above is close in distance from the endoscope tip 12d to the object being observed and has high resolution, it is not very suitable to determine, for example, the stage of ulcerative colitis by image processing, but there is no problem. Therefore, it is determined to perform image processing (YES in step ST140).

[0135] Image processing is performed on a specific image for which image processing is to be performed. The spatial frequency component distribution calculation unit 71 calculates the spatial frequency component distribution based on the specific image (step ST160). The first frequency characteristic region extraction unit 72a extracts the low-frequency first frequency characteristic region based on the spatial frequency component distribution (step ST170). Furthermore, the third frequency characteristic region extraction unit 72b extracts the high-frequency third frequency characteristic region based on the spatial frequency component distribution (step ST180). The analysis target region detection unit 72c detects the analysis target region from which the first frequency characteristic region has been removed from the medical image (step ST190). The second frequency characteristic region extraction unit 72d extracts the mid-frequency second frequency characteristic region by removing the third frequency characteristic region from the analysis target region (step ST200).

[0136] In the structure detection unit 74, extramucosal hemorrhage is detected by performing a first region discrimination process on a low-frequency first frequency characteristic region (step ST210). Furthermore, in the structure detection unit 74, intramucosal hemorrhage is detected by performing a second region discrimination process on a mid-frequency second frequency characteristic region (step ST220). Additionally, in the structure detection unit 74, a third frequency characteristic region is detected as a dense area of ​​superficial blood vessels.

[0137] The determination unit 76 determines whether ulcerative colitis has been relieved or not relieved based on the density of superficial blood vessels, intramucosal bleeding, and extramucosal bleeding detected by the structure detection unit 74 (step ST230). Information related to the determination made by the determination unit 76 is displayed on the monitor 18 (step ST240).

[0138] Additionally, refer to Figure 24 The flowchart shown illustrates a series of processes by which the disease-related processing unit 66 uses image analysis results obtained by analyzing the photographic conditions and / or special images sent from the image processing implementation control unit 81 to process special images that have been determined to undergo image processing. Figure 24 The flowchart shown is in Figure 21 Step ST150 has been added to the flowchart shown. That is, before performing image processing on the special image that is determined to be subject to image processing, the parameters for image processing are obtained from the image processing implementation control unit 81 as the image analysis results obtained by analyzing the photographic conditions of the special image and / or the special image (step ST150).

[0139] For image processing parameters, for example, based on the insight that the more densely vascularized an area, the more severe the inflammatory disease, when the magnification of the observed object in a special image is low, the kernel size used to calculate the density of superficial blood vessels also needs to be reduced when evaluating the density of superficial blood vessels. It can be determined that the magnification of the special image that satisfies both of the above equations (1) and (2) is low. Therefore, the image processing implementation control unit 81 sends the image processing result of the special image that satisfies both of the above equations (1) and (2) to the disease-related processing unit 66. The disease-related processing unit 66 uses the image processing result of the special image that satisfies both of the above equations (1) and (2) sent from the image processing implementation control unit 81 as the kernel size for calculating the density of superficial blood vessels, and sets the image processing parameters using a low-magnification kernel from among a variety of pre-prepared kernel sizes. The subsequent process is the same as... Figure 21 The flowchart shown is the same.

[0140] [Second Implementation]

[0141] In the second embodiment, instead of the four-color LEDs 20a to 20d described in the first embodiment, a broadband light source such as a xenon lamp and a rotating filter are used to illuminate the object being observed. Furthermore, instead of a color camera sensor 44, a monochrome camera sensor is used to capture images of the object being observed. Otherwise, it is the same as in the first embodiment.

[0142] like Figure 25 As shown, in the endoscope system 100 of the second embodiment, a broadband light source 102, a rotating filter 104, and a filter switching unit 105 are provided in the light source device 14 instead of the four-color LEDs 20a to 20d. Furthermore, a monochrome image sensor 106 without a color filter is provided in the imaging optical system 30b instead of the color image sensor 44.

[0143] The broadband light source 102 is a xenon lamp, white LED, etc., emitting white light in a wavelength range from blue to red. The rotating filter 104 contains, from the inside, a filter 107 for normal light mode, and filters 108 for special light mode and disease-related treatment mode (see reference). Figure 26 The filter switching unit 105 moves radially to rotate the filter 104. When the mode switch SW12f is set to normal light mode, the normal light mode filter 107 is inserted into the light path of white light. When the mode switch is set to special light mode or disease-related treatment mode, the special light mode and disease-related treatment mode filter 108 are inserted into the light path of white light.

[0144] like Figure 26As shown, the filter 107 used in normal light mode includes a B filter 107a for transmitting broadband blue light B in white light, a G filter 107b for transmitting broadband green light G in white light, and an R filter 107c for transmitting broadband red light R in white light, arranged circumferentially. Therefore, in normal light mode, by rotating the rotating filter 104, broadband blue light B, broadband green light G, and broadband red light R are alternately irradiated onto the observed object as normal light.

[0145] In the special light mode and disease-related treatment mode, the filter 108 has a Bn filter 108a that transmits a narrow band of blue light in white light and a Gn filter 108b that transmits a narrow band of green light in white light, arranged circumferentially. Therefore, in the special light mode or disease-related treatment mode, by rotating the rotating filter 104, the blue narrow band light and the green narrow band light, which are short wavelengths, alternately illuminate the observed object as special light. Furthermore, it is preferable that the wavelength band of the blue narrow band light is 400–450 nm and the wavelength band of the green narrow band light is 540–560 nm.

[0146] In the endoscope system 100, in normal light mode, the object being observed is photographed by a monochrome camera sensor 106 whenever it is illuminated by broadband blue light B, broadband green light G, and broadband red light R. This yields Bc image signals, Gc image signals, and Rc image signals. Then, a normal image is generated based on these three color image signals using the same method as in the first embodiment described above.

[0147] In the endoscope system 100, during special light mode or disease-related treatment mode, the observed object is photographed by a monochrome camera sensor 106 whenever it is illuminated by narrow-band blue and narrow-band green light. This yields Bs and Gs image signals. Then, a special image is generated based on these two color image signals using the same method as in the first embodiment described above.

[0148] [Third Implementation]

[0149] In the third embodiment, instead of the four colors of LEDs 20a to 20d shown in the first embodiment, a laser light source and a phosphor are used to illuminate the object being observed. Hereinafter, only the parts that differ from the first embodiment will be described, omitting descriptions of the parts that are substantially the same as the first embodiment.

[0150] like Figure 27As shown, in the endoscope system 200 of the third embodiment, in the light source section 20 of the light source device 14, instead of the four-color LEDs 20a to 20d, a violet laser light source section 203 (marked as "405LD"; LD stands for "Laser Diode") that emits a violet laser beam with a center wavelength of 405±10nm, which is equivalent to a short wavelength of light, and a blue laser light source section (marked as "445LD") 204 that emits a blue laser beam with a center wavelength of 445±10nm are provided. The light emission from the semiconductor light-emitting elements of these light source sections 203 and 204 is individually controlled by the light source control section 208.

[0151] In normal light mode, the light source control unit 208 illuminates the blue laser light source unit 204. In contrast, in special light mode or disease-related treatment mode, both the purple laser light source unit 203 and the blue laser light source unit 204 are illuminated simultaneously.

[0152] Furthermore, the half-width at half-maximum (WHM) of the violet, blue, or blue-green laser beam is preferably set to approximately ±10 nm. Moreover, the violet laser source section 203 and the blue laser source section 204 can utilize wide-area InGaN-based laser diodes, and can also utilize InGaNA-based and GaNA-based laser diodes. Furthermore, the aforementioned light source can also be configured using a light-emitting element such as a light-emitting diode.

[0153] In addition to the illumination lens 32, the illumination optical system 30a also includes a phosphor 210 incident upon a blue laser beam or a blue-green laser beam from the light guide 25. The phosphor 210 is excited by the blue laser beam and emits fluorescence. Therefore, the blue laser beam acts as the excitation light. Furthermore, a portion of the blue laser beam does not excite the phosphor 210 and is transmitted. The blue-green laser beam also does not excite the phosphor 210 and is transmitted. The light emitted from the phosphor 210 illuminates the body of the object being observed via the illumination lens 32.

[0154] Here, in the normal light mode, the blue laser beam is mainly incident on the phosphor 210, therefore, as Figure 28 As shown, a blue laser beam and ordinary light, a combination of fluorescence emitted from phosphor 210 excited by the blue laser beam, are irradiated onto the object being observed. An ordinary image composed of Bc, Gc, and Rc image signals is obtained by capturing the object being observed illuminated by the ordinary light using the image sensor 44.

[0155] Furthermore, in special light modes or disease-related treatment modes, both violet and blue laser beams are simultaneously incident on phosphor 210, such as... Figure 29As shown, in addition to the violet and blue laser beams, pseudo-white light emitted by the fluorescence excited from the phosphor 210 via the violet and blue laser beams is also emitted as a special light. A special image composed of Bs, Gs, and Rs image signals is obtained by capturing the observed object illuminated by this special light using the image sensor 44. Furthermore, the pseudo-white light can be a combination of violet (V), blue (B), green (G), and red light emitted from V-LED20a, B-LED20b, G-LED20c, and R-LED20d.

[0156] In addition, phosphor 210 preferably uses a portion of the blue laser beam and includes various phosphors that emit green to yellow light (e.g., YKG-based phosphors or BAM (BaMgAl)). 10 A phosphor is constructed using phosphors such as 017. In this structural example, if a semiconductor light-emitting element is used as the excitation source of the phosphor 210, high-intensity white light can be obtained with high luminous efficiency, the intensity of the white light can be easily adjusted, and the changes in the color temperature and chromaticity of the white light can be suppressed to a lesser extent.

[0157] Furthermore, while the above embodiments apply to endoscopic systems that process endoscopic images, which are considered medical images, the invention can also be applied to medical image processing systems that process medical images other than endoscopic images. Moreover, the invention is also applicable to diagnostic support devices that use medical images to provide diagnostic support to users. Furthermore, using medical images, the invention is also applicable to medical service support devices used to support medical work such as diagnostic reports.

[0158] For example, such as Figure 30 As shown, the diagnostic support device 600 combines medical imaging equipment such as the medical image processing system 602 with a PACS (Picture Archiving and Communication Systems) 604. Furthermore, as... Figure 31 As shown, the medical service support device 610 is connected to various examination devices, such as the first medical image processing system 621, the second medical image processing system 622, ..., the Nth medical image processing system 623, via any network 626. The medical service support device 610 receives medical images from the first to the Nth medical image processing systems 621, 622, ..., 623, and provides medical service support based on the received medical images.

[0159] In the above embodiment, the hardware structure of the processing unit 58, which includes a general image generation unit 62, a special image generation unit 64, a disease-related processing unit 66, a spatial frequency component distribution calculation unit 71, a frequency characteristic region extraction unit 72, a first frequency characteristic region extraction unit 72a, a third frequency characteristic region extraction unit 72b, an analysis object region detection unit 72c, a second frequency characteristic region extraction unit 72d, a structure detection unit 74, a determination unit 76, an image processing implementation control unit 81, a magnification index analysis unit 82, a halo distribution analysis unit 83, a frequency characteristic analysis unit 84, a brightness value analysis unit 85, a shadow distribution analysis unit 86, and a measurement auxiliary light analysis unit 87, is as shown below. Various processors include general-purpose processors that execute software (programs) and function as various processing units, such as CPUs (Central Processing Units), FPGAs (Field Programmable Gate Arrays), and processors whose circuit structure can be changed after manufacturing, such as Programmable Logic Devices (PLDs), as well as processors with circuit structures specifically designed to perform various processes, such as dedicated circuits.

[0160] A processing unit can be composed of one of these various processors, or it can be composed of a combination of two or more processors of the same or different types (e.g., multiple FPGAs, a combination of CPUs and FPGAs). Furthermore, multiple processing units can also be composed of a single processor. As examples of assembling multiple processing units with a single processor, firstly, in the case of a computer such as a client or server, a processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Secondly, in the case of a System-on-Chip (SoC), a processor is used to implement the functionality of the entire system containing multiple processing units using a single IC (Integrated Circuit) chip. Thus, various processing units are constructed as hardware structures using one or more of the aforementioned processors.

[0161] Moreover, more specifically, the hardware architecture of these various processors is an electrical circuit that combines circuit elements such as semiconductor components.

[0162] The present invention can also be implemented in another way as described below.

[0163] In the processor device,

[0164] Medical images of the observed object are acquired through the image acquisition unit.

[0165] The image processing determination unit determines whether to perform image processing on the medical image based on image analysis results obtained by analyzing the photographic conditions and / or the medical image itself.

[0166] The disease-related processing unit performs at least one of the following steps on the medical image that the image processing determination unit has determined to be processed: calculating index values ​​related to the stage of ulcerative colitis based on the density of superficial blood vessels, intramucosal hemorrhage, and extramucosal hemorrhage obtained from the medical image; determining the stage of the ulcerative colitis; and determining whether the ulcerative colitis is in remission or not.

[0167] Symbol Explanation

[0168] 10-Endoscope system, 12-Endoscope, 12a-Insertion section, 12b-Operating section, 12c-Bend section, 12d-Front end, 12e-Angle knob, 12f-Mode switching switch, 12g-Zoom operating section, 14-Light source device, 16-Processor device, 18-Monitor, 19-Control console, 20-Light source unit, 20a-V-LED, 20b-B-LED, 20c-G-LED, 20d-R-LED, 21-Light source control unit, 23-Optical path coupling unit, 25-Light guide, 30a-Illumination optical system, 30b-Camera optical system, 30c-Measuring auxiliary light lens, 30d-Measuring auxiliary light emitting section, 32-Illumination lens, 42-Objective lens, 44-Camera transmission Sensor, 45-Camera Control Unit, 46-CDS / AGC Circuit, 48-A / D Inverter, 50-Image Acquisition Unit, 52-DSP, 54-Noise Reduction Unit, 56-Image Processing Switching Unit, 57-Zoom Control Unit, 58-Image Processing Unit, 60-Video Signal Generation Unit, 62-Ordinary Image Generation Unit, 64-Special Image Generation Unit, 66-Disease-Related Processing Unit, 70-Image Processing Determination Unit, 71-Spatial Frequency Component Distribution Calculation Unit, 72-Frequency Characteristic Region Extraction Unit, 72a-First Frequency Characteristic Region Extraction Unit, 72b-Third Frequency Characteristic Region Extraction Unit, 72c-Analysis Object Region Detection Unit, 72d-Second Frequency Characteristic Region Extraction Unit, 74-Structure Detection Unit, 76-Determination Unit 81-Image Processing Implementation Control Unit; 82-Magnification Index Analysis Unit; 83-Halo Distribution Analysis Unit; 84-Frequency Characteristic Analysis Unit; 85-Brightness Value Analysis Unit; 86-Shadow Distribution Analysis Unit; 87-Measurement Auxiliary Light Analysis Unit; 91, 95-Monitor Images; 91a, 95a, 99, 100, 100a, 100b-Special Images; 91b, 95b-Halo Distribution Images; 91c, 95c-Spatial Frequency Distribution Images; 92-Light Shield; 92a-Light Shield Shadow Area; 93-Magnification Display; 96-Halo Area; 97-High Brightness Area; 98-Low Frequency Area; 101-Endoscope System; 102-Broadband Light Source; 104-Rotating Filter; 105-Filter Switching Unit. 106 - Camera sensor; 107 - Filters for normal light modes; 107a-B filters; 107b-G filters; 107c-R filters; 108 - Filters for special light modes and disease-related processing modes; 108a-Bn filters; 108b-Gn filters; 200 - Endoscopic system; 203 - Violet laser light source unit; 204 - Blue laser light source unit; 208 - Light source control unit; 210 - Phosphor; 600 - Diagnostic support device; 602 - Medical image processing system; 604 - PACS; 610 - Medical business support device; 621 - First medical image processing system; 622 - Second medical image processing system; 623 - Nth medical image processing system; 626 - Network.SP, SP1, SP2 - Spots; Lm - Optical axis; Rx - Range; Px - Proximal end; Py - Near center; Pz - Distal end; Qx, Qy, Qz - Imaging range; P1 - Position; Ax - Optical axis; Dv - Observation distance.

Claims

1. An image processing apparatus comprising a processor, the processor acquires a medical image obtained by capturing an observation target with an endoscope, determines whether to perform image processing on the medical image using an index of a distance between the endoscope and the observation target at the time of capturing the medical image obtained from an image analysis result obtained by analyzing a photographic condition of the medical image and / or the medical image, for the medical image on which the image processing has been determined to be performed, performs at least one of a step of calculating an index value related to a stage of ulcerative colitis, a step of determining the stage of the ulcerative colitis, and a step of determining remission or non-remission of the ulcerative colitis, from a density of a superficial blood vessel, intramucosal bleeding, and extramucosal bleeding obtained from the medical image.

2. The image processing apparatus according to claim 1, wherein the photographic condition of the medical image is a magnification index of the medical image.

3. The image processing apparatus according to claim 1, wherein the image analysis result is at least one of a halo distribution, a frequency characteristic, a luminance value, and a shadow distribution obtained from the medical image.

4. The image processing apparatus according to claim 2, wherein the image analysis result is at least one of a halo distribution, a frequency characteristic, a luminance value, and a shadow distribution obtained from the medical image.

5. The image processing apparatus according to claim 1, wherein the medical image is obtained by capturing the observation target on which a measurement auxiliary light for performing measurement of the observation target is irradiated, the image analysis result is a position of a measurement auxiliary light irradiation region formed on the observation target in the medical image.

6. The image processing apparatus according to claim 2, wherein the medical image is obtained by capturing the observation target on which a measurement auxiliary light for performing measurement of the observation target is irradiated, the image analysis result is a position of a measurement auxiliary light irradiation region formed on the observation target in the medical image.

7. The image processing apparatus according to claim 3, wherein the medical image is obtained by capturing the observation target on which a measurement auxiliary light for performing measurement of the observation target is irradiated, the image analysis result is a position of a measurement auxiliary light irradiation region formed on the observation target in the medical image.

8. The image processing apparatus according to any one of claims 1 to 7, wherein for the medical image on which the image processing has been determined to be performed, the processor performs at least one of a step of calculating the index value related to the stage of ulcerative colitis, a step of determining the stage of the ulcerative colitis, and a step of determining remission or non-remission of the ulcerative colitis, from an image analysis result obtained by analyzing a photographic condition of the medical image and / or the medical image.

9. The image processing apparatus according to any one of claims 1 to 7, wherein When the remission or non-remission of the ulcerative colitis is determined by the processor, the processor classifies the density of the superficial blood vessels, the intramucosal bleeding, and the extramucosal bleeding from the frequency characteristics or the luminance values obtained from the medical image, and determines the remission or non-remission of the ulcerative colitis according to the classification.

10. The image processing apparatus according to any one of claims 1 to 7, wherein When the remission or non-remission of the ulcerative colitis is determined by the processor, the processor: calculates a spatial frequency component distribution from the medical image, extracts a first frequency characteristic region having a first frequency characteristic, a second frequency characteristic region having a second frequency characteristic higher than the first frequency characteristic, and a third frequency characteristic region having a third frequency characteristic higher than the second frequency characteristic, from the spatial frequency component distribution, detects the density of the superficial blood vessels, the intramucosal bleeding, and the extramucosal bleeding from the first frequency characteristic region in which a first region discrimination process using a luminance value is implemented, the second frequency characteristic region in which a second region discrimination process using a luminance value is implemented, and the third frequency characteristic region, determines the remission or non-remission of the ulcerative colitis according to the density of the superficial blood vessels, the intramucosal bleeding, and the extramucosal bleeding.

11. The image processing apparatus according to any one of claims 1 to 7, wherein The medical image is obtained by photographing the observation object illuminated by an illumination light including light of a short wavelength.

12. The image processing apparatus according to claim 8, wherein The medical image is obtained by photographing the observation object illuminated by an illumination light including light of a short wavelength.

13. The image processing apparatus according to claim 9, wherein The medical image is obtained by photographing the observation object illuminated by an illumination light including light of a short wavelength.

14. The image processing apparatus according to claim 10, wherein The medical image is obtained by photographing the observation object illuminated by an illumination light including light of a short wavelength.

15. The image processing apparatus according to claim 11, wherein The illumination light is a violet light in which a center wavelength or a peak wavelength includes 410 nm.

16. The image processing apparatus according to any one of claims 12 to 14, wherein The illumination light is a violet light in which a center wavelength or a peak wavelength includes 410 nm.

17. A method of operating an image processing apparatus provided with a processor, wherein the method comprises the following steps performed by the processor: an image acquisition step of acquiring a medical image obtained by photographing an observation object with an endoscope; an image processing determination step of determining whether to perform image processing on the medical image using an index of a distance between the endoscope and the observation object at the time of photographing the medical image, which is obtained based on an image analysis result obtained by analyzing a photographic condition of the medical image or the medical image; and and In the determination step, at least one of the following steps is performed based on the density of superficial blood vessels, intramucosal bleeding, and extramucosal bleeding obtained from the medical image on which the image processing has been performed: a step of calculating an index value related to the stage of ulcerative colitis; a step of determining the stage of the ulcerative colitis; or a step of determining remission or non-remission of the ulcerative colitis.

18. The method of operating an image processing apparatus according to claim 17, wherein, In the determination step, when the processor determines remission or non-remission of the ulcerative colitis, The processor has: In the determination step, a step of calculating a spatial frequency component distribution from the medical image; a step of extracting a first frequency characteristic region having a first frequency characteristic, a second frequency characteristic region having a second frequency characteristic higher than the first frequency characteristic, and a third frequency characteristic region having a third frequency characteristic higher than the second frequency characteristic, from the spatial frequency component distribution; a step of detecting the density of superficial blood vessels, intramucosal bleeding, and extramucosal bleeding, based on the first frequency characteristic region on which first region discrimination processing using a luminance value has been performed, the second frequency characteristic region on which second region discrimination processing using a luminance value has been performed, and the third frequency characteristic region; and a step of determining remission or non-remission of the ulcerative colitis, based on the density of superficial blood vessels, intramucosal bleeding, and extramucosal bleeding.

19. The method of operating an image processing apparatus according to claim 17 or 18, wherein The medical image is obtained by capturing the observation object illuminated by illumination light that is violet light having a central wavelength or peak wavelength containing 410 nm.

Images