Medical image processing device, endoscope system, working method of medical image processing device, and non-transitory computer-readable medium

Abstract

The present invention provides a medical image processing device, an endoscope system, a working method of the medical image processing device, and a program for the medical image processing device, which are capable of identifying a reduction in the visual recognition of an emphasized display. The processor device (16) includes an image signal acquisition unit (50), a display control unit (58), and an image processing unit (64) in a region of interest detection mode. The image signal acquisition unit (50) acquires an image signal from an endoscope (12). The image processing unit (64) in a region of interest detection mode detects a region of interest from an endoscopic image. The display control unit (58) causes the emphasized display of the region of interest to be displayed by overlapping it on the endoscopic image. The visual recognition of the emphasized display is determined based on the image information of the endoscopic image and the emphasized display, and the determination result is notified to the user.

Description

Technical Field

[0001] The present invention relates to a medical image processing device for detecting a region of interest such as a lesion, an endoscope system, a working method of the medical image processing device, and a non-transitory computer-readable medium. Background Art

[0002] In the medical field, medical images such as endoscopic images, X-ray images, CT (Computed Tomography) images, and MR (Magnetic Resonance) images are used for diagnostic imaging, such as diagnosing patient conditions and conducting regular observations. Doctors and others use these diagnostic images to determine treatment plans.

[0003] In recent years, diagnostic imaging using medical images has been increasingly focused on analyzing medical images to automatically detect regions of interest (ROIs) within organs, such as lesions and tumors. In particular, the accuracy of ROI detection has dramatically improved through machine learning techniques such as deep learning.

[0004] Patent Documents 1 and 2 describe medical image processing devices that perform image processing based on the detection information when a region of interest, such as a lesion, is detected from a medical image. The medical image processing devices described in Patent Documents 1 and 2 perform emphasis processing in which an emphasis display for emphasizing the region of interest is superimposed on the medical image.

[0005] Previous technical literature

[0006] Patent Literature

[0007] Patent Document 1: International Publication No. 2018 / 198161

[0008] Patent Document 2: International Publication No. 2017 / 081976 Summary of the Invention

[0009] Technical issues to be solved by the invention

[0010] However, the medical image processing devices described in Patent Documents 1 and 2 do not take into account the visual recognition of the highlighted display. Therefore, depending on the color of the subject in the medical image, the presence or absence of objects within the subject, etc., the highlighted display may sometimes blend in with the surrounding area or become less noticeable relative to the surrounding area. As a result, if the visual recognition of the highlighted display is reduced, the doctor may not be able to notice the area of interest.

[0011] The object of the present invention is to provide a medical image processing device, an endoscope system, a working method of the medical image processing device, and a non-temporary computer-readable medium that can identify the reduced visual recognition of the emphasized display.

[0012] Means for solving technical problems

[0013] The present invention is a medical image processing device equipped with a processor, wherein the processor performs the following processing: acquiring a medical image; detecting a region of interest from within the medical image; setting an emphasis display for emphasizing the detected region of interest, and causing the emphasis display to overlap on the medical image for display; determining the visual recognizability of the emphasis display based on image information acquired from the medical image in which the region of interest is detected and the set emphasis display; and notifying the user of the result of the visual recognizability determination.

[0014] The processor preferably acquires image information from the inner side of the highlighted display in the medical image. Alternatively, the processor preferably acquires image information from the outer side of the highlighted display in the medical image.

[0015] The processor preferably obtains a color difference between the medical image and the highlighted display based on color information calculated from the image information and color information calculated from the highlighted display, and determines visual recognition based on the color difference. The processor preferably calculates an average value calculated from the image information as the color information.

[0016] The processor preferably displays a frame-shaped graphic surrounding the region of interest as a highlight display, and determines visual recognition based on the thickness of a line of the frame-shaped graphic relative to the region of interest. Furthermore, the processor preferably displays a frame-shaped graphic surrounding the region of interest as a highlight display, and determines visual recognition based on a similarity of the frame shape relative to the region of interest.

[0017] The processor preferably displays the determination result on a display screen. Furthermore, the processor preferably calculates a numerical index value as the determination result of visual recognition and displays it on the display screen. Furthermore, when the index value is below a predetermined threshold, the processor preferably displays the index value as a notification.

[0018] The processor preferably uses the color difference calculated from the image information and the highlighted display as an index value. Furthermore, the processor may calculate an index value that quantifies the result of the visual recognition determination and display identification information or an identification pattern corresponding to the index value.

[0019] The processor preferably determines visual recognition based on the presence or absence of an object outside the detection target within the highlighted display. The processor further preferably determines that an object outside the detection target is present when an area ratio of a portion of the highlighted display having a brightness or luminance above a second threshold relative to a range within the highlighted display in the medical image is above a third threshold.

[0020] The processor may cause the determination result to be displayed on a display screen different from the display screen displaying the medical image. When the indicator value is below a predetermined first threshold, the processor may automatically save the medical image in which the region of interest was detected. When the indicator value is below the predetermined first threshold, the processor may issue a warning to the user.

[0021] An endoscope system of the present invention includes a light source device, an endoscope, a processor, and a display.

[0022] The processor performs the following processing: acquiring a medical image; detecting a region of interest from within the medical image; setting an emphasis display for emphasizing the detected region of interest, and causing the emphasis display to overlap with the medical image and be displayed on the display; determining the visual recognition of the emphasis display based on image information acquired from the medical image in which the region of interest was detected and the set emphasis display; and notifying the user of the result of the visual recognition determination. The light source device emits illumination light for illuminating the observation object. The endoscope has an imaging sensor for photographing the observation object illuminated by the illumination light. The display displays a medical image obtained by signal processing the image signal output by the imaging sensor.

[0023] The working method of the medical image processing device of the present invention includes the following steps: acquiring a medical image; detecting a region of interest from the acquired medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display overlapping the medical image; determining the visual recognizability of the emphasis display based on image information acquired from the medical image in which the region of interest is detected and the set emphasis display; and notifying the user of the determination result of the visual recognizability.

[0024] The non-transitory computer-readable medium of the present invention stores a program executable by a computer, wherein the program executable by the computer is used to enable the computer to function as a medical image processing device that acquires medical images and performs image processing on the medical images. The program executable by the computer enables the computer to implement the following functions: acquire a medical image; detect a region of interest from within the medical image; set an emphasis display for emphasizing the detected region of interest, and display the emphasis display overlapping the medical image; determine the visual recognizability of the emphasis display based on image information acquired from the medical image in which the region of interest is detected and the set emphasis display; and notify the user of the result of the visual recognizability determination.

[0025] Effects of the Invention

[0026] According to the present invention, the user can recognize that the visibility of the emphasized display is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is an external view of the endoscope system.

[0028] Figure 2 This is a block diagram showing the functions of the endoscope system according to the first embodiment including a plurality of LED light sources.

[0029] Figure 3 Graph showing the spectral spectra of violet light V, blue light B, blue light Bx, green light G, and red light R.

[0030] Figure 4 This is a graph showing the spectral spectrum of normal light in the first embodiment.

[0031] Figure 5 This is a graph showing the spectral spectrum of the special light according to the first embodiment.

[0032] Figure 6 This is a block diagram showing the functions of the image processing unit and the display control unit in the region of interest detection mode.

[0033] Figure 7 This is an explanatory diagram showing an emphasis region set when the display control unit performs emphasis display of the region of interest.

[0034] Figure 8 This is an explanatory diagram for explaining a state in which the visibility determination unit calculates color information based on image information acquired from an endoscopic image and setting information for highlight display, and determines the visibility of highlight display.

[0035] Figure 9 This is an example of a display screen when the display control unit performs highlight display of the region of interest and display of notification information.

[0036] Figure 10 This is a flowchart showing a series of processes in the region of interest detection mode.

[0037] Figure 11 1 and 2 are explanatory diagrams showing display states, showing an example in which a lesion is detected from an endoscopic image (A) and a graphic is superimposed on the endoscopic image as a highlight display (B).

[0038] Figure 12 1 and 2 are explanatory diagrams showing display states, and are explanatory diagrams showing an example (B) of displaying notification information when (A) color difference is reduced and visibility is reduced.

[0039] Figure 13 These are explanatory diagrams showing display states in the second embodiment, and are explanatory diagrams showing examples of displaying identification information in a case where visibility is low (A) and a case where visibility is high (B).

[0040] Figure 14 These are explanatory diagrams showing display states in the third embodiment, and are explanatory diagrams showing examples of displaying identification patterns in a case where visibility is low (A) and a case where visibility is high (B).

[0041] Figure 15 1 is an explanatory diagram showing a display state in the fourth embodiment, showing an example in which a main image and a sub-image are displayed on one display screen, and an identification pattern is superimposed on the sub-image.

[0042] Figure 16 It is an explanatory diagram showing a display state in the fifth embodiment, and is an explanatory diagram showing an example (B) of displaying an identification pattern when (A) there is an object outside the detection target inside the pattern.

[0043] Figure 17 This is an explanatory diagram showing an example of a modified example in which an emphasis display is formed by four L-shaped figures surrounding a lesion.

[0044] Figure 18 It is an explanatory diagram showing a display state in the sixth embodiment.

[0045] Figure 19 It is an explanatory diagram showing a display state in the seventh embodiment. DETAILED DESCRIPTION

[0046] [First embodiment]

[0047] like Figure 1 As shown, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18 (display unit), and a console 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion portion 12a for insertion into a subject, an operating portion 12b provided at the base end of the insertion portion 12a, a bending portion 12c provided at the distal end of the insertion portion 12a, and a distal end portion 12d. Operating the angle knob 13a of the operating portion 12b bends the bending portion 12c. This bending operation directs the distal end portion 12d in the desired direction.

[0048] The distal end portion 12d has an illumination window, an observation window, an air / water supply nozzle, and a forceps outlet (none shown) on its front end. The illumination window is used to illuminate the observed area. The observation window is used to capture light from the observed area. The air / water supply nozzle is used to clean the illumination window and observation window. The forceps outlet is used to perform various procedures using instruments such as forceps and an electric scalpel.

[0049] In addition to the angle button 13a, the operation unit 12b is provided with a still image acquisition unit 13b for acquiring a still image, a mode switching unit 13c for switching between observation modes, and a zoom operation unit 13d for changing the zoom factor. The still image acquisition unit 13b can be used to freeze a still image of the observation object displayed on the display 18 and to release the still image to store it in memory.

[0050] The endoscope system 10 has observation modes: normal mode, special mode, and region of interest detection mode. When the observation mode is normal mode, normal light is emitted, which is a combination of multiple colors of light at a light intensity ratio Lc used for normal mode. When the observation mode is special mode, special light is emitted, which is a combination of multiple colors of light at a light intensity ratio Ls used for special mode.

[0051] Furthermore, when the observation mode is the region of interest detection mode, illumination light for the region of interest detection mode is emitted. In this embodiment, normal light is emitted as the illumination light for the region of interest detection mode, but special light may also be emitted.

[0052] The processor device 16 is electrically connected to a display 18 and a console 19. The display 18 outputs an image of the observation object or information accompanying the image. The console 19 functions as a user interface for accepting input operations such as designation of a region of interest (ROI) and function settings.

[0053] like Figure 2 As shown, the light source device 14 includes a light source unit 20 that emits illumination light for illuminating the observation object, and a light source control unit 22 that controls the light source unit 20. The light source unit 20 is a semiconductor light source such as LEDs (Light Emitting Diodes) of various colors. The light source control unit 22 controls the amount of illumination light by turning the LEDs on and off and adjusting the drive current or voltage of the LEDs. Furthermore, the light source control unit 22 controls the wavelength band of the illumination light by, for example, changing the optical filter.

[0054] In the first embodiment, the light source unit 20 includes four color LEDs, namely, a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d, and a wavelength cutoff filter 23. Figure 3 As shown, the V-LED 20a emits purple light V in a wavelength band of 380 nm to 420 nm.

[0055] B-LED 20b emits blue light B in the wavelength range of 420 nm to 500 nm. Wavelength cutoff filter 23 cuts off at least the wavelengths longer than the peak wavelength of 450 nm in the blue light B emitted by B-LED 23b. As a result, the blue light Bx that passes through wavelength cutoff filter 23 falls within the wavelength range of 420 to 460 nm. Light in the wavelength range longer than 460 nm is cut off because this wavelength range is responsible for reducing the contrast of the blood vessels being observed. Alternatively, wavelength cutoff filter 23 can attenuate light in the wavelength range longer than 460 nm, rather than blocking it.

[0056] The G-LED 20c emits green light G in a wavelength range of 480 nm to 600 nm. The R-LED 20d emits red light R in a wavelength range of 600 nm to 650 nm. The center wavelength and peak wavelength of light emitted from the LEDs 20a to 20d may be the same or different.

[0057] The light source control unit 22 independently controls the lighting and extinguishing of each LED 20a-20d, as well as the amount of light emitted when on, to adjust the timing, duration, amount of light, and spectrum of the illumination light. The lighting and extinguishing control by the light source control unit 22 varies for each observation mode. Furthermore, the reference brightness can be set using the brightness setting unit of the light source device 14 or the console 19.

[0058] In the normal mode or the region of interest detection mode, the light source control unit 22 turns on all of the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. Figure 4As shown, the light quantity ratio Lc among the violet light V, blue light B, green light G, and red light R is set so that the peak light intensity of the blue light Bx is greater than the peak light intensity of any of the violet light V, green light G, and red light R. Thus, in the normal mode or the region of interest detection mode, the multicolor light for the normal mode or the region of interest detection mode, including the violet light V, blue light Bx, green light G, and red light R, is emitted from the light source device 14 as the normal light. The normal light has an intensity greater than a predetermined value from the blue band to the red band, and is therefore substantially white.

[0059] In the special mode, the light source control unit 22 turns on all of the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. Figure 5 As shown, regarding the light quantity ratio Ls among the purple light V, blue light B, green light G, and red light R, the peak light intensity of the purple light V is set to be greater than the peak light intensity of any one of the blue light Bx, green light G, and red light R. Furthermore, the peak light intensity of the green light G and the red light R is set to be smaller than the peak light intensity of the purple light V and the blue light Bx. Thus, in the special mode, multi-color light for the special mode including the purple light V, blue light Bx, green light G, and red light R is emitted from the light source device 14 as special light. In the special light, since the proportion of the purple light V is relatively large, it becomes light with a blue hue. In addition, the special light does not need to include light of all four colors, as long as it includes light from at least one color of the four colors of LEDs 20a to 20d. Furthermore, the special light preferably has a main wavelength region below 450nm, such as a peak wavelength or a center wavelength.

[0060] like Figure 2 As shown, the illumination light emitted by the light source unit 20 passes through an optical path coupling unit (not shown) formed by a reflector or lens, etc., and enters a light guide 24 inserted into the insertion portion 12a. The light guide 24 is built into the endoscope 12 and a universal cord, and transmits the illumination light to the distal end portion 12d of the endoscope 12. The universal cord is a cord that connects the endoscope 12 to the light source device 14 and the processor device 16. Furthermore, a multimode optical fiber can be used as the light guide 24. As an example, a thin-diameter optical fiber cable with a core diameter of 105 μm, a cladding diameter of 125 μm, and a protective layer having a diameter of 0.3 mm to 0.5 mm (including the outer sheath) can be used for the light guide 24.

[0061] The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30a includes an illumination lens 32. Illumination light propagating through the light guide 24 via the illumination lens 32 illuminates the object under observation. The imaging optical system 30b includes an objective lens 34, a magnifying optical system 36, and an imaging sensor 38 (corresponding to the "imaging unit" of the present invention). Various types of light, such as reflected light, scattered light, and fluorescence from the object under observation, pass through the objective lens 34 and magnifying optical system 36 and enter the imaging sensor 38. As a result, an image of the object under observation is formed on the imaging sensor 38.

[0062] The magnifying optical system 36 includes a zoom lens 36a for magnifying an observation object, and a lens driving unit 36b for moving the zoom lens 36a in the optical axis direction CL. The zoom lens 36a moves freely between a telephoto end and a wide-angle end according to zoom control by the lens driving unit 36b, thereby magnifying or reducing the observation object imaged on the imaging sensor 38.

[0063] The imaging sensor 38 is a color imaging sensor that captures an object illuminated by illumination light. Each pixel of the imaging sensor 38 is provided with one of an R (red), G (green), or B (blue) filter. The imaging sensor 38 receives light ranging from violet to blue at B pixels provided with the B filter, receives green light at G pixels provided with the G filter, and receives red light at R pixels provided with the R filter. Each pixel then outputs an image signal for each of the RGB colors. The imaging sensor 38 transmits the output image signal to the CDS circuit 40.

[0064] In normal mode or region of interest detection mode, the imaging sensor 38 captures an observation object illuminated by normal light, outputting a Bc image signal from the B pixel, a Gc image signal from the G pixel, and an Rc image signal from the R pixel. Furthermore, in special mode, the imaging sensor 38 captures an observation object illuminated by special light, outputting a Bs image signal from the B pixel, a Gs image signal from the G pixel, and an Rs image signal from the R pixel.

[0065] The image sensor 38 can be a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. Furthermore, instead of the image sensor 38 having primary color filters of RGB, a complementary color image sensor having complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) can be used. When using a complementary color image sensor, image signals of the four colors of CMYG are output. Therefore, through complementary color-to-primary color conversion, the four CMYG image signals can be converted into three RGB image signals, thereby obtaining the same RGB image signals as the image sensor 38. Furthermore, a monochrome sensor without color filters can be used instead of the image sensor 38.

[0066] The CDS circuit 40 performs correlated double sampling (CDS) on the analog image signal received from the imaging sensor 38. The image signal passing through the CDS circuit 40 is input to the AGC circuit 42. The AGC circuit 42 performs automatic gain control (AGC) on the input image signal. The A / D (Analog to Digital) conversion circuit 44 converts the analog image signal passing through the AGC circuit 42 into a digital image signal. The A / D conversion circuit 44 inputs the A / D-converted digital image signal to the processor device 16.

[0067] like Figure 2 As shown, the processor device 16 includes an image signal acquisition unit 50 , a DSP (Digital Signal Processor) 52 , a noise reduction unit 54 , an image processing unit 56 , and a display control unit 58 .

[0068] The processor device 16 includes the function of serving as a medical image processing device. As described later, in the image processing unit 56, an endoscopic image is acquired and a region of interest within the observation object is detected from the endoscopic image. In the display control unit 58, the region of interest is highlighted and displayed on the endoscopic image 75.

[0069] The image signal acquisition unit 50 acquires digital image signals corresponding to the observation mode from the endoscope 12. In normal mode or region of interest detection mode, the image signal acquisition unit 50 acquires Bc, Gc, and Rc image signals. In special mode, the image signal acquisition unit 50 acquires Bs, Gs, and Rs image signals. In region of interest detection mode, the image signal acquisition unit 50 acquires one frame of Bc, Gc, and Rc image signals when illuminated by normal light, and one frame of Bs, Gs, and Rs image signals when illuminated by special light.

[0070] The DSP 52 performs various signal processing operations on the image signal acquired by the image signal acquisition unit 50, including defect correction, offset processing, DSP gain correction, linear matrix processing, gamma conversion, and demosaicing. The defect correction process corrects signals from defective pixels in the imaging sensor 38. The offset process removes dark current components from the defect-corrected image signal and sets an accurate zero level. The DSP gain correction process adjusts the signal level by multiplying the offset-processed image signal by a specific DSP gain.

[0071] Linear matrix processing improves the color reproducibility of image signals that have undergone DSP gain correction. Gamma conversion adjusts the brightness and saturation of linear matrix processed image signals. Demosaicing (also known as isotropic processing or synchronization processing) is performed on gamma-converted image signals to interpolate and generate signals for the colors missing in each pixel. This demosaicing ensures that all pixels have signals for each RGB color. The noise reduction unit 54 applies noise reduction processing, such as a moving average method or a median filter, to the image signals that have undergone demosaicing by the DSP 52 to reduce noise. The noise-reduced image signals are input to the image processing unit 56.

[0072] The image processing unit 56 includes a normal mode image processing unit 60, a special mode image processing unit 62, and a region of interest detection mode image processing unit 64. When set to normal mode, the normal mode image processing unit 60 performs color conversion, color emphasis, and structure emphasis on the received Bc, Gc, and Rc image signals. The color conversion process is performed by applying 3x3 matrix processing, grayscale conversion, and a three-dimensional LUT (Look Up Table) to the RGB image signals.

[0073] Color emphasis processing is performed on the RGB image signals that have undergone color conversion. Structure emphasis processing, which emphasizes the structure of the object being observed, is performed on the RGB image signals that have undergone color emphasis processing. By performing the various image processing steps described above, a normal image can be obtained. A normal image is obtained based on a well-balanced normal light source composed of violet light V, blue light Bx, green light G, and red light R, resulting in an image with natural colors. The normal image is input to the display control unit 58.

[0074] The special mode image processing unit 62 operates when set to special mode. In the special mode image processing unit 62, color conversion processing, color emphasis processing, and structure emphasis processing are performed on the received Bs, Gs, and Rs image signals. The color conversion, color emphasis, and structure emphasis processing are the same as those of the normal mode image processing unit 60. By performing the various image processing steps described above, a special image can be obtained. A special image is an image obtained based on special light, in which the amount of violet light V emitted due to the high absorption coefficient of hemoglobin in blood vessels is greater than that of other colors, such as blue light Bx, green light G, and red light R. Therefore, the resolution of vascular and glandular duct structures is higher than that of other structures. The special image is input to the display control unit 58.

[0075] The image processing unit 64 operates when the region of interest detection mode is set. Figure 6 As shown, the region of interest detection mode image processing unit 64 includes a detection image processing unit 70, a region of interest detection unit 71, a visibility determination unit 72, and a visibility notification control unit 73. The detection image processing unit 70 performs the same image processing as the normal mode image processing unit 60, such as color conversion processing, on the received Bc image signal, Gc image signal, and Rc image signal, thereby sequentially acquiring an endoscopic image 75.

[0076] The region of interest detection unit 71 performs image analysis on the endoscopic image 75 and performs a region of interest detection process to detect a region of interest within the observation object. In this embodiment, the region of interest detection unit 71 detects a lesion (e.g., a tumor or inflammation) within the observation object as the region of interest. In this case, the region of interest detection unit 71 first divides the endoscopic image 75 into a plurality of small regions, for example, into square regions of a plurality of pixels. Next, the region of interest detection unit 71 calculates image features from the divided endoscopic image 75. Based on the calculated features, each small region is identified as a lesion. A machine learning algorithm such as a convolutional neural network or deep learning is preferably used for this identification process.

[0077] Furthermore, the feature quantity calculated from the endoscopic image 75 by the region of interest detection unit 71 is preferably the shape or color of a predetermined portion of the observation object, or a value derived from such shape or color. For example, the feature quantity is preferably at least one of vascular density, vascular shape, the number of vascular branches, vascular thickness, vascular length, vascular tortuosity, vascular invasion depth, glandular duct shape, glandular duct opening shape, glandular duct length, glandular duct tortuosity, and color information, or a combination of two or more of these.

[0078] Finally, a small area of the same type is extracted as a lesion. The region of interest detection unit 71 associates the position, size, and type of the extracted lesion with the endoscopic image 75 as detection information 76. The region of interest detection mode image processing unit 64 outputs the endoscopic image 75 associated with the detection information 76 to the display control unit 58.

[0079] The display control unit 58 performs display control for displaying the image or data from the image processing unit 56 on the display 18. When the normal mode is set, the display control unit 58 performs control for displaying a normal image on the display 18. When the special mode is set, the display control unit 58 performs control for displaying a special image on the display 18.

[0080] When the region of interest detection mode is set, the display control unit 58 emphasizes and displays the region of interest detected by the region of interest detection unit 71 on the endoscopic image 75. When emphasizing and displaying the region of interest, the display control unit 58 first sets an emphasis region for emphasizing the region of interest based on the endoscopic image 75 output from the region of interest detection mode image processing unit 64 and the detection information 76 associated with the endoscopic image 75.

[0081] like Figure 7 As shown, based on detection information 76 such as the position, size, and type of lesion 77, display control unit 58 sets an emphasis region 78 that is wider than lesion 77 and includes lesion 77. In this embodiment, a square region is set as emphasis region 78. For example, emphasis region 78 has a square periphery that is set at a predetermined distance from the periphery of lesion 77. However, emphasis region 78 is not limited to this; a square region that contacts the periphery of lesion 77 may also be set.

[0082] The display control unit 58 emphasizes the emphasis area 78 set as described above. Specifically, the display control unit 58 overlays and displays a graphic for emphasis at the location of the emphasis area 78 within the endoscopic image 75. In this embodiment, the display control unit 58 displays a square frame (frame-shaped) graphic 79 surrounding the lesion 77 based on the location of the emphasis area 78. After setting the emphasis area 78, the display control unit 58 resets the emphasis area 78 based on the amount of movement of the lesion 77 within the endoscopic image 75 and displays the graphic 79 based on the reset location of the emphasis area 78.

[0083] Furthermore, the highlighted graphic 79 is displayed differently from the other parts of the endoscopic image 75. For example, the display control unit 58 usually displays the graphic 79 in a color different from the hue of a color that is commonly included in the endoscopic image. Furthermore, the color of the graphic 79 can also be set based on user input.

[0084] The display control unit 58 outputs setting information 81 for the highlighted graphic 79 to the image processing unit 56. The setting information 81 includes information such as positional information and color information of the graphic 79 relative to the endoscopic image 75. Furthermore, the setting information 81 includes information about the original endoscopic image 75 in which the lesion 77 was detected.

[0085] The visibility determination unit 72 determines the visibility of the highlighted display based on the image information obtained from the endoscopic image 75 in which the lesion 77 is detected and the highlighted display setting information 81 set by the display control unit 58, and calculates a numerical index value as the determination result. In this embodiment, the visibility determination unit 72 calculates color information based on the image information obtained from the endoscopic image 75 in which the lesion 77 is detected by the region of interest detection unit 71 and the highlighted display setting information 81. Based on this color information, the color difference between the endoscopic image 75 and the graphic 79 is calculated as the index value. The color information represents information related to color, such as hue, lightness, and saturation.

[0086] like Figure 8 As shown, when calculating color information based on image information obtained from the endoscopic image 75, the visibility determination unit 72 calculates the inside of the highlighted display including the lesion 77, specifically, the area 82 surrounded by the graphic 79 in the endoscopic image 75 (see also Figure 7) as color information. As described above, since the setting information 81 includes the position information of the graphic 79, the color information can be calculated by cutting out the area 82 surrounded by the graphic 79 from the endoscopic image 75 based on the position information. On the other hand, when calculating color information based on the setting information for highlight display, the average value of the graphic 79 is calculated as the color information. Based on this color information, the visibility determination unit 72 calculates the color difference between the endoscopic image 75 and the graphic 79.

[0087] The color difference calculation in the visual recognition determination unit 72 can be performed using, for example, a color difference formula based on CIEDE2000 specified in JIS Z 8730 7.3. This allows the use of a standardized calculation method to determine a color difference that is consistent with human visual characteristics. Furthermore, when using the color difference formula based on CIEDE2000, information from the CIELab color space, consisting of an L component representing lightness, an a component representing the degree of red or green, and a b component representing the degree of yellow or blue, is used as color information for determining the color difference.

[0088] The calculation method for obtaining color difference is not limited to the above method, and any calculation method that takes human vision into consideration may be used. For example, the calculation may be performed using the Euclidean distance in the CIELab color space (also known as CIE76).

[0089] The visibility notification control unit 73 notifies the user of the determination result made by the visibility determination unit 72. Figure 9 As shown, the visibility notification control unit 73 outputs the color difference, which is the determination result calculated as described above, as notification information 83 to the display control unit 58, and displays it on the display screen 84 of the display 18. In the present embodiment, when the color difference is less than or equal to a preset first threshold value, the visibility notification control unit 73 outputs the color difference information as notification information 83 to the display control unit 58.

[0090] In the visibility notification control unit 73, for example, a preset first threshold value of 2.0 is set. According to the aforementioned JIS regulations, when the color difference is around 1, the difference between two colors is discernible when viewed side by side, while when the color difference is 2 to 3, the difference is noticeable when the two colors are viewed separately. As described above, by setting the first threshold value to 2.0 based on the JIS regulations, the visibility notification control unit 73 can notify users of reduced visibility.

[0091] Next, in the region of interest detection mode, the image processing unit 56 and the display control unit 58 determine the visual recognition of the emphasized display, and the process of displaying the determination result on the display screen 84 of the display 18 is as follows. Figure 10 The flowchart shown and Figure 11 The following diagram illustrates the process. A doctor, acting as a user, operates the mode switch 13c to switch to the region of interest detection mode. This causes the region of interest detection mode illumination light to illuminate the observation object. The imaging sensor 38 captures the observation object illuminated by the region of interest detection mode illumination light, thereby acquiring an endoscopic image 75. When the region of interest detection mode is switched, the display control unit 58 sequentially acquires the endoscopic images 75 (S11) and displays them in real time on the display screen 84 of the monitor 18.

[0092] During real-time display in the region of interest detection mode, a region of interest detection process is performed on the acquired endoscopic image 75 to detect a region of interest within the observation object by the region of interest detection unit 71. If a region of interest is detected (YES in S12), detection information 76 generated by the region of interest detection unit 71 is associated with the endoscopic image 75 and output.

[0093] Then, if Figure 11 As shown in (A), when a lesion 77 as a region of interest is detected within the observation object, that is, when the detection information 76 is associated with the endoscopic image 75, the display control unit 58 uses the detection information 76 associated with the endoscopic image 75, especially the information on the position and size of the lesion 77, to set the emphasis area 78 (S13).

[0094] After setting the emphasis area 78, as shown in FIG. Figure 11 As shown in FIG. 5B , the display control unit 58 displays a graphic 79 as an emphasis display superimposed on the position of the emphasis area 78 in the endoscopic image 75 ( S14 ), and outputs setting information 81 of the graphic 79 to the image processing unit 56 . Figures 7 to 9 、 Figure 11 and Figure 12 For ease of explanation, the color difference between the pattern 79 and other parts in the endoscopic image 75 is indicated by the presence or absence of shading. On the other hand, if the lesion 77 is not detected in the observation object (No in S12), it is naturally not highlighted.

[0095] The information of the original endoscopic image 75 in which the lesion 77 was detected is marked in the setting information 81 of the graphic 79. Therefore, the visual recognition determination unit 72 reads the original endoscopic image 75 and calculates the color information based on the image information obtained from the endoscopic image 75 and the setting information 81 of the graphic 79, thereby determining the visual recognition of the emphasized display (S15). As described above, the color difference calculated based on the color information is compared with the first threshold value to determine the visual recognition. Figure 12 As shown in (A), when highlighting graphic 79, graphic 79 may blend in with its surroundings or become less noticeable relative to its surroundings, depending on the color of the subject within endoscopic image 75, the presence or absence of objects within the subject, and other factors, thereby reducing its visibility. In such cases, the color difference between endoscopic image 75 and graphic 79 is typically reduced.

[0096] When the color difference is below the first threshold value (yes in S16), as shown in FIG. Figure 12 As shown in (B), the color difference information is output as notification information 83 to the display control unit 58. The display control unit 58 displays notification information 83 on the display screen 84 and notifies the user of the reduced visibility (S17). If the color difference exceeds the first threshold (No in S16), the visibility determination unit 72 does not make any notification.

[0097] As described above, the visual recognizability of the emphasized display in the endoscopic image 75 is determined, and notification is given when the visual recognizability of the emphasized display is reduced. Therefore, the doctor who is the user can recognize that the visual recognizability of the emphasized display is reduced, thereby avoiding the situation where the user fails to notice the area of interest such as the lesion.

[0098] [Second embodiment]

[0099] In the first embodiment, as the result of the visual recognition determination, the example of displaying the color difference information on the display screen is given, but the present invention is not limited to this, and identification information may be displayed based on the index value as the determination result. Figure 13 , an example is shown in which identification information 85A and 85B corresponding to the color difference as the determination result is displayed on the display screen 84.

[0100] In this embodiment, color information is calculated based on the image information obtained from the endoscopic image 75 and the setting information 81 for highlighting the display, and the color difference between the endoscopic image 75 and the graphic 79 is calculated based on this color information. The process of comparing the color difference with the first threshold value is the same as that of the first embodiment described above. Then, when the color difference is below the first threshold value, the visual recognition determination unit 72 outputs the identification information 85A to the display control unit 58. The display control unit 58 displays the identification information 85A on the display screen 84 and notifies the user of the reduced visual recognition. Figure 13 In the example shown in (A), character information such as “low visibility” is displayed as identification information 85A. Thus, as in the first embodiment described above, the doctor as a user can recognize that the visibility of the highlighted display is reduced.

[0101] In addition, as a modification of this embodiment, identification information may be displayed not only when the color difference is below the first threshold, but also when the color difference exceeds the first threshold, that is, when the visual recognition is high. Figure 13 As shown in (B), when the color difference exceeds the first threshold, the visibility determination unit 72 outputs identification information 85B to the display control unit 58. The display control unit 58 displays the identification information 85B on the display screen 84 and notifies that the visibility is high. Figure 13 In the example shown in (B), character information such as "high visual recognition" is displayed as identification information 85B.

[0102] Furthermore, the setting for displaying identification information based on color difference is not limited to the two-level setting of the case below the first threshold or the case exceeding the first threshold, and can also be set to three or more levels. For example, the color difference is pre-set to have three levels of numerical values of 2.0 or less, greater than 2.0 and less than 4.0, and greater than 4.0, and the visual recognition determination unit 72 makes a determination based on this setting. Then, the case where the visual recognition is low when the color difference is less than 2.0, the visual recognition is medium when the color difference is greater than 2.0 and less than 4.0, and the visual recognition is high when the color difference is greater than 4.0 is displayed on the display screen 84 as identification information. In this case, it is preferable to display character information such as "low visual recognition", "medium visual recognition", and "high visual recognition" according to the level of color difference as identification information.

[0103] [Third embodiment]

[0104] In the second embodiment, as the result of the visual recognition determination, an example of displaying identification information corresponding to the color difference as the determination result is given, but the present invention is not limited to this. Figure 14 As shown in the example shown, an identification pattern corresponding to the color difference as the determination result may be displayed.

[0105] In this embodiment, color information is calculated based on the image information obtained from the endoscopic image 75 and the setting information 81 for highlighting the display, and the color difference between the endoscopic image 75 and the graphic 79 is calculated based on this color information. The process of comparing the color difference with the first threshold value is the same as in the first and second embodiments described above. Then, when the color difference is below the first threshold value, the information of the icon 86A is output to the display control unit 58. The display control unit 58 displays the icon 86A as an identification graphic on the display screen 84 and notifies the user of the reduced visual recognition. Figure 14 In the example shown in (A), a mark imitating a sign indicating the presence of danger is displayed as icon 86A. Thus, as in the first and second embodiments described above, the doctor serving as the user can recognize that the visibility of the emphasized display has been reduced.

[0106] In addition, as a modification of this embodiment, the identification pattern may be displayed not only when the color difference is below the first threshold, but also when the color difference exceeds the first threshold, that is, when the visual recognition is high. Figure 14 As shown in (B), when the color difference exceeds the first threshold, the visibility determination unit 72 outputs information of the icon 86B to the display control unit 58. The display control unit 58 displays the icon 86B as an identification graphic on the display screen 84 and notifies that the visibility is high. Figure 14 In the example shown in (B) , a double circle mark is displayed as the icon 86B.

[0107] Furthermore, as with the second embodiment described above, the setting for displaying identification information based on color difference can be set to three or more levels. For example, as with the second embodiment described above, three levels of numerical values are pre-set, and the visual recognition determination unit 72 makes a determination based on this setting. Then, when the color difference is 2.0 or less, the visual recognition is low; when the color difference is greater than 2.0 and less than 4.0, the visual recognition is medium; and when the color difference is greater than 4.0, the visual recognition is high, which is displayed as an identification graphic on the display screen 84. In this case, it is preferable to display an icon with a different shape according to the level of color difference as the identification graphic.

[0108] [Fourth embodiment]

[0109] In each of the above-mentioned embodiments, an image with a highlighted display superimposed on an endoscopic image is displayed on one display screen, and notification information, etc. is displayed in a non-display area of the endoscopic image. However, an image with superimposed notification information, etc. may also be displayed on a display screen different from a display screen for displaying a normal endoscopic image.

[0110] exist Figure 15In the example shown, two display screens are displayed side by side on a single display 18. A normal endoscopic image 87 can be displayed as a main image with a larger display area, and an image 88 with a highlighted display superimposed on the endoscopic image can be displayed as a sub-image with a smaller display area than the main image. Furthermore, an icon 86A serving as identification information is superimposed on the highlighted image 88. In this embodiment, the normal endoscopic image 87 refers to the endoscopic image 75 itself, acquired by the image processing unit 56 in the region of interest detection mode in the aforementioned embodiments, without any superimposed graphics or the like for highlighting.

[0111] As in the aforementioned embodiments, a highlighted image 88 is superimposed on the endoscopic image 75, and the highlighted graphic 79 and the like are superimposed on the endoscopic image 75. Then, as in the aforementioned embodiments, color information is calculated based on the image information acquired from the endoscopic image 75 and the highlighted display setting information 81. Based on this color information, the color difference between the endoscopic image 75 and the graphic 79 is calculated and compared with a first threshold. If the color difference is below the first threshold, information about an icon 86A is output to the display control unit 58. The display control unit 58 further superimposes the icon 86A on the highlighted image 88 and notifies the user of the reduced visibility.

[0112] In addition, the information displayed as the result of the visual recognition determination is not limited to the icon 86A. As in the above-mentioned embodiments, information on the color difference as an indicator value or identification information corresponding to the color difference may be displayed, or identification information or identification graphics that differ according to the color difference may be displayed. Figure 15 In the example shown, two display screens are displayed side by side on one display 18 , but the main image and the sub-image may be displayed on different displays.

[0113] Furthermore, in the first to fourth embodiments described above, when calculating the color difference between the endoscopic image 75 and the highlighted graphic 79, the visibility determination unit 72 calculates color information based on the interior of the graphic 79. However, this is not limiting and the visibility determination unit 72 may also calculate color information based on the exterior of the graphic 79. Specifically, the visibility determination unit 72 may use the average value of the portion of the endoscopic image 75 excluding the graphic 79 and the area 82 surrounded by the graphic 79 as the color information. As described above, since the setting information 81 includes the positional information of the graphic 79, the color information can be calculated by excluding the exterior of the graphic 79 from the endoscopic image 75 based on the positional information. Thus, even if the color difference between the highlighted graphic 79 and the exterior of the graphic 79 decreases, the doctor, acting as a user, can recognize that the visibility of the highlighted display has decreased.

[0114] Furthermore, in the above-mentioned first to fourth embodiments, the visual recognition determination unit 72 uses a pre-set first threshold value to determine the color difference between the endoscopic image 75 and the emphasized display, but the first threshold value used for the determination is not always the same value, and the first threshold value can be weighted according to the thickness of the emphasized display line.

[0115] For example, the display control unit 58 can change the thickness of the line of the graphic 79 displayed as a highlight according to the size of the region of interest, or can change the thickness of the line of the graphic 79 according to a user input operation. In this manner, if the thickness of the line of the graphic 79 can be changed, when the visibility determination unit 72 obtains the setting information for highlighting, it weights the first threshold value according to the thickness of the line of the graphic 79. As the weighting relative to the first threshold value, the first threshold value is set to be smaller in inverse proportion to the thickness of the line of the graphic 79.

[0116] For example, when the thickness of the line of graphic 79 is at the initial setting, the first threshold is 2.0. When the thickness of the line of graphic 79 is thicker than the initial setting, the first threshold is set to less than 2.0. When the thickness of the line of graphic 79 is thinner than the initial setting, the first threshold is set to greater than 2.0. With respect to the visibility of the highlighted display, the thicker the thickness of the line of graphic 79, the higher the visibility. Therefore, even if the first threshold is set to a smaller value depending on the thickness of the line of graphic 79, the doctor as a user can still recognize the reduced visibility of the highlighted display, as in the above-described embodiments.

[0117] [Fifth embodiment]

[0118] In the above-mentioned embodiments, the visibility determination unit 72 uses the color difference calculated based on the image information of the endoscopic image 75 and the setting information of the emphasized display as an index value for determining the visibility. However, the present invention is not limited to this, and the visibility determination may be performed based on the presence or absence of an object outside the detection target inside the emphasized display. In this case, for example, Figure 16 As shown in (A), in endoscopic image 75, objects 89 or phenomena outside the detection target, such as water, halos, bubbles, pigment, and other substances other than the lesion 77, may sometimes intrude into the highlighted graphic 79. When acquiring color information, the portion of these objects or phenomena outside the detection target 89 that is reflected in endoscopic image 75 typically has high brightness or luminance. Therefore, in this embodiment, brightness values are used as color information acquired from the endoscopic image. However, this is not limiting, and brightness values may also be used as color information acquired from the endoscopic image.

[0119] To detect the presence of an object 89 or phenomenon outside the detection target within the pattern 79, the visibility determination unit 72 compares the brightness of each pixel within the area 82 surrounded by the pattern 79 with a second threshold. This second threshold is set to a high brightness value, assuming the presence of water, halos, bubbles, etc. Therefore, when the brightness is above the second threshold, there is a high probability that an object 89 or phenomenon outside the detection target is reflected.

[0120] The visibility determination unit 72 also compares the area ratio of the portion with a brightness greater than or equal to the second threshold value (the ratio of the area of the portion with a brightness greater than or equal to the second threshold value to the area of the entire range 82) with a third threshold value for the range 82 surrounded by the pattern 79. The third threshold value is set, for example, at an area ratio of 50%, assuming that the portion with high brightness is relatively large relative to the range 82.

[0121] Then, if Figure 16 As shown in (B), when the area ratio of the portion with a brightness exceeding the second threshold is greater than or equal to the third threshold, the visibility determination unit 72 determines that an object 89 or phenomenon outside the detection target exists inside the pattern 79, i.e., that visibility is reduced, and outputs information about icon 86A to the display control unit 58. The display control unit 58 superimposes icon 86A on the endoscopic image 75 to notify the user of the reduced visibility. The information displayed as a result of the visibility determination is not limited to icon 86A. Information about the area ratio of the portion with a brightness exceeding the second threshold, identification information corresponding to the area ratio, or identification information or an identification pattern different from the area ratio may also be displayed.

[0122] In the above embodiments, the highlighted graphic is a square frame, but the present invention is not limited thereto and may be any frame shape other than a rectangle (quadrilateral), such as a polygon, circle, or ellipse that can surround the region of interest.

[0123] Furthermore, the graphics displayed as an emphasis are not limited to a single frame shape surrounding the region of interest, but may be composed of a plurality of shapes. Figure 17 In the example shown, the display control unit 58 arranges four L-shaped graphics 91A to 91D surrounding the lesion 77 at the corners of the emphasis area 78 as an emphasis display. Figure 17 In FIG. 1 , the two-dot chain line is shown for convenience of explaining the arrangement of the L-shaped figures 91A to 91D, and is not actually shown.

[0124] exist Figure 17In the example shown, as shown in the above-mentioned first to fourth embodiments, when the color difference calculated based on the image information of the endoscopic image 75 and the setting information of the emphasized display is used as an indicator value as a judgment of visual recognizability, it is preferred that the visual recognizability judgment unit 72 calculates the average value of the four L-shaped figures 91A to 91D as color information, calculates the color difference based on the color information of the endoscopic image 75 and the color information of the average value of the L-shaped figures 91A to 91D, and compares it with the first threshold value.

[0125] While not limited to this, the visibility determination unit 72 calculates color information for each of the four L-shaped patterns 91A to 91D, calculates four color differences based on the color information of the endoscopic image 75 and the color information of each of the L-shaped patterns 91A to 91D, and compares these with a first threshold. In this case, for example, if any of the four color differences is below the first threshold, the visibility is determined to be low. If the color difference is below the first threshold, the display control unit 58 outputs information such as color difference notification information 83, identification information, and identification patterns. Subsequently, notification is performed in the same manner as in the aforementioned embodiments.

[0126] Furthermore, if the color difference or other indicator value calculated based on the image information of the endoscopic image 75 and the highlighted display setting information is below a pre-set first threshold, or if the area ratio of the portion within the highlighted display whose brightness or luminance exceeds a second threshold is above a third threshold, the image processing unit 56 determines that visibility is low, notifies the user in the same manner as in the aforementioned embodiments, and automatically saves the endoscopic image in which the region of interest was detected. This allows the user to subsequently confirm the endoscopic image with reduced visibility while the region of interest is detected, thereby reliably preventing the user from failing to notice the region of interest, such as a lesion. Furthermore, as described above, the endoscopic image determined to have low visibility and in which the region of interest is detected can be stored in, for example, a storage device provided in the processor device 16 or a cloud server.

[0127] Furthermore, as described above, when it is determined that the visibility is low, the image processing unit 56 not only notifies the user but also issues a warning by sounding, lighting an indicator, or flashing a portion of the screen.

[0128] [Sixth embodiment]

[0129] In the above embodiments, the visibility determination unit 72 determines visibility based on color difference information or the presence or absence of an object other than the detection target. However, the present invention is not limited thereto and the determination may be made based on the thickness of the line highlighting the region of interest. Figure 18This is an example of a display screen when the visibility determination unit 72 determines the visibility of the highlighted display based on the thickness of the line for highlighting the region of interest.

[0130] In this embodiment, the visual recognition determination unit 72 calculates as an index the ratio of the thickness T1 of the line of the graphic 79 to the maximum size LM of the lesion 77 detected from the endoscopic image 75 by the region of interest detection unit 71. The maximum size LM of the lesion 77 refers to, for example, the size of the largest portion of the lesion 77 in either the X-axis direction or the Y-axis direction of the endoscopic image 75. Figure 18 In the example shown, the dimension of the largest portion of the lesion 77 in the X-axis direction is defined as the maximum dimension LM.

[0131] As a judgment of visual recognition, the visual recognition judgment unit 72 compares the ratio of the thickness T1 of the line of the graphic 79 to the maximum size LM of the lesion 77 with a threshold value, and judges that the visual recognition is low when the ratio is below the threshold value. Then, similarly to the above-mentioned embodiments, information such as notification information, identification information, and identification graphics is output to the display control unit 58. Thereafter, notification is performed in the same manner as the above-mentioned embodiments. Figure 18 In the example shown, text information such as "low visibility" is displayed as identification information 92. As the ratio of line thickness of the graphic for the region of interest decreases, the visibility decreases. Therefore, as in the above-described embodiments, the doctor as a user can recognize the reduced visibility of the highlighted display.

[0132] [Seventh embodiment]

[0133] In the sixth embodiment, the visual recognition determination unit 72 determines the visual recognition based on the thickness of the line used to highlight the region of interest. However, the present invention is not limited thereto. A frame-shaped graphic surrounding the region of interest may be displayed as a highlight display, and the visual recognition may be determined based on the similarity of the frame shape to the region of interest. Figure 19 This is an example of a display screen when the visibility of the emphasized display is determined based on the similarity of the frame shape to the region of interest.

[0134] In the present embodiment, the visual recognition determination unit 72 analyzes the similarity between the shape of the lesion 77 detected from the endoscopic image 75 by the region of interest detection unit 71 and the frame-shaped graphic 93 surrounding the lesion 77. In addition, the graphic 93 is a circular frame-shaped graphic that surrounds the lesion 77 and contacts multiple locations on the periphery of the lesion 77. In the case of determining visual recognition based on similarity, for example, the visual recognition determination unit 72 analyzes the similarity between the contour shape of the lesion 77 and the inner peripheral shape of the graphic 93 by a well-known method such as template matching, and compares the similarity with a threshold value. When the similarity is above the threshold value, it is determined that the visual recognition is low. Then, as in the above-mentioned embodiments, information such as notification information, identification information, and identification graphics is output to the display control unit 58. Thereafter, notification is performed in the same manner as in the above-mentioned embodiments. Figure 19 In the example shown, text information such as "low visibility" is displayed as identification information 92. As the similarity of the frame shape to the region of interest increases, the visibility decreases. Therefore, as in the above embodiments, the doctor as a user can recognize the reduced visibility of the highlighted display.

[0135] In each of the above-described embodiments, the display control unit 58 superimposes a frame-shaped graphic at the location of the emphasis area. However, this is not limiting. Alternatively, the color of the emphasis area may be changed for emphasis display. In this case, when the lesion 77 is detected as the region of interest and the emphasis area is set, the display control unit 58 may select a color different from the original color of the emphasis area for emphasis display, for example, a color that is prevalent in the endoscopic image 75, and change the emphasis area to a color different from the rest of the endoscopic image 75. The different color here refers to, for example, a color with a different hue.

[0136] Furthermore, the emphasized display of the emphasized area is not limited to the above-described methods; any image processing that allows visual distinction from the surrounding area, such as color saturation change processing, contrast processing, negative / positive inversion processing, and filtering, may be used. Alternatively, the emphasized display based on image processing of the emphasized area and the emphasized display based on the graphic surrounding the lesion in each of the above-described embodiments may be combined.

[0137] In the above embodiments, the observation object is illuminated using four-color LEDs 20a to 20d. However, laser beams and phosphors may also be used. Furthermore, in the above embodiments, the observation object is illuminated using four-color LEDs 20a to 20d. However, a white light source such as a xenon lamp and a rotating filter may also be used. Furthermore, a monochrome imaging sensor may be used instead of a color imaging sensor 38 to capture an image of the observation object.

[0138] In addition, in the above-mentioned embodiment, the medical image processing device of the present invention is applied to an endoscope system that obtains endoscopic images as medical images, but it goes without saying that it can also be applied to various endoscope systems such as capsule endoscopes. As other medical images, the medical image processing device of the present invention can also be applied to various medical image devices that obtain X-ray images, CT images, MR images, ultrasonic images, pathological images, PET (Positron Emission Tomography) images, etc.

[0139] In the above-described embodiment, the hardware configuration of processing units (processing units) that perform various processes, such as the image processing unit 56 and the display control unit 58, is composed of various processors as described below. These processors include general-purpose processors such as CPUs (Central Processing Units) that execute software (programs) to function as various processing units, GPUs (Graphical Processing Units), FPGAs (Field Programmable Gate Arrays), and other processors with circuit configurations that can be modified after manufacturing, such as programmable logic devices (PLDs), as well as processors with circuit configurations specifically designed to perform various processes, such as dedicated circuits.

[0140] One processing unit can be composed of one of these various processors, or a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, etc.). In addition, multiple processing units can be composed of one processor. As an example of how multiple processing units are composed of one processor, there is the following method: as represented by a computer such as a client or a server, one processor is composed of a combination of one or more CPUs and software, and the processor functions as multiple processing units. Second, there is the following method: as represented by a system on chip (SoC), a processor is used that implements the functions of the entire system including multiple processing units using one IC (Integrated Circuit) chip. In this way, various processing units are constructed as a hardware structure using one or more of the above-mentioned various processors.

[0141] Furthermore, more specifically, the hardware structure of these various processors is a circuit (circuitry) formed by combining circuit elements such as semiconductor elements.

[0142] Explanation of symbols

[0143] 10-Endoscope system, 12-Endoscope, 12a-Insertion portion, 12b-Operation portion, 12c-Bending portion, 12d-Front end portion, 13a-Angle knob, 13b-Still image acquisition portion, 13c-Mode switching portion, 13d-Zoom operation portion, 14-Light source device, 16-Processor device, 18-Display, 19-Console, 20-Light source portion, 20a-V-LED, 20b-B-LED, 20c-G-LED, 20d-R-LED, 22-Light source control portion, 23-Wavelength cutoff filter, 24-Light guide, 30a-Illumination optical system, 30b-Image pickup optical system, 32-Illumination lens, 34-Objective lens, 36-Magnification optical system, 36a-Zoom lens, 36b-Lens drive portion, 38-Image pickup sensor, 40-CDS circuit, 42-AGC circuit, 44-A / D converter Switching circuit, 50-image signal acquisition unit, 52-DSP, 54-noise reduction unit, 56-image processing unit, 58-display control unit, 60-normal mode image processing unit, 62-special mode image processing unit, 64-region of interest detection mode image processing unit, 70-detection image processing unit, 71-region of interest detection unit, 72-visual recognition determination unit, 73-visual recognition notification control unit, 75-endoscopic image, 76-detection information, 77-lesion part, 78-emphasized area, 79-graphics, 81-setting information, 82-range, 83-notification information, 84-display screen, 85A-identification information, 85B-identification information, 86A-icon, 86B-icon, 87-normal endoscopic image, 88-overlapping emphasized display image, 89-object outside the detection target, 91A~91D-L-shaped graphics.

Claims

1. A medical image processing device comprising a processor, wherein: The processor performs the following processing: Acquiring medical images; detecting a region of interest from within the medical image; Setting an emphasis display for emphasizing the detected region of interest, and causing the emphasis display to be overlapped on the medical image for display; determining visual recognition of the emphasized display based on image information acquired from the medical image in which the region of interest is detected and the set emphasized display; and Notifying the user of the visual recognition determination result, The processor displays a frame-shaped graphic surrounding the region of interest as the emphasized display, and determines the visual recognition based on the thickness of a line of the frame-shaped graphic for the region of interest.

2. The medical image processing apparatus according to claim 1, wherein: The processor acquires the image information from the inner side of the emphasized display in the medical image.

3. The medical image processing apparatus according to claim 1, wherein: The processor acquires the image information from outside the emphasized display in the medical image.

4. The medical image processing apparatus according to any one of claims 1 to 3, wherein: The processor obtains a color difference between the medical image and the emphasized display based on color information calculated from the image information and color information calculated from the emphasized display, and determines the visual recognition based on the color difference.

5. The medical image processing apparatus according to claim 4, wherein: The processor calculates an average value calculated from the image information as the color information.

6. The medical image processing apparatus according to any one of claims 1 to 3, wherein: The processor determines the visual recognition based on a similarity of the frame shape to the region of interest.

7. The medical image processing apparatus according to any one of claims 1 to 3, wherein: The processor causes the determination result to be displayed on a display screen.

8. The medical image processing apparatus according to any one of claims 1 to 3, wherein: The processor calculates a numerical index value as the visual recognition determination result and displays the numerical index value on a display screen.

9. The medical image processing apparatus according to claim 8, wherein: When the index value is equal to or less than a preset threshold value, the processor displays the index value as the notification.

10. The medical image processing apparatus according to claim 8, wherein: The processor uses a color difference calculated from the image information and the emphasized display as the index value.

11. The medical image processing apparatus according to any one of claims 1 to 3, wherein: The processor calculates an index value that quantifies the result of the visual recognition determination, and displays identification information or an identification pattern corresponding to the index value.

12. The medical image processing apparatus according to claim 1, wherein: The processor determines the visibility based on the presence or absence of an object outside the detection target that exists inside the emphasized display.

13. The medical image processing apparatus according to claim 12, wherein: When the area ratio of the portion whose brightness or luminance is above the second threshold value within the highlighted display in the medical image is above the third threshold value relative to the range of the inner side of the highlighted display, the processor determines that there is an object outside the detection object.

14. The medical image processing apparatus according to claim 7, wherein: The processor causes the determination result to be displayed on a display screen different from a display screen displaying the medical image.

15. The medical image processing apparatus according to claim 8, wherein: When the index value is below a preset first threshold, the processor automatically saves the medical image in which the region of interest is detected.

16. The medical image processing apparatus according to claim 8, wherein: When the indicator value is below a preset first threshold, the processor issues a warning to the user.

17. An endoscope system comprising: a light source device for emitting illumination light for illuminating an observation object; an endoscope having an imaging sensor that captures an image of an observation object illuminated by the illumination light; processor; and a display for displaying a medical image obtained by processing the image signal output by the imaging sensor; The processor performs the following processing: acquiring the medical image; detecting a region of interest from within the medical image; Setting an emphasis display for emphasizing the detected region of interest, and causing the emphasis display to overlap with the medical image and be displayed on the display; determining visual recognition of the emphasized display based on image information acquired from the medical image in which the region of interest is detected and the set emphasized display; and Notifying the user of the visual recognition determination result, The processor displays a frame-shaped graphic surrounding the region of interest as the emphasized display, and determines the visual recognition based on the thickness of a line of the frame-shaped graphic for the region of interest.

18. A method for operating a medical image processing device, comprising the following steps: Acquiring medical images; detecting a region of interest from the acquired medical image; Setting an emphasis display for emphasizing the detected region of interest, and causing the emphasis display to be overlapped on the medical image for display; determining visual recognition of the emphasized display based on image information acquired from the medical image in which the region of interest is detected and the set emphasized display; and Notifying the user of the visual recognition determination result, Here, a frame-shaped graphic surrounding the region of interest is displayed as the emphasized display, and the visual recognition is determined based on the thickness of a line of the frame-shaped graphic for the region of interest.

19. A non-transitory computer-readable medium storing a computer-executable program, the computer-executable program being configured to cause the computer to function as a medical image processing device that acquires and processes medical images, the computer-executable program causing the computer to implement the following functions: acquiring the medical image; detecting a region of interest from within the medical image; Setting an emphasis display for emphasizing the detected region of interest, and causing the emphasis display to be overlapped on the medical image for display; determining visual recognition of the emphasized display based on image information acquired from the medical image in which the region of interest is detected and the set emphasized display; and Notifying the user of the visual recognition determination result, Here, a frame-shaped graphic surrounding the region of interest is displayed as the emphasized display, and the visual recognition is determined based on the thickness of a line of the frame-shaped graphic for the region of interest.

Images