Medical image processing device, endoscope system, operation method of medical image processing device, and program for medical image processing device

Abstract

Provided are a medical image processing apparatus whereby a decrease in visibility of highlighting can be recognized, an endoscope system, an operation method for a medical image processing apparatus, and a program for a medical image processing apparatus.　A processor device (16) comprises an image signal acquisition unit (50), a display control unit (58), and a region-of-interest detection mode image processing unit (64). The image signal acquisition unit (50) acquires an image signal from an endoscope (12). The region-of-interest detection mode image processing unit (64) detects a region of interest from the endoscope image. The display control unit (58) displays highlighting of a region of interest so as to be superimposed on the endoscope image. The visibility of the highlighting is determined from the highlighting and image information of the endoscope image, and the determination result is communicated to a user.

Description

[Technical field]

[0001] The present invention relates to a medical image processing device for detecting a region of interest such as a lesion, an endoscope system, an operation method of the medical image processing device, and a program for the medical image processing device. [Background technology]

[0002] In the medical field, endoscopic images, X-ray images, CT (Computed Tomography) images, MR (Magnetic Resonance Medical images such as CT scans are used to diagnose and monitor the progress of patients' illnesses. Doctors and other medical professionals use these images to make decisions about treatment.

[0003] In recent years, in image diagnosis using medical images, it has become common to analyze medical images and automatically detect areas of interest that require careful observation, such as lesions and tumors in organs. In particular, the accuracy of detecting areas of interest has improved dramatically by using machine learning such as deep learning.

[0004] Patent Documents 1 and 2 describe medical image processing devices that, when a region of interest such as a lesion is detected from a medical image, perform image processing based on the detection information. The medical image processing devices described in Patent Documents 1 and 2 perform emphasis processing in which an emphasis display for emphasizing the region of interest is superimposed on the medical image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2018 / 198161 [Patent Document 2] International Publication No. 2017 / 081976 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the medical image processing devices described in Patent Documents 1 and 2 do not take into consideration the visibility of the highlighting, so the highlighting may blend in with the surroundings or may not stand out against the surroundings depending on the color of the subject in the medical image, the presence or absence of an object in the subject, etc. If the visibility of the highlighting is reduced in this way, there is a possibility that a doctor will not notice the region of interest.

[0007] An object of the present invention is to provide a medical image processing device, an endoscope system, an operating method of a medical image processing device, and a program for a medical image processing device, which are capable of recognizing a decrease in visibility of highlighting. [Means for solving the problem]

[0008] The present invention is a medical image processing device equipped with a processor, which acquires a medical image, detects an area of ​​interest within the medical image, sets an emphasis display to highlight the detected area of ​​interest, displays the emphasis display superimposed on the medical image, determines the visibility of the emphasis display from image information acquired from the medical image in which the area of ​​interest is detected and the set emphasis display, and notifies the user of the visibility determination result.

[0009] The processor preferably obtains image information from inside the highlight in the medical image, or alternatively, the processor preferably obtains image information from outside the highlight in the medical image.

[0010] The processor preferably obtains a color difference between the medical image and the highlighted display from color information calculated from the image information and color information calculated from the highlighted display, and determines visibility from the color difference. The processor preferably calculates an average value calculated from the image information as the color information.

[0011] The processor displays a frame-shaped figure surrounding the region of interest as the highlighting, and ShapeIt is preferable that the processor displays a frame-shaped figure surrounding the region of interest as the highlighting display, and determines the visibility from the similarity of the frame shape to the region of interest.

[0012] The processor preferably displays the determination result on the display screen. The processor also preferably calculates a quantified index value as the visibility determination result and displays the quantified index value on the display screen, and more preferably, when the index value is equal to or less than a preset threshold, displays the index value as a notification.

[0013] The processor preferably uses the color difference calculated from the image information and the highlighted display as the index value. The processor may also calculate an index value that quantifies the visibility determination result, and display identification information or an identification figure according to the index value.

[0014] The processor preferably determines visibility based on the presence or absence of an object outside the detection target that exists inside the highlighting. It is more preferable that the processor determines that an object outside the detection target exists when an area ratio of a portion inside the highlighting that has a lightness or luminance equal to or greater than a second threshold value to a range inside the highlighting in the medical image is equal to or greater than a third threshold value.

[0015] The processor may display the determination result on a display screen different from the display screen that displays the medical image. When the index value is equal to or less than a first threshold value set in advance, the processor preferably automatically saves the medical image in which the region of interest is detected, and when the index value is equal to or less than the first threshold value set in advance, the processor preferably issues a warning to the user.

[0016] The endoscope system of the present invention includes a light source device, an endoscope, a processor, and Monitor With The processor acquires a medical image, detects an area of ​​interest within the medical image, sets an emphasis display to highlight the detected area of ​​interest, superimposes the emphasis display on the medical image and displays it on a monitor, determines the visibility of the emphasis display from image information acquired from the medical image in which the area of ​​interest is detected and the emphasis display that has been set, and notifies the user of the visibility determination result. The light source device emits illumination light for illuminating the observation object. The endoscope has an imaging sensor that captures an image of the observation object illuminated with the illumination light. The monitor displays a medical image obtained by signal processing the image signal output by the imaging sensor.

[0017] The operating method of the medical image processing device of the present invention includes the steps of acquiring a medical image, detecting an area of ​​interest from the acquired medical image, setting an emphasis to highlight the detected area of ​​interest and displaying the emphasis superimposed on the medical image, determining the visibility of the emphasis from image information acquired from the medical image in which the area of ​​interest has been detected and the set emphasis, and notifying a user of the visibility determination result.

[0018] The program for a medical image processing device of the present invention is installed in a medical image processing device that acquires medical images and performs image processing on the medical images, and provides the computer with the following functions: acquiring medical images; detecting an area of ​​interest from within the medical image; setting an emphasis display to highlight the detected area of ​​interest and displaying the emphasis display superimposed on the medical image; determining the visibility of the emphasis display from the image information acquired from the medical image in which the area of ​​interest has been detected and the emphasis display that has been set; and notifying the user of the visibility determination result. Effect of the Invention

[0019] According to the present invention, the user can recognize the decrease in visibility of the highlighted display. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is an external view of an endoscope system. [Diagram 2]1 is a block diagram showing the functions of an endoscope system according to a first embodiment having a plurality of LED light sources. [Diagram 3] 1 is a graph showing the optical spectra of purple light V, blue light B, blue light Bx, green light G, and red light R. [Figure 4] 4 is a graph showing the spectrum of normal light in the first embodiment. [Diagram 5] 4 is a graph showing the spectrum of special light in the first embodiment. [Figure 6] 11 is a block diagram showing functions of an attention area detection mode image processing unit and a display control unit. FIG. [Figure 7] 11 is an explanatory diagram showing an emphasis region that is set when a display control unit performs emphasis display of an attention region; FIG. [Figure 8] 11 is an explanatory diagram illustrating a state in which a visibility determination unit calculates color information from image information acquired from an endoscopic image and highlighting setting information, and determines the visibility of highlighting. FIG. [Figure 9] 13 is a diagram illustrating an example of a display screen when a display control unit highlights an attention area and displays notification information. [Figure 10] 13 is a flowchart showing a sequence of operations in an attention area detection mode. [Figure 11] 1A is an explanatory diagram showing a display state, and FIG. 1B is an explanatory diagram showing an example in which a lesion is detected from an endoscopic image and a figure is superimposed on the endoscopic image as a highlighted display. [Figure 12] 1A is an explanatory diagram showing a display state when the color difference is reduced and visibility is reduced, and FIG. 1B is an explanatory diagram showing an example in which notification information is displayed. [Figure 13] 13A and 13B are explanatory diagrams showing display states in a second embodiment, illustrating examples in which identification information is displayed when visibility is low (A) and when visibility is high (B). [Figure 14] 13A and 13B are explanatory diagrams showing display states in a third embodiment, illustrating examples of how an identification graphic is displayed when visibility is low (A) and when visibility is high (B). [Figure 15]FIG. 13 is an explanatory diagram showing a display state in a fourth embodiment, and is an explanatory diagram showing an example in which a main image and a sub-image are displayed on one display screen and an identification graphic is superimposed on the sub-image. [Figure 16] 13A and 13B are explanatory diagrams showing display states in the fifth embodiment, and are explanatory diagrams showing an example in which an identification figure is displayed when an object that is not a detection target is inside the figure (A) and an example in which an identification figure is displayed (B). [Figure 17] FIG. 13 is an explanatory diagram showing an example of a modified example in which highlighting is configured from four L-shaped figures surrounding a lesion area. [Figure 18] FIG. 13 is an explanatory diagram showing a display state in the sixth embodiment. [Figure 19] FIG. 23 is an explanatory diagram showing a display state in the seventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] [First embodiment] As shown in Fig. 1, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18 (display unit), and a console 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion section 12a to be inserted into a subject, an operation section 12b provided at a base end portion of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip side of the insertion section 12a. The bending section 12c is bent by operating an angle knob 13a of the operation section 12b. This bending operation causes the tip section 12d to be oriented in a desired direction.

[0022] The tip portion 12d has an illumination window, an observation window, an air / water nozzle, and a forceps outlet on its tip surface (none of which are shown). The illumination window is for applying illumination light to the observation site. The observation window is for taking in light from the observation site. The air / water nozzle is for cleaning the illumination window and observation window. The forceps outlet is for performing various treatments using forceps and treatment tools such as an electric scalpel.

[0023] In addition to the angle knob 13a, the operation unit 12b is provided with a still image acquisition unit 13b used to acquire a still image, a mode switching unit 13c used to switch between observation modes, and a zoom operation unit 13d used to change the zoom magnification. The still image acquisition unit 13b is capable of a freeze operation for displaying a still image of the observation target on the monitor 18, and a release operation for saving the still image in storage.

[0024] The endoscope system 10 has three observation modes: a normal mode, a special mode, and an area of ​​interest detection mode. When the observation mode is the normal mode, normal light is emitted in which multiple colors of light are combined at a light intensity ratio Lc for the normal mode. When the observation mode is the special mode, special light is emitted in which multiple colors of light are combined at a light intensity ratio Ls for the special mode.

[0025] When the observation mode is the attention area detection mode, illumination light for the attention area detection mode is emitted. In this embodiment, normal light is emitted as the illumination light for the attention area detection mode, but special light may be emitted.

[0026] The processor device 16 is electrically connected to a monitor 18 and a console 19. The monitor 18 outputs and displays an image of an object to be observed, information associated with the image, etc. The console 19 functions as a user interface that accepts input operations such as designation of a region of interest (ROI) and function settings.

[0027] 2, the light source device 14 includes a light source section 20 that emits illumination light used to illuminate an object to be observed, and a light source control section 22 that controls the light source section 20. The light source section 20 is a semiconductor light source such as a multi-colored LED (Light Emitting Diode). The light source control section 22 controls the amount of emitted illumination light by turning on / off the LED or adjusting the drive current or drive voltage of the LED. The light source control section 22 also controls the wavelength band of the illumination light by changing an optical filter or the like.

[0028] In the first embodiment, the light source unit 20 has four color LEDs, a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d, and a wavelength cut filter 23. As shown in Fig. 3, the V-LED 20a emits purple light V in a wavelength band of 380 nm to 420 nm.

[0029] The B-LED 20b emits blue light B in a wavelength band of 420 nm to 500 nm. At least the wavelength longer than the peak wavelength of 450 nm of the blue light B emitted from the B-LED 23b is cut by the wavelength cut filter 23. As a result, the blue light Bx after passing through the wavelength cut filter 23 has a wavelength range of 420 to 460 nm. The reason why light in the wavelength range longer than 460 nm is cut in this way is because light in the wavelength range longer than 460 nm reduces the vascular contrast of the blood vessels to be observed. Note that the wavelength cut filter 23 may attenuate light in the wavelength range longer than 460 nm instead of cutting light in the wavelength range longer than 460 nm.

[0030] The G-LED 20c emits green light G with a wavelength band of 480 nm to 600 nm. The R-LED 20d emits green light G with a wavelength band of 600 nm to 650 nm. reaches They emit red light R. The light emitted from each of the LEDs 20a to 20d may have the same center wavelength and peak wavelength, or may be different.

[0031] The light source control unit 22 adjusts the timing of emitting the illumination light, the duration of emitting light, the amount of light, and the spectrum by independently controlling the on / off and the amount of light emitted when emitting light of each of the LEDs 20a to 20d. The control of turning on and off the lights in the light source control unit 22 differs for each observation mode. The reference brightness can be set by the brightness setting unit of the light source device 14 or the console 19, etc.

[0032] In the normal mode or the attention area detection mode, the light source control unit 22 turns on all of the V-LED 20a, the B-LED 20b, the G-LED 20c, and the R-LED 20d. At this time, as shown in FIG. 4, the light amount ratio Lc between the purple light V, the blue light B, the green light G, and the red light R is set so that the peak of the light intensity of the blue light Bx is greater than the peak of the light intensity of any of the purple light V, the green light G, and the red light R. As a result, in the normal mode or the attention area detection mode, the light source device 14 emits multi-color light for the normal mode or the attention area detection mode including the purple light V, the blue light Bx, the green light G, and the red light R as the normal light. The normal light has a certain level of intensity from the blue band to the red band, and is therefore almost white.

[0033] In the special mode, the light source control unit 22 turns on all of the V-LED 20a, the B-LED 20b, the G-LED 20c, and the R-LED 20d. At this time, as shown in FIG. 5, the light amount ratio Ls between the purple light V, the blue light B, the green light G, and the red light R is set so that the peak of the light intensity of the purple light V is greater than the peak of the light intensity of any of the blue light Bx, the green light G, and the red light R. Also, the peaks of the light intensity of the green light G and the red light R are set so that they are smaller than the peaks of the light intensity of the purple light V and the blue light Bx. As a result, in the special mode, the light source device 14 emits multi-color light for the special mode, including the purple light V, the blue light Bx, the green light G, and the red light R, as the special light. The special light is bluish light because the proportion of the purple light V is large. The special light does not have to include all four colors, but only needs to include light from at least one of the four colors of the LEDs 20a to 20d. The special light preferably has a main wavelength range, for example a peak wavelength or a center wavelength, of 450 nm or less.

[0034] As shown in FIG. 2, the illumination light emitted by the light source unit 20 is incident on a light guide 24 inserted into the insertion unit 12a via an optical path coupling unit (not shown) formed of a mirror, a lens, etc. The light guide 24 is built into the endoscope 12 and the universal cord, and propagates the illumination light to the tip 12d of the endoscope 12. The universal cord is a cord that connects the endoscope 12 to the light source device 14 and the processor device 16. Note that a multimode fiber can be used as the light guide 24. As an example, a thin-diameter fiber cable having a core diameter of 105 μm, a cladding diameter of 125 μm, and a diameter of φ0.3 mm to φ0.5 mm including a protective layer serving as an outer cover can be used as the light guide 24.

[0035] An illumination optical system 30a and an imaging optical system 30b are provided at the tip 12d of the endoscope 12. The illumination optical system 30a has an illumination lens 32. An observation object is illuminated by illumination light propagating through the light guide 24 via the illumination lens 32. The imaging optical system 30b has an objective lens 34, a magnifying optical system 36, and an imaging sensor 38 (corresponding to the "imaging section" of the present invention). Various types of light, such as reflected light, scattered light, and fluorescent light from the observation object, are incident on the imaging sensor 38 via the objective lens 34 and the magnifying optical system 36. As a result, an image of the observation object is formed on the imaging sensor 38.

[0036] The magnifying optical system 36 includes a zoom lens 36a that magnifies the observation target, and a lens driver 36b that moves the zoom lens 36a in the optical axis direction CL. The zoom lens 36a is freely moved between the telephoto end and the wide-angle end in accordance with zoom control by the lens driver 36b, thereby enlarging or reducing the observation target imaged on the imaging sensor 38.

[0037] The imaging sensor 38 is a color imaging sensor that captures an image of an object illuminated with illumination light. Each pixel of the imaging sensor 38 is provided with an R (red) color filter, a G (green) color filter, or a B (blue) color filter. The imaging sensor 38 receives purple to blue light at B pixels provided with B color filters, receives green light at G pixels provided with G color filters, and receives red light at R pixels provided with R color filters. Then, image signals of each color, RGB, are output from the pixels of each color. The imaging sensor 38 transmits the output image signals to the CDS circuit 40.

[0038] In the normal mode or the attention area detection mode, the imaging sensor 38 captures an image of an object illuminated with normal light, thereby outputting a Bc image signal from the B pixel, a Gc image signal from the G pixel, and an Rc image signal from the R pixel. In the special mode, the imaging sensor 38 captures an image of an object illuminated with special light, thereby outputting a Bs image signal from the B pixel, a Gs image signal from the G pixel, and an Rs image signal from the R pixel.

[0039] As the imaging sensor 38, a CCD (Charge Coupled Device) imaging sensor, a CMOS (Complementary Metal-Oxide Semiconductor) imaging sensor, or the like can be used. Also, instead of the imaging sensor 38 provided with RGB primary color filters, a complementary color imaging sensor provided with C (cyan), M (magenta), Y (yellow) and G (green) complementary color filters may be used. When a complementary color imaging sensor is used, image signals of four colors, CMYG, are output. Therefore, by converting the four color image signals of CMYG into three color image signals of RGB by complementary color-primary color conversion, it is possible to obtain image signals of each color of RGB similar to those of the imaging sensor 38. Also, instead of the imaging sensor 38, a monochrome sensor not provided with color filters may be used.

[0040] The CDS circuit 40 performs correlated double sampling (CDS) on the analog image signal received from the image sensor 38. The image signal that has passed through the CDS circuit 40 is input to the AGC circuit 42. AGC Circuit 42 performs automatic gain control (AGC) on the input image signal. The A / D (Analog to Digital) conversion circuit 44 converts the analog image signal that has passed through the AGC circuit 42 into a digital image signal. The A / D conversion circuit 44 inputs the digital image signal after A / D conversion to the processor device 16.

[0041] As shown in FIG. 2, the processor device 16 includes an image signal acquisition section 50, a DSP (Digital Signal Processor) 52, a noise reduction section 54, an image processing section 56, and a display control section 58.

[0042] The processor device 16 includes functions as a medical image processing device, and as described below, the image processing unit 56 acquires an endoscopic image and detects a region of interest within the object of observation from the endoscopic image, and the display control unit 58 highlights the region of interest on the endoscopic image 75.

[0043] The image signal acquisition unit 50 acquires digital image signals corresponding to the observation mode from the endoscope 12. In the normal mode or the area of ​​interest detection mode, the image signal acquisition unit 50 acquires the Bc, Gc, and Rc image signals. In the special mode, the image signal acquisition unit 50 acquires the Bs, Gs, and Rs image signals for one frame when illuminated with normal light, and the image signal acquisition unit 50 acquires the Bs, Gs, and Rs image signals for one frame when illuminated with special light.

[0044] The DSP 52 performs various signal processing such as defect correction, offset processing, DSP gain correction, linear matrix processing, gamma conversion, and demosaic processing on the image signal acquired by the image signal acquisition unit 50. The defect correction process corrects signals from defective pixels of the imaging sensor 38. The offset process removes dark current components from the defect-corrected image signal to set an accurate zero level. The DSP gain correction process adjusts the signal level by multiplying the offset-processed image signal by a specific DSP gain.

[0045] The linear matrix processing improves the color reproducibility of the image signal that has been subjected to the DSP gain correction processing. The gamma conversion processing adjusts the brightness and saturation of the image signal that has been subjected to the linear matrix processing. The image signal that has been subjected to the gamma conversion processing is subjected to demosaic processing (also called isotropic processing or synchronization processing) to generate signals of colors that are missing in each pixel by interpolation. This demosaic processing results in all pixels having signals of each of the RGB colors. The noise reduction unit 54 reduces noise by performing noise reduction processing using, for example, a moving average method or a median filter method on the image signal that has been subjected to the demosaic processing etc. by the DSP 52. The image signal after noise reduction is input to the image processing unit 56.

[0046] The image processing unit 56 includes a normal mode image processing unit 60, a special mode image processing unit 62, and an attention area detection mode image processing unit 64. The normal mode image processing unit 60 operates when the normal mode is set, and performs color conversion processing, color enhancement processing, and structure enhancement processing on the received Bc, Gc, and Rc image signals. In the color conversion processing, the RGB image signal is subjected to color conversion processing such as 3×3 matrix processing, tone conversion processing, and 3D LUT (Look Up Table) processing.

[0047] The color enhancement process is performed on the RGB image signal that has already been subjected to the color conversion process. The structure enhancement process is a process for enhancing the structure of the object of observation, and is performed on the RGB image signal after the color enhancement process. By performing various image processing such as those described above, a normal image is obtained. The normal image is an image obtained based on normal light in which purple light V, blue light Bx, green light G, and red light R are emitted in a well-balanced manner, and therefore has a natural color tone. The normal image is input to the display control unit 58.

[0048] The special mode image processing unit 62 operates when the special mode is set. The special mode image processing unit 62 performs color conversion processing, color enhancement processing, and structure enhancement processing on the received Bs image signal, Gs image signal, and Rs image signal. The processing contents of the color conversion processing, color enhancement processing, and structure enhancement processing are the same as those of the normal mode image processing unit 60. By performing various image processing as described above, a special image is obtained. The special image is an image obtained based on special light in which the purple light V, which has a high absorption coefficient of hemoglobin in blood vessels, emits a larger amount of light than the other colors of blue light Bx, green light G, and red light R, so that the resolution of the blood vessel structure and glandular duct structure is higher than other structures. The special image is input to the display control unit 58.

[0049] The attention area detection mode image processing unit 64 operates when the attention area detection mode is set. As shown in Fig. 6, the attention area detection mode image processing unit 64 has a detection image processing unit 70, an attention area detection unit 71, a visibility determination unit 72, and a visibility notification control unit 73. The detection image processing unit 70 sequentially obtains endoscopic images 75 by performing image processing, such as color conversion processing, similar to that performed by the normal mode image processing unit 60, on the received Bc image signal, Gc image signal, and Rc image signal.

[0050] The attention area detection unit 71 performs an attention area detection process for analyzing the endoscopic image 75 and detecting an attention area within the observation target. In this embodiment, the attention area detection unit 71 detects a lesion (e.g., a tumor or inflammation) within the observation target as an attention area. In this case, the attention area detection unit 71 first divides the endoscopic image 75 into a plurality of small areas, for example, square areas each having a number of pixels. Next, image features are calculated from the divided endoscopic image 75. Next, based on the calculated features, recognition processing is performed to determine whether each small area is a lesion area or not. As such recognition processing, a machine learning algorithm such as a Convolutional Neural Network or Deep Learning is preferable.

[0051] Moreover, the feature amount calculated from the endoscopic image 75 by the attention area detection unit 71 is preferably a shape or color of a predetermined part of the observation target, or a value obtained from the shape, color, etc. For example, the feature amount is preferably at least any one of blood vessel density, blood vessel shape, number of blood vessel branches, blood vessel thickness, blood vessel length, blood vessel tortuosity, blood vessel penetration depth, duct shape, duct opening shape, duct length, duct tortuosity, and color information, or a value combining two or more of them.

[0052] Finally, a group of small regions that have been identified as being of the same type are extracted as one lesion. The region-of-interest detection unit 71 associates information such as the position, size, and type of the extracted lesion with the endoscopic image 75 as detection information 76. The region-of-interest detection mode image processing unit 64 outputs the endoscopic image 75 associated with the detection information 76 to the display control unit 58.

[0053] The display control unit 58 performs display control for displaying images and data from the image processing unit 56 on the monitor 18. When set to the normal mode, the display control unit 58 performs control for displaying normal images on the monitor 18. When set to the special mode, the display control unit 58 performs control for displaying special images on the monitor 18.

[0054] When the attention area detection mode is set, the display control unit 58 highlights the attention area detected by the attention area detection unit 71 in the endoscopic image 75. When highlighting the attention area, the display control unit 58 first sets an emphasis area for highlighting the attention area based on the endoscopic image 75 output from the attention area detection mode image processing unit 64 and the detection information 76 associated with the endoscopic image 75.

[0055] 7, the display controller 58 sets an emphasis region 78 that is larger in area than the lesion 77 and that includes the lesion 77, based on detection information 76 such as the position, size, and type of the lesion 77. In this embodiment, a square region is set as the emphasis region 78. The emphasis region 78 has, for example, a square periphery that is set at a predetermined distance from the periphery of the lesion 77. Note that the emphasis region 78 is not limited to this, and may be set to a square that is in contact with the periphery of the lesion 77.

[0056] The display control unit 58 highlights the highlighted region 78 set as described above. That is, the display control unit 58 superimposes a graphic as a highlight on the position of the highlighted region 78 in the endoscopic image 75. In this embodiment, the display control unit 58 displays a square frame-shaped (frame-shaped) graphic 79 surrounding the lesion 77, aligned with the position of the highlighted region 78. After setting the highlighted region 78, the display control unit 58 resets the highlighted region 78 in accordance with the amount of movement of the lesion 77 in the endoscopic image 75, and displays the graphic 79 aligned with the position of the reset highlighted region 78.

[0057] Moreover, the graphic 79 as the highlighted display has a display mode different from other parts of the endoscopic image 75, and the display control unit 58 displays the graphic 79 in a color with a different hue from the colors generally contained in endoscopic images. Moreover, the color of the graphic 79 may be set according to an input operation by the user.

[0058] The display control unit 58 outputs setting information 81 of the graphic 79 as the highlighted display to the image processing unit 56. The setting information 81 includes position information and color information of the graphic 79 with respect to the endoscopic image 75. Note that the setting information 81 is tagged with information of the original endoscopic image 75 in which the lesion area 77 was detected.

[0059] The visibility determination unit 72 determines the visibility of the highlighting from the image information acquired from the endoscopic image 75 in which the lesion 77 is detected and the highlighting setting information 81 set by the display control unit 58, and calculates a numerical index value as the determination result. In this embodiment, the visibility determination unit 72 calculates color information from the image information acquired from the endoscopic image 75 in which the lesion 77 is detected by the attention area detection unit 71 and the highlighting setting information 81, and calculates the color difference between the endoscopic image 75 and the figure 79 as an index value from this color information. Note that the color information refers to information related to color, such as hue, lightness, and saturation.

[0060] As shown in Fig. 8, when calculating color information from image information acquired from an endoscopic image 75, the visibility determination unit 72 calculates, as color information, an average value inside the highlighted display including the diseased area 77, specifically, within an area 82 surrounded by a graphic 79 in the endoscopic image 75 (see also Fig. 7). As described above, the setting information 81 includes position information of the graphic 79, so that the area 82 surrounded by the graphic 79 can be cut out from the endoscopic image 75 based on the position information to calculate color information. On the other hand, when calculating color information from setting information of the highlighted display, the average value of the graphic 79 is calculated as color information. The visibility determination unit 72 calculates the color difference between the endoscopic image 75 and the graphic 79 from this color information.

[0061] The color difference is calculated by the visibility determining unit 72, for example, using the color difference formula according to CIEDE2000 defined in JIS Z 8730 7.3. By using such a standardized calculation method, it is possible to calculate the color difference according to the human visual characteristics. When using the color difference formula according to CIEDE2000, the color information used to calculate the color difference is information in the CIELab color space consisting of an L component indicating lightness, an a component indicating a degree of red or green, and a b component indicating a degree of yellow or blue.

[0062] The method of calculating color difference is not limited to the above, and any calculation method that takes human vision into consideration may be used. For example, the calculation may be performed using Euclidean distance (also known as CIE76) in the CIELab color space.

[0063] The visibility notification control unit 73 notifies the user of the determination result determined by the visibility determination unit 72. As shown in Fig. 9, the visibility notification control unit 73 outputs the color difference calculated as above as the determination result to the display control unit 58 as notification information 83, and causes the display control unit 58 to display the color difference information on the display screen 84 of the monitor 18. In this embodiment, when the color difference is equal to or less than a preset first threshold value, the visibility notification control unit 73 outputs the color difference information as notification information 83 to the display control unit 58.

[0064] In the visibility notification control unit 73, for example, the preset first threshold value is set to 2.0. According to the above JIS standard, a color difference of about 1 is a level at which the difference between two colors can be distinguished when they are next to each other horizontally, and a color difference of 2 to 3 is a level at which the difference between two colors can be distinguished when they are separated and compared. Since the visibility notification control unit 73 sets the first threshold value to 2.0 as described above based on the JIS standard, it can notify that visibility has decreased.

[0065] In the following, the process in which the image processing unit 56 and the display control unit 58 determine the visibility of the highlighting in the attention area detection mode and display the determination result on the display screen 84 of the monitor 18 will be described with reference to the flowchart shown in FIG. 10 and the explanatory diagram shown in FIG. 11. The doctor, who is the user, operates the mode switching unit 13c to switch to the attention area detection mode. This illuminates the observation object with the illumination light for the attention area detection mode. The observation object illuminated with the illumination light for the attention area detection mode is imaged by the imaging sensor 38 to obtain an endoscopic image 75. When switching to the attention area detection mode, the display control unit 58 sequentially acquires the endoscopic images 75 (S11) and displays them in real time on the display screen 84 of the monitor 18.

[0066] During real-time display in the area-of-interest detection mode, an area-of-interest detection process for detecting an area of ​​interest within an observation target is performed on the acquired endoscopic image 75 by the area-of-interest detection unit 71. When an area of ​​interest is detected (Y in S12), detection information 76 by the area-of-interest detection unit 71 is output in association with the endoscopic image 75.

[0067] Then, as shown in FIG. 11(A), when a lesion 77 is detected as an area of ​​interest within the object of observation, i.e., when detection information 76 is associated with endoscopic image 75, display control unit 58 sets an emphasis area 78 using the detection information 76 associated with endoscopic image 75, particularly information on the position and size of lesion 77 (S13).

[0068] After setting the highlighted region 78, the display control unit 58 superimposes a graphic 79 as a highlighted display at the position of the highlighted region 78 in the endoscopic image 75 as shown in Fig. 11(B) (S14), and outputs setting information 81 for the graphic 79 to the image processing unit 56. For convenience of illustration, in Figs. 7 to 9, 11, and 12, the difference in color between the graphic 79 and other parts in the endoscopic image 75 is represented by the presence or absence of shading. On the other hand, if no lesion 77 is detected in the observation target (N in S12), naturally no highlighted display is performed.

[0069] Since the setting information 81 of the figure 79 is tagged with information of the original endoscopic image 75 in which the lesion 77 was detected, the visibility determination unit 72 reads out the original endoscopic image 75, calculates color information from the image information acquired from the endoscopic image 75 and the setting information 81 of the figure 79, and determines the visibility of the highlighting (S15). The visibility is determined by comparing the color difference calculated from the color information with the first threshold value, as described above. As shown in FIG. 12(A), when the figure 79 is displayed as a highlighting, the figure 79 may blend in with the surroundings or become less noticeable compared to the surroundings, decreasing the visibility, depending on the color of the subject in the endoscopic image 75, the presence or absence of an object in the subject, and the like. In such a case, the value of the color difference between the endoscopic image 75 and the figure 79 generally also decreases.

[0070] If the color difference is equal to or less than the first threshold (Y in S16), as shown in Fig. 12(B), the color difference information is output to the display control unit 58 as notification information 83. The display control unit 58 displays the notification information 83 on the display screen 84 to notify that the visibility has decreased (S17). Note that if the color difference exceeds the first threshold (N in S16), the visibility determination unit 72 does not provide any notification.

[0071] As described above, the visibility of the highlighting in the endoscopic image 75 is judged, and if the visibility of the highlighting has decreased, an alert is issued, so that the user (doctor) can recognize the decrease in visibility of the highlighting, and it is possible to avoid a situation in which the user does not notice an area of ​​interest such as a lesion.

[0072] [Second embodiment] In the first embodiment, an example is given in which color difference information is displayed on the display screen as the visibility judgment result, but the present invention is not limited to this, and identification information may be displayed according to an index value as the judgment result. Fig. 13 shows an example in which identification information 85A, 85B according to the color difference as the judgment result is displayed on the display screen 84.

[0073] In this embodiment, the process of calculating color information from image information acquired from the endoscopic image 75 and the highlighting setting information 81, calculating the color difference between the endoscopic image 75 and the figure 79 from the color information, and comparing the color difference with the first threshold is the same as that of the first embodiment. If the color difference is equal to or less than the first threshold, the visibility determination unit 72 outputs the identification information 85A to the display control unit 58. The display control unit 58 displays the identification information 85A on the display screen 84 to notify that the visibility has decreased. In the example shown in FIG. 13(A), text information saying "low visibility" is displayed as the identification information 85A. This allows the doctor, who is the user, to recognize the decrease in visibility of the highlighting, as in the first embodiment.

[0074] As a modification of this embodiment, the identification information may be displayed not only when the color difference is equal to or less than the first threshold, but also when the color difference exceeds the first threshold, i.e., when visibility is high. As shown in FIG. 13(B), when the color difference exceeds the first threshold, the visibility determination unit 72 outputs the identification information 85B to the display control unit 58. The display control unit 58 displays the identification information 85B on the display screen 84 to notify that visibility is high. In the example shown in FIG. 13(B), text information saying "high visibility" is displayed as the identification information 85B.

[0075] The setting for displaying the identification information according to the color difference is not limited to two-stage setting, that is, when the color difference is equal to or less than the first threshold value, or when the color difference is greater than the first threshold value, but may be set to three or more stages. The visibility determination unit 72 performs the determination based on three preset stages, for example, color difference 2.0 or less, greater than 2.0 and 4.0 or less, and greater than 4.0. The display screen 84 displays, as the identification information, that the visibility is low when the color difference is 2.0 or less, that the visibility is medium when the color difference is greater than 2.0 and 4.0 or less, and that the visibility is high when the color difference is greater than 4.0. In this case, it is preferable to display, as the identification information, text information such as "low visibility", "medium visibility", and "high visibility" according to the stage of the color difference.

[0076] [Third embodiment] In the above second embodiment, an example is given in which identification information corresponding to the color difference as the judgment result is displayed as the visibility judgment result, but this is not limited to this, and an identification figure corresponding to the color difference as the judgment result may be displayed as in the example shown in Figure 14.

[0077] In this embodiment, the process of calculating color information from image information acquired from the endoscopic image 75 and the highlighting setting information 81, calculating the color difference between the endoscopic image 75 and the figure 79 from the color information, and comparing the color difference with the first threshold is the same as in the first and second embodiments. Then, when the color difference is equal to or less than the first threshold, the information of the icon 86A is output to the display control unit 58. The display control unit 58 displays the icon 86A as an identification figure on the display screen 84 to notify that the visibility is reduced. In the example shown in FIG. 14(A), a mark imitating a sign indicating the presence of a danger is displayed as the icon 86A. This allows the user, a doctor, to recognize the reduced visibility of the highlighting, as in the first and second embodiments.

[0078] As a modification of this embodiment, the identification figure may be displayed not only when the color difference is equal to or less than the first threshold, but also when the color difference exceeds the first threshold, i.e., when the visibility is high. As shown in FIG. 14(B), when the color difference exceeds the first threshold, the visibility determination unit 72 outputs information of an icon 86B to the display control unit 58. The display control unit 58 displays the icon 86B as the identification figure on the display screen 84 to notify that the visibility is high. In the example shown in FIG. 14(B), a double circle mark is displayed as the icon 86B.

[0079] Also, the setting for displaying the identification information according to the color difference may be set to three or more stages as in the second embodiment. For example, three stages of numerical values ​​are set in advance as in the second embodiment, and the visibility determination unit 72 performs the determination based on this setting. Then, when the color difference is 2.0 or less, the visibility is low, when the color difference is greater than 2.0 and less than 4.0, the visibility is medium, and when the color difference is greater than 4.0, the visibility is high, as an identification figure, which is displayed on the display screen 84. In this case, it is preferable to display an icon of a different shape as the identification figure according to the stage of the color difference.

[0080] [Fourth embodiment] In each of the above embodiments, an image with highlighting superimposed on an endoscopic image is displayed within one display screen, and notification information, etc. is displayed in a non-display area of ​​the endoscopic image, but the image with notification information, etc. superimposed may also be displayed on a display screen other than the display screen that displays a normal endoscopic image.

[0081] In the example shown in Fig. 15, two display screens are displayed side by side on one monitor 18, with a normal endoscopic image 87 being displayed as a main image having a larger display area, and an image 88 in which highlighting is superimposed on the endoscopic image being displayed as a sub-image having a smaller display area than the main image. An icon 86A is further superimposed as identification information on the image 88 in which highlighting is superimposed. Note that in this embodiment, the normal endoscopic image 87 is the endoscopic image 75 itself acquired by the image processing unit 56 in the region of interest detection mode in each of the above embodiments, and indicates a state in which no figure or the like is superimposed as highlighting.

[0082] The image 88 with the highlighting superimposed as a sub-image is obtained by superimposing a graphic 79 or the like as a highlight on the endoscopic image 75, as in the above-described embodiments. Then, as in the above-described embodiments, color information is calculated from the image information acquired from the endoscopic image 75 and the highlighting setting information 81, and the color difference between the endoscopic image 75 and the graphic 79 is calculated from the color information and compared with a first threshold value. Then, if the color difference is equal to or less than the first threshold value, information of an icon 86A is output to the display control unit 58. The display control unit 58 further superimposes the icon 86A on the image 88 with the highlighting superimposed to notify that visibility has decreased.

[0083] The information displayed as the visibility determination result is not limited to the icon 86A, but may be color difference information as an index value, identification information according to the color difference, or different identification information or identification figures according to the color difference, as in the above-mentioned embodiments. In addition, in the example shown in Fig. 15, two display screens are displayed side by side on one monitor 18, but the main image and the sub-image may be displayed on separate monitors.

[0084] In the first to fourth embodiments, when calculating the color difference between the endoscopic image 75 and the graphic 79 as the highlighted display, the visibility determination unit 72 calculates color information from the inside of the graphic 79. However, the present invention is not limited to this. The visibility determination unit 72 may calculate the color information from the outside of the graphic 79, specifically, the average value of the part of the endoscopic image 75 excluding the graphic 79 and the range 82 surrounded by the graphic 79. As described above, the setting information 81 includes the position information of the graphic 79, so that the outside of the graphic 79 can be cut out from the endoscopic image 75 based on the position information to calculate the color information. This allows the doctor, who is the user, to recognize the reduced visibility of the highlighted display even if the color difference between the graphic 79 as the highlighted display and the outside of the graphic 79 is reduced.

[0085] In addition, in the above first to fourth embodiments, the visibility determination unit 72 uses a preset first threshold value to determine the color difference between the endoscopic image 75 and the highlighted display. However, the first threshold value used for this determination does not always have to be the same value, and the first threshold value may be weighted depending on the thickness of the highlighted display line.

[0086] For example, the display control unit 58 can change the line thickness of the graphic 79 as the highlighted display according to the size of the attention area, or can change the line thickness of the graphic 79 according to the user's input operation. In this way, when the line thickness of the graphic 79 is made variable, the visibility determination unit 72 weights the first threshold value according to the line thickness of the graphic 79 when acquiring the highlighting setting information. As the weighting of the first threshold value, the first threshold value is set to be small in inverse proportion to the line thickness of the graphic 79.

[0087] For example, when the line thickness of figure 79 is the initial setting, the first threshold is set to 2.0, when the line thickness of figure 79 is thicker than the initial setting, the first threshold is set to less than 2.0, and when the line thickness of figure 79 is thinner than the initial setting, the first threshold is set to more than 2.0. Since the visibility of the highlighting increases as the line thickness of figure 79 increases, even if the first threshold is set to a small value according to the line thickness of figure 79, the doctor, who is the user, can recognize the reduced visibility of the highlighting, as in the above-described embodiments.

[0088] [Fifth embodiment] In each of the above embodiments, the visibility determination unit 72 uses the color difference calculated from the image information of the endoscopic image 75 and the setting information of the highlighting as an index value to determine the visibility, but the visibility may be determined based on the presence or absence of an object that is not a detection target that exists inside the highlighting. In this case, for example, as shown in FIG. 16(A), in the endoscopic image 75, as an object 89 or phenomenon that is not a detection target, for example, water, halation, bubbles, pigments, etc., other than the lesion 77 may enter inside the figure 79 as the highlighting. When color information is acquired, the part where the object 89 or phenomenon that is not a detection target is reflected in the endoscopic image 75 generally has high lightness or luminance. For this reason, in this embodiment, a lightness value is used as the color information acquired from the endoscopic image. Note that the invention is not limited to this, and a luminance value may be used as the color information acquired from the endoscopic image.

[0089] In order to detect whether an object 89 or phenomenon not to be detected exists inside the figure 79, the visibility determination unit 72 compares the brightness of each pixel within the range 82 surrounded by the figure 79 with a second threshold value. This second threshold value is set to a high brightness value assuming water, halation, bubbles, etc. Therefore, if the brightness is equal to or greater than the second threshold value, there is a high possibility that an object 89 or phenomenon not to be detected is reflected.

[0090] The visibility determination unit 72 further compares the area ratio of the portion of the range 82 surrounded by the figure 79 whose brightness is equal to or greater than the second threshold (the ratio of the area of ​​the portion of the range 82 whose brightness is equal to or greater than the second threshold) with a third threshold. The third threshold assumes that there are many portions with high brightness in the range 82, and is set to an area ratio of, for example, 50%.

[0091] When the area ratio of the portion having the brightness equal to or greater than the second threshold is equal to or greater than the third threshold, the visibility determination unit 72 determines that an object 89 or phenomenon not to be detected is inside the figure 79, i.e., visibility is reduced, and outputs information of the icon 86A to the display control unit 58. The display control unit 58 notifies the user that visibility is reduced by superimposing the icon 86A on the endoscopic image 75. Note that the information displayed as the visibility determination result is not limited to the icon 86A, and information on the area ratio of the portion having the brightness equal to or greater than the second threshold, identification information according to the area ratio, or the like may be displayed, and different identification information or identification figures may be displayed according to the area ratio.

[0092] In each of the above embodiments, the figure used for highlighting is a square frame, but this is not limited to this and any frame shape that can surround the area of ​​interest, such as a polygon other than a rectangle (quadrilateral), a circle, or an ellipse, may be used.

[0093] Furthermore, the figure as the highlighting is not limited to one frame shape surrounding the region of interest, and may be composed of multiple shapes. In the example shown in Fig. 17, the display control unit 58 places, as the highlighting, four L-shaped figures 91A-91D surrounding the lesion 77 at each corner of the highlighted region 78. Note that the two-dot chain lines in Fig. 17 are shown for the convenience of explaining the arrangement of the L-shaped figures 91A-91D, and are not actually displayed.

[0094] In the example shown in Figure 17, when the color difference calculated from the image information of the endoscopic image 75 and the highlighting setting information is used as an index value to judge visibility, as in the first to fourth embodiments, it is preferable that the visibility judgment unit 72 calculates the average value of the four L-shaped figures 91A to 91D as color information, calculates the color difference from the color information of the endoscopic image 75 and the color information of the average value of the L-shaped figures 91A to 91D, and compares it with a first threshold value.

[0095] However, the visibility determination unit 72 is not limited to this, and calculates color information for each of the four L-shaped figures 91A-91D, calculates a total of four color differences from the color information of the endoscopic image 75 and the color information of each of the L-shaped figures 91A-91D, and compares them with the first threshold. In this case, for example, if any one of the four color differences is equal to or less than the first threshold, it determines that the visibility is low. If the color difference is equal to or less than the first threshold, it outputs information such as color difference notification information 83, identification information, and identification figure to the display control unit 58. Thereafter, notification is performed in the same manner as in each of the above embodiments.

[0096] In addition, when an index value such as a color difference calculated from the image information of the endoscopic image 75 and the setting information of the highlighting is equal to or less than a first threshold value set in advance, or when the area ratio of the part inside the highlighting where the brightness or luminance is equal to or more than a second threshold value is equal to or more than a third threshold value, the image processing unit 56 may determine that the visibility is low, and may perform a notification as in the above-mentioned embodiments, and may automatically save the endoscopic image in which the attention area is detected. This allows the user to later check the endoscopic image in which the attention area is detected but the visibility is reduced, so that it is possible to reliably avoid a situation in which the user does not notice the attention area such as a lesion. Note that the destination for saving the endoscopic image in which the visibility is determined to be low and the attention area is detected as described above may be, for example, a storage device provided in the processor device 16, or a server such as a cloud.

[0097] Furthermore, when it is determined that visibility is low as described above, image processing unit 56 may not only notify the user, but also issue a warning such as emitting a sound, illuminating an indicator, or flashing part of the screen.

[0098] [Sixth embodiment] In each of the above embodiments, the visibility determination unit 72 determines visibility based on color difference information or the presence or absence of an object other than the detection target, but the visibility determination unit 72 may determine visibility based on the thickness of the line of the highlighting for the attention area. Fig. 18 shows an example of a display screen when the visibility determination unit 72 determines the visibility of the highlighting for the attention area based on the thickness of the line of the highlighting.

[0099] In this embodiment, the visibility determination unit 72 calculates, as an index value, the ratio of the line thickness T1 of the figure 79 to the maximum dimension LM of the lesion 77 detected from the endoscopic image 75 by the attention area detection unit 71. Note that the maximum dimension LM of the lesion 77 is, for example, the dimension of the largest part of the lesion 77 in either the X-axis direction or the Y-axis direction of the endoscopic image 75. In the example shown in Fig. 18, the dimension of the largest part of the lesion 77 in the X-axis direction is set to the maximum dimension LM.

[0100] The visibility determination unit 72 determines the visibility by comparing the ratio of the line thickness T1 of the figure 79 to the maximum dimension LM of the lesion 77 described above with a threshold value, and determines that the visibility is low if the ratio is equal to or less than the threshold value. Then, similar to the above-mentioned embodiments, the visibility determination unit 72 outputs information such as notification information, identification information, and identification figure to the display control unit 58. Thereafter, notification is performed similar to the above-mentioned embodiments. In the example shown in FIG. 18, text information saying "low visibility" is displayed as the identification information 92. The smaller the ratio of the line thickness of the figure to the attention area, the lower the visibility, so similar to the above-mentioned embodiments, the doctor, who is the user, can recognize the reduced visibility of the highlighting.

[0101] [Seventh embodiment] In the sixth embodiment, the visibility determination unit 72 determines the visibility from the thickness of the highlighting line for the attention area, but this is not limiting. A frame-shaped figure surrounding the attention area may be displayed as the highlighting, and the visibility may be determined from the similarity of the frame shape to the attention area. Fig. 19 shows an example of a display screen when the visibility of the highlighting is determined from the similarity of the frame shape to the attention area.

[0102] In this embodiment, the visibility determination unit 72 analyzes the similarity between the shape of the lesion 77 detected from the endoscopic image 75 by the attention area detection unit 71 and a frame-shaped figure 93 surrounding the lesion 77. The figure 93 is a circular frame-shaped figure that surrounds the lesion 77 and contacts multiple points on the outer periphery of the lesion 77. When determining the visibility from the similarity, the visibility determination unit 72 analyzes the similarity between the outline shape of the lesion 77 and the inner periphery shape of the figure 93 by a method such as well-known template matching, compares this similarity with a threshold, and determines that the visibility is low if the similarity is equal to or higher than the threshold. Then, as in each of the above embodiments, the visibility determination unit 72 outputs information such as notification information, identification information, and identification figure to the display control unit 58. Thereafter, notification is performed in the same manner as in each of the above embodiments. In the example shown in FIG. 19, text information saying "low visibility" is displayed as the identification information 92. The higher the similarity of the frame shape to the region of interest, the lower the visibility becomes. Therefore, similarly to the above-described embodiments, the doctor who is the user can recognize the decrease in visibility of the highlighting.

[0103] In each of the above-mentioned embodiments, the display control unit 58 superimposes a frame-shaped figure on the position of the highlighted region, but the present invention is not limited to this, and the color of the highlighted region may be changed as a highlighted display. In this case, when the lesion 77 is detected as the attention region and the highlighted region is set, the display control unit 58 may change the highlighted region to a color different from the original color as a highlighted display, for example, by extracting a color that is often included in the endoscopic image 75 and changing the highlighted region to a color different from the other parts of the endoscopic image 75. Note that the different color here means, for example, a color with a different hue.

[0104] The highlighting of the highlighted region is not limited to the above, and may be any image processing that can visually distinguish the highlighted region from the surroundings, such as saturation change processing, contrast processing, negative-positive inversion processing, filtering processing, etc. Alternatively, the highlighting of the highlighted region by image processing may be combined with the highlighting by a figure surrounding the lesion in each of the above embodiments.

[0105] In each of the above embodiments, the observation target is illuminated using the four color LEDs 20a to 20d, but the observation target may be illuminated using a laser light source and a phosphor. Also, in each of the above embodiments, the observation target is illuminated using the four color LEDs 20a to 20d, but the observation target may be illuminated using a white light source such as a xenon lamp and a rotary filter. Also, instead of the color image sensor 38, the observation target may be imaged using a monochrome image sensor.

[0106] In the above embodiment, the medical image processing device of the present invention is applied to an endoscopic system that acquires endoscopic images as medical images. However, it goes without saying that the medical image processing device of the present invention can be applied to various endoscopic systems such as capsule endoscopes. The medical image processing device of the present invention can also be applied to various medical image devices that acquire other medical images such as X-ray images, CT images, MR images, ultrasound images, pathological images, and PET (Positron Emission Tomography) images.

[0107] In the above embodiment, the hardware structure of the processing unit that executes various processes, such as the image processing unit 56 and the display control unit 58, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a GPU (Graphical Processing Unit), a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing various processes.

[0108] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same type or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, etc.). Also, multiple processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by a computer such as a client or a server, there is a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0109] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements. [Explanation of symbols]

[0110] 10 Endoscope System 12 Endoscopy 12a Insertion part 12b Operation section 12c Curved section 12d Tip 13a Angle knob 13b Still image acquisition section 13c Mode switching section 13d Zoom operation section 14 Light source device 16 Processor Unit 18 Monitors 19 Console 20 Light source section 20a V-LED 20b B-LED 20c G-LED 20d R-LED 22 Light source control unit 23 Wavelength cut filter 24 Light Guide 30a illumination optical system 30b Imaging optical system 32 Lighting Lens 34 Objective Lens 36 Magnification Optical System 36a Zoom lens 36b Lens drive unit 38 Image Sensor 40 CDS circuit 42 AGC circuit 44 A / D conversion circuit 50 Image signal acquisition unit 52 DSP 54 Noise reduction section 56 Image Processing Unit 58 Display control section 60 Normal mode image processing section 62 Special mode image processing section 64 Image processing unit for detecting area of ​​interest 70 Detection image processing unit 71 Attention area detection section 72 Visibility Judgment Section 73 Visibility notification control unit 75 Endoscopic Images 76 Detection Information 77 Lesion 78 Emphasis area 79 Shapes 81 Setting information 82 Range 83 Notification Information 84 Display screen 85A Identification Information 85B Identification information 86A Icon 86B Icon 87 Normal Endoscopic Images 88 Image with highlighting overlay 89 Objects not to be detected 91A~91D L-shaped

Claims

1. 1. A medical imaging device comprising a processor, The processor, Acquire medical images; Detecting an area of ​​interest within the medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display superimposed on the medical image; determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; notifying a user of the visibility determination result; The processor obtains a color difference between the medical image and the highlighted display from color information calculated from the image information and color information calculated from the highlighted display, and determines the visibility from the color difference using a first threshold value set in advance; A medical image processing device in which the first threshold is weighted according to a thickness of the highlighting line.

2. The medical image processing apparatus of claim 1 , wherein the processor obtains the image information from inside the highlight in the medical image.

3. The medical image processing apparatus of claim 1 , wherein the processor obtains the image information from outside the highlight in the medical image.

4. The medical image processing apparatus according to claim 3 , wherein the processor calculates an average value calculated from the image information as the color information.

5. A medical image processing device having a processor, The processor, Acquire medical images; Detecting an area of ​​interest within the medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display superimposed on the medical image; determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; notifying a user of the visibility determination result; The processor displays a frame-shaped figure surrounding the attention area as the highlighting, and determines the visibility based on a line thickness of the frame-shaped figure relative to the attention area; A medical image processing device that determines the visibility using a ratio of a line thickness of the graphic to a maximum dimension of the region of interest.

6. A medical image processing device comprising a processor, The processor, Acquire medical images; Detecting an area of ​​interest within the medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display superimposed on the medical image; determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; notifying a user of the visibility determination result; The processor displays a frame-shaped figure surrounding the region of interest as the highlighting, and determines the visibility based on a similarity of the frame shape to the region of interest; A medical image processing apparatus, wherein the degree of similarity is a degree of similarity between a contour shape of the region of interest and an inner periphery shape of the figure.

7. 7. The medical image processing apparatus according to claim 1, wherein the processor displays the determination result on a display screen.

8. 8. The medical image processing apparatus according to claim 1, wherein the processor calculates a numerical index value as the result of the visibility determination, and displays the numerical index value on a display screen.

9. The medical image processing apparatus according to claim 8 , wherein the processor displays the index value as the notification when the index value is equal to or smaller than a preset threshold value.

10. 10. The medical image processing apparatus according to claim 8, wherein the processor uses a color difference calculated from the image information and the highlighted display as the index value.

11. 11. The medical image processing apparatus according to claim 1, wherein the processor calculates an index value by converting the result of the visibility determination into a numerical value, and displays identification information or an identification pattern corresponding to the index value.

12. A medical imaging device comprising a processor, The processor, Acquire medical images; Detecting an area of ​​interest within the medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display superimposed on the medical image; determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; notifying a user of the visibility determination result; The processor is a medical image processing device that determines the visibility based on the presence or absence of an object that is not a detection target and is present inside the highlighted display.

13. 1. A medical imaging device comprising a processor, The processor, Acquire medical images; Detecting an area of ​​interest within the medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display superimposed on the medical image; determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; notifying a user of the visibility determination result; The medical image processing device, in which the processor determines that there is an object not to be detected when the area ratio of the portion inside the highlighting in the medical image where the brightness or luminance is equal to or greater than a second threshold value to the range inside the highlighting is equal to or greater than a third threshold value.

14. The medical image processing apparatus according to claim 7 , wherein the processor displays the determination result on a display screen different from a display screen that displays the medical image.

15. The medical image processing apparatus according to claim 8 , wherein the processor automatically saves the medical image in which the region of interest is detected when the index value is equal to or less than a first threshold value set in advance.

16. The medical image processing apparatus according to claim 8 , wherein the processor issues a warning to a user when the index value is equal to or smaller than a first threshold value set in advance.

17. A light source device that emits illumination light for illuminating an object to be observed; an endoscope having an image sensor for capturing an image of an object illuminated by the illumination light; A processor; a monitor for displaying a medical image obtained by signal processing the image signal output by the imaging sensor; The processor, acquiring said medical image; Detecting an area of ​​interest within the medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display on the monitor while superimposing the emphasis display on the medical image; determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; notifying a user of the visibility determination result; The processor obtains a color difference between the medical image and the highlighted display from color information calculated from the image information and color information calculated from the highlighted display, and determines the visibility from the color difference using a first threshold value set in advance; An endoscope system in which the first threshold is weighted according to the thickness of the highlighting line.

18. acquiring a medical image; detecting an area of ​​interest within the acquired medical image; setting an emphasis display for emphasizing the detected region of interest, and displaying the emphasis display superimposed on the medical image; determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; notifying a user of the result of the visibility determination; In the step of determining the visibility, a color difference between the medical image and the highlighted display is obtained from color information calculated from the image information and color information calculated from the highlighted display, and the visibility is determined from the color difference using a first threshold value set in advance; A method of operating a medical imaging device, wherein the first threshold is weighted according to a thickness of the highlighting line.

19. A program for a medical image processing device, which is installed in a medical image processing device that acquires a medical image and performs image processing on the medical image, comprising: On the computer, acquiring said medical images; detecting an area of ​​interest within the medical image; a function of setting an emphasis display for emphasizing the detected region of interest and displaying the emphasis display superimposed on the medical image; a function of determining visibility of the highlighting based on image information acquired from the medical image in which the region of interest is detected and the set highlighting; and realizing a function of notifying a user of the result of the visibility determination. a function of acquiring a color difference between the medical image and the highlighted display from color information calculated from the image information and color information calculated from the highlighted display, and determining the visibility from the color difference using a first threshold value set in advance; A program for a medical image processing device, wherein the first threshold is weighted according to a thickness of the highlighted line.

Images