IMAGE PROCESSING SYSTEM, PROCESSOR DEVICE, ENDOSCOPIC SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM

Abstract

Provided are an image processing system, a processor device, an endoscope system, an image processing method, and a program that are capable of notifying a level of confidence of biopsy which suppresses the obstruction of a user's view. In this invention, a medical image (38) is obtained, a region of interest (500) is detected from the medical image, a level of confidence of biopsy representing the degree of certainty of a biopsy region that is to be biopsied is calculated for the region of interest, a notification image (502), for which the degree of notification varies according to the calculated level of confidence of biopsy and which is displayed superimposed on the medical image, is generated, and a display signal representing the notification image is transmitted to a display device.

Description

[Technical field]

[0001] The present invention relates to an image processing system, a processor device, an endoscope system, an image processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art Diagnosis support systems are known that use image recognition processing to automatically recognize regions suspected of being lesions from medical images such as endoscopic images, highlight the regions suspected of being lesions, and display the recognition results on a display device.

[0003] Patent Document 1 describes an endoscope system that acquires a normal light image and a special light image. The system described in this document acquires a white light image corresponding to a wavelength band of white light and a special light image corresponding to a specific wavelength band, discriminates the type of subject image contained in the special light image, and performs enhancement processing on the white light image based on the discriminated type of subject.

[0004] Patent Document 2 describes a visual function evaluation device that evaluates the visual function of a subject's eye. The device described in this document displays the acquired oxygen saturation distribution by superimposing contour lines indicating areas of equal saturation. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2011-135983 A [Patent Document 2] JP 2013-119019 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, since areas suspected of being a lesion are biopsied to make a definitive diagnosis, it is important to notify users such as doctors of the results of automatic recognition. However, there is a concern that the automatic recognition results that are notified may obstruct the doctor's view during a biopsy.

[0007] The system described in Patent Document 1 does not take into consideration the viewpoint of not obstructing the user's field of vision. The same is true for the device described in Patent Document 2.

[0008] The above problem is not limited to the notification of automatic recognition results in endoscopic images, but similar problems also exist in the notification of automatic recognition results in medical images such as CT images, MRI images, and ultrasound images.

[0009] The present invention has been made in consideration of the above circumstances, and aims to provide an image processing system, a processor device, an endoscopic system, an image processing method, and a program that enable notification of biopsy certainty while minimizing obstruction to the user's field of vision. [Means for solving the problem]

[0010] In order to achieve the above object, the following aspects of the invention are provided.

[0011] The image processing system according to the present disclosure is an image processing system having one or more processors, which acquires a medical image, detects an area of ​​interest from the medical image, calculates a biopsy certainty for the area of ​​interest indicating the certainty of the biopsy area being the target of a biopsy, generates an alarm image having a different degree of notification depending on the calculated biopsy certainty, the alarm image being displayed superimposed on the medical image, and transmits a display signal representing the alarm image to a display device.

[0012] According to the image processing system of the present disclosure, a notification image having a different level of notification according to the biopsy certainty level is generated, and the notification image is displayed superimposed on a medical image. This allows a notification according to the biopsy certainty level to be implemented, and an area in the notification image where the level of notification is reduced can be prevented from obstructing the view of a user such as a doctor.

[0013] In an image processing system according to another aspect, the processor sets a threshold for the biopsy certainty, and displays a biopsy certainty exceeding the threshold as an alarm image.

[0014] According to this aspect, a biopsy certainty exceeding a prescribed threshold value can be notified as the notification image.

[0015] In an image processing system according to another aspect, the processor generates an informative image having different image quality depending on the calculated biopsy certainty.

[0016] According to this aspect, the image quality can be applied as the degree of notification in the notification image.

[0017] In an image processing system according to another aspect, the processor generates an indicator image that reduces image quality in an area where the biopsy certainty factor is relatively low compared to an area where the biopsy certainty factor is relatively high.

[0018] According to this aspect, the visibility of the medical image can be improved in an area where the biopsy certainty is relatively high.

[0019] In an image processing system according to another aspect, the processor generates an informative image in which the saturation of the informative image is varied according to the calculated biopsy certainty.

[0020] According to this aspect, saturation can be applied as the degree of notification in the notification image.

[0021] In an image processing system according to another aspect, the processor generates an indicator image in which saturation of an area where the biopsy certainty is relatively low is reduced relative to an area where the biopsy certainty is relatively high.

[0022] According to this aspect, the visibility of the medical image can be improved in an area where the biopsy certainty is relatively high.

[0023] In an image processing system according to another aspect, the processor generates an informative image in which the luminance of the informative image is varied according to the calculated biopsy certainty.

[0024] According to this aspect, the brightness can be used as the degree of notification in the notification image.

[0025] In an image processing system according to another aspect, the processor generates an indicator image in which the brightness of an area where the biopsy certainty is relatively low is reduced relative to an area where the biopsy certainty is relatively high.

[0026] According to this aspect, the visibility of the medical image can be improved in an area where the biopsy certainty is relatively high.

[0027] In another aspect of the image processing system, the processor measures the elapsed period from the start timing of displaying the alarm image, and stops transmitting the display signal representing the alarm image when the elapsed period from the start timing of displaying the alarm image exceeds a specified period.

[0028] According to this aspect, the notification image can be hidden, so that the notification image does not interfere with the user's visibility.

[0029] In another aspect of the image processing system, the processor detects whether or not a biopsy has been performed, and when a detection result indicating that a biopsy has been performed is obtained, stops transmission of a display signal representing an alarm image.

[0030] According to this aspect, the notification image can be hidden when a biopsy is performed, so that the notification image does not interfere with the user's view.

[0031] In an image processing system according to another aspect, a processor calculates a lesion certainty indicating a likelihood of a lesion for a region of interest, and calculates a biopsy certainty for a region of interest whose lesion certainty exceeds a specified value.

[0032] According to this embodiment, the lesion and othersA biopsy confidence level can be derived according to the severity.

[0033] The processor device according to the present disclosure is a processor device that includes one or more processors and controls an endoscope, in which the processor acquires an endoscopic image from the endoscope, detects a region of interest from the endoscopic image, calculates a biopsy certainty for the region of interest that indicates the certainty of the biopsy region being the target of the biopsy, generates an alarm image having a different degree of notification depending on the calculated biopsy certainty, the alarm image being displayed superimposed on the endoscopic image, and transmits a display signal representing the alarm image to a display device.

[0034] According to the processor device of the present disclosure, it is possible to obtain the same effects as those of the image processing system of the present disclosure. The processor device of the present disclosure may employ components of other aspects of the image processing system of the present disclosure.

[0035] The endoscopic system according to the present disclosure is an endoscopic system including an endoscope, a processor device that controls the endoscope, and one or more processors, in which the processor acquires an endoscopic image from the endoscope, detects a region of interest from the endoscopic image, calculates a biopsy certainty for the region of interest that indicates the certainty of the biopsy region being the target of the biopsy, generates an alarm image having a different degree of notification depending on the calculated biopsy certainty, the alarm image being displayed superimposed on the endoscopic image, and transmits a display signal representing the alarm image to a display device.

[0036] According to the endoscope system according to the present disclosure, it is possible to obtain the same effects as those of the image processing system according to the present disclosure. In the endoscope system according to the present disclosure, components of other aspects of the image processing system according to the present disclosure may be adopted.

[0037] The image processing method according to the present disclosure includes acquiring a medical image, detecting an area of ​​interest from the medical image, calculating a biopsy certainty for the area of ​​interest that indicates the certainty of the biopsy area being the target of the biopsy, generating a notification image that has a different degree of notification depending on the calculated biopsy certainty and is displayed superimposed on the medical image, and transmitting a display signal representing the notification image to a display device. It is an image processing method. .

[0038] According to the image processing method of the present disclosure, it is possible to obtain the same effects as those of the image processing system of the present disclosure. In the image processing method of the present disclosure, components of other aspects of the image processing system of the present disclosure may be adopted.

[0039] The program of the present disclosure is a program that enables a computer to realize the functions of acquiring a medical image, detecting an area of ​​interest from the medical image, calculating a biopsy certainty for the area of ​​interest that indicates the certainty that the biopsy area will be subject to biopsy, generating an alarm image that has a different degree of notification depending on the calculated biopsy certainty and is displayed superimposed on the medical image, and transmitting a display signal representing the alarm image to a display device.

[0040] According to the program of the present disclosure, it is possible to obtain the same effect as the image processing system of the present disclosure. In the program of the present disclosure, components of other aspects of the image processing system of the present disclosure may be adopted. Effect of the Invention

[0041] According to the present invention, a notification image having a different level of notification according to the biopsy certainty is generated. This allows notification according to the biopsy certainty to be performed, and the area in the notification image where the level of notification is reduced can be prevented from obstructing the view of a user such as a doctor. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 is a diagram showing the overall configuration of an endoscope system according to a first embodiment. [Diagram 2] FIG. 2 is a functional block diagram of the endoscope system shown in FIG. [Diagram 3] 3 is a block diagram of the endoscope image processing unit shown in FIG. [Figure 4] FIG. 4 is a flowchart showing the procedure of the image processing method according to the first embodiment. [Diagram 5] FIG. 5 is a schematic diagram of a notification image according to the first example. [Figure 6] FIG. 6 is a schematic diagram of a notification image according to the second example. [Figure 7] FIG. 7 is a functional block diagram of an endoscope system according to the second embodiment. [Figure 8] FIG. 8 is a flowchart showing the procedure of an image processing method according to the second embodiment. [Figure 9] FIG. 9 is a schematic diagram of the fibers in the notification image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification, the same components are given the same reference numerals and duplicated explanations will be omitted as appropriate.

[0044] [Endoscope system according to the first embodiment] [Overall configuration of endoscope system] 1 is a diagram showing the overall configuration of an endoscope system according to a first embodiment. The endoscope system 10 includes an endoscope body 100, a processor device 200, a light source device 300, and a display device 400. Note that in this figure, a part of a hard tip section 116 provided in the endoscope body 100 is illustrated in an enlarged manner.

[0045] [Example of the configuration of the endoscope body] The endoscope main body 100 includes a hand-operated operation unit 102 and an insertion unit 104. A user grasps and operates the hand-operated operation unit 102, and inserts the insertion unit 104 into the body of a subject to observe the inside of the subject's body. The term "user" is synonymous with a doctor, an operator, and the like. The term "subject" is also synonymous with a patient and an examinee.

[0046] The handheld operation unit 102 includes an air / water supply button 141, a suction button 142, a function button 143, and an image capture button 144. The air / water supply button 141 receives an air supply instruction and a water supply instruction.

[0047] The suction button 142 receives a suction instruction. Various functions are assigned to the function button 143. The function button 143 receives instructions for various functions. The image capture button 144 receives an image capture instruction operation. Image capture includes moving image capture and still image capture.

[0048] The insertion section 104 includes a flexible section 112, a bending section 114, and a tip rigid section 116. The flexible section 112, the bending section 114, and the tip rigid section 116 are arranged in this order from the side of the handheld operation section 102: the flexible section 112, the bending section 114, and the tip rigid section 116. That is, the bending section 114 is connected to the base end side of the tip rigid section 116, the flexible section 112 is connected to the base end side of the bending section 114, and the handheld operation section 102 is connected to the base end side of the insertion section 104.

[0049] A user can bend the bending section 114 by operating the hand operation section 102 to change the orientation of the tip rigid section 116 up, down, left, or right. The tip rigid section 116 includes an imaging section, an illumination section, and a forceps port 126.

[0050] Fig. 1 illustrates an imaging lens 132 that constitutes the imaging unit. Also illustrated in the same figure are illumination lenses 123A and 123B that constitute the illumination unit. The imaging unit is shown in Fig. 2 with the reference numeral 130. Also illustrated in Fig. 2 with the reference numeral 123.

[0051] During observation and treatment, at least one of white light and narrow band light is output via illumination lens 123A and illumination lens 123B in response to the operation of operation unit 208 shown in FIG.

[0052] When the air / water supply button 141 is operated, cleaning water is discharged from the water supply nozzle, or gas is discharged from the air supply nozzle. The cleaning water and gas are used to clean the illumination lens 123A and the like. The water supply nozzle and the air supply nozzle are not shown. The water supply nozzle and the air supply nozzle may be a common nozzle.

[0053] The forceps port 126 communicates with the duct. A treatment tool is inserted into the duct. The treatment tool is supported so that it can move forward and backward as appropriate. When removing a tumor or the like, the treatment tool is applied to perform the necessary treatment. Reference numeral 106 in FIG. 1 denotes a universal cable. Reference numeral 108 denotes a light guide connector.

[0054] 2 is a functional block diagram of the endoscope system. The endoscope body 100 includes an imaging unit 130. The imaging unit 130 is disposed inside the distal end rigid portion 116. The imaging unit 130 includes a photographing lens 132, an imaging element 134, a driving circuit 136, and an analog front end 138.

[0055] The photographing lens 132 is disposed on the tip end surface 116A of the tip hard portion 116. An imaging element 134 is disposed on the opposite side of the photographing lens 132 from the tip end surface 116A. A CMOS type image sensor is applied as the imaging element 134. A CCD type image sensor may also be applied as the imaging element 134. Note that CMOS is an abbreviation for Complementary Metal-Oxide Semiconductor. CCD is an abbreviation for Charge Coupled Device.

[0056] A color imaging element is applied to the imaging element 134. An example of a color imaging element is an imaging element equipped with color filters corresponding to RGB. Note that RGB stands for red, green, and blue, respectively, in English. blue are the initials.

[0057] A monochrome imaging element may be applied to the imaging element 134. When a monochrome imaging element is applied to the imaging element 134, the imaging section 130 can switch the wavelength band of the light incident on the imaging element 134 to perform frame sequential or color sequential imaging.

[0058] The drive circuit 136 supplies various timing signals required for the operation of the image sensor 134 to the image sensor 134 based on control signals transmitted from the processor unit 200 .

[0059] The analog front end 138 includes an amplifier, a filter, and an AD converter. Note that AD is the initial letter of analog and digital, which are the English terms for analog and digital, respectively. The analog front end 138 performs processes such as amplification, noise removal, and analog-to-digital conversion on the output signal of the image sensor 134. The output signal of the analog front end 138 is transmitted to the processor device 200. Note that AFE shown in FIG. 2 is an abbreviation of Analog Front End, which is the English term for analog front end.

[0060] An optical image of the observation target is formed on the light receiving surface of the image sensor 134 via the photographing lens 132. The image sensor 134 converts the optical image of the observation target into an electrical signal. The electrical signal output from the image sensor 134 is transmitted to the processor device 200 via a signal line.

[0061] The illumination unit 123 is disposed on the distal end hard portion 116. The illumination unit 123 includes an illumination lens 123A and an illumination lens 123B. The illumination lens 123A and the illumination lens 123B are disposed adjacent to the photographing lens 132 on the distal end surface 116A.

[0062] The illumination unit 123 includes a light guide 170. An exit end of the light guide 170 is disposed at a position opposite to the tip end faces 116A of the illumination lenses 123A and 123B.

[0063] 1, the light guide 170 is inserted into the insertion section 104, the handheld operation section 102, and the universal cable 106. The incident end of the light guide 170 is disposed inside the light guide connector 108.

[0064] [Example of the configuration of the processor device] The processor device 200 includes an imaging signal acquisition unit 202, an imaging signal processing unit 204, a communication control unit 205, and a video output unit 206. The imaging signal acquisition unit 202 acquires an electrical signal transmitted from the endoscope body 100 and corresponding to an optical image of the observation target.

[0065] The imaging signal processor 204 generates an endoscopic image of the observation target based on the imaging signal, which is an electrical signal corresponding to an optical image of the observation target. The endoscopic image is illustrated in FIG.

[0066] The imaging signal processor 204 may perform image quality correction by applying digital signal processing such as white balance processing and shading correction processing to the imaging signal. The imaging signal processor 204 may add supplementary information defined by the DICOM standard to the endoscopic image. DICOM is an abbreviation for Digital Imaging and Communications in Medicine.

[0067] The video output unit 206 transmits a display signal representing the image generated by the imaging signal processing unit 204 to the display device 400. The display device 400 displays the image of the object to be observed.

[0068] The processor device 200 operates the imaging signal acquisition section 202, the imaging signal processing section 204, etc. in response to an imaging command signal transmitted from the endoscope body 100 when the imaging button 144 shown in FIG.

[0069] When the processor device 200 receives a freeze command signal representing capturing a still image from the endoscope body 100, it applies the image capturing signal processing unit 204 to generate a still image based on a frame image at the operation timing of the image capturing button 144. The processor device 200 causes the display device 400 to display the still image. The frame image is indicated by reference numeral 38B and shown in FIG. 3. The still image is indicated by reference numeral 39 and shown in FIG. 3.

[0070] The processor device 200 includes a communication control unit 205. The communication control unit 205 controls communication with devices communicably connected via an intra-hospital system and an intra-hospital LAN or the like. The communication control unit 205 may apply a communication protocol conforming to the DICOM standard. An example of an intra-hospital system is a Hospital Information System (HIS). LAN is an abbreviation for Local Area Network.

[0071] The processor device 200 includes a storage unit 207. The storage unit 207 stores an endoscopic image generated using the endoscope main body 100. The storage unit 207 may store various information associated with the endoscopic image.

[0072] The processor device 200 includes an operation unit 208. The operation unit 208 outputs a command signal in response to a user's operation. The operation unit 208 may be a keyboard, a mouse, a joystick, or the like.

[0073] The processor device 200 includes an audio processor 209 and a speaker 209A. The audio processor 209 generates an audio signal representing information to be notified as audio. The speaker 209A converts the audio signal generated by the audio processor 209 into audio. Examples of audio output from the speaker 209A include messages, audio guidance, and warning sounds.

[0074] The processor device 200 includes a system control unit 210, a ROM 211, and a RAM 212. Note that ROM is an abbreviation for Read Only Memory, and RAM is an abbreviation for Random Access Memory.

[0075] The system control unit 210 functions as an overall control unit for the processor device 200. The system control unit 210 functions as a memory controller that controls a ROM 211 and a RAM 212. The ROM 211 stores various programs and control parameters applied to the processor device 200.

[0076] The RAM 212 is used as a temporary storage area for data in various processes and as a processing area for calculation processes using the system control unit 210. The RAM 212 can be used as a buffer memory when an endoscopic image is acquired.

[0077] The processor device 200 performs various processes on the endoscopic image generated by the endoscope body 100, and displays the endoscopic image and various information associated with the endoscopic image by using the display device 400. The processor device 200 stores the endoscopic image and various information associated with the endoscopic image.

[0078] That is, during endoscopic examination using the endoscope body 100, the processor unit 200 displays endoscopic images and the like using the display unit 400, outputs audio information using the speaker 209A, and performs various processes on the endoscopic images.

[0079] The processor device 200 includes an endoscopic image processing unit 220. The endoscopic image processing unit 220 performs various types of image processing on the endoscopic image generated by the imaging signal processing unit 204.

[0080] [Hardware configuration of the processor unit] A computer may be used as the processor device 200. The computer may use the following hardware and execute a specified program to realize the functions of the processor device 200. Note that the program is synonymous with software.

[0081] The processor device 200 may employ various processor devices as a signal processing unit that performs signal processing. Examples of processor devices include a CPU and a GPU (Graphics Processing Unit). A CPU is a general-purpose processor device that executes a program and functions as a signal processing unit. A GPU is a processor device specialized for image processing. An electric circuit that combines electric circuit elements such as semiconductor elements is applied to the hardware of the processor device. Each control unit includes a ROM in which programs and the like are stored and a RAM that is a working area for various calculations.

[0082] Two or more processor devices may be applied to one signal processing unit. The two or more processor devices may be the same type of processor device or different types of processor devices. Also, one processor device may be applied to multiple signal processing units. The processor device 200 described in the embodiment corresponds to an example of an endoscope control unit.

[0083] [Configuration example of light source device] The light source device 300 includes a light source 310, an aperture 330, a condenser lens 340, and a light source control unit 350. The light source device 300 causes observation light to enter the light guide 170. The light source 310 includes a red light source 310R, a green light source 310G, and a blue light source 310B. The red light source 310R, the green light source 310G, and the blue light source 310B emit narrowband light of red, green, and blue, respectively.

[0084] The light source 310 may generate illumination light that is any combination of narrow-band light of red, green, and blue. For example, the light source 310 may generate white light by combining narrow-band light of red, green, and blue. The light source 310 may also generate narrow-band light by combining any two of narrow-band light of red, green, and blue.

[0085] The light source 310 may generate narrowband light using any one of red, green, and blue narrowband light. The light source 310 may selectively emit white light or narrowband light. Note that narrowband light is synonymous with special light. The light source 310 may include an infrared light source that emits infrared light, an ultraviolet light source that emits ultraviolet light, and the like.

[0086] The light source 310 may employ an embodiment including a white light source that emits white light, a filter that passes white light, and a filter that passes narrow-band light. The light source 310 in this embodiment may selectively emit either white light or narrow-band light by switching between the filter that passes white light and the filter that passes narrow-band light.

[0087] The filter that passes the narrowband light may include a plurality of filters corresponding to different bands. The light source 310 may selectively switch between the plurality of filters corresponding to different bands to selectively emit a plurality of narrowband lights having different bands.

[0088] The type and wavelength band of the light source 310 may be selected according to the type of object to be observed and the purpose of the observation. Examples of the types of the light source 310 include a laser light source, a xenon light source, and an LED light source. LED is an abbreviation for Light-Emitting Diode.

[0089] When the light guide connector 108 is connected to the light source device 300, the observation light emitted from the light source 310 passes through the aperture 330 and the condenser lens 340 and reaches the incident end of the light guide 170. The observation light passes through the light guide 170 and the illumination lens 123A, etc., and is irradiated onto the observation object.

[0090] The light source control unit 350 transmits control signals to the light source 310 and the aperture 330 based on a command signal transmitted from the processor device 200. The light source control unit 350 controls the illuminance of the observation light emitted from the light source 310, switching of the observation light, and turning the observation light on and off.

[0091] [Example of the configuration of an endoscope image processing unit] Fig. 3 is a block diagram of the endoscope image processing unit shown in Fig. 1. For convenience of explanation, Fig. 3 illustrates some of the components of the processor device 200 shown in Fig. 2.

[0092] The endoscopic image processing unit 220 includes a processor device 240. The processor device 240 includes functions as an endoscopic image acquisition unit 222, a lesion area detection unit 224, a biopsy candidate area recognition unit 226, a confidence factor calculation unit 228, and a notification information generation unit 230. The endoscopic image processing unit 220 includes a notification information storage unit 232. The processor device 240 described in the embodiment is an example of one or more processors.

[0093] The endoscopic image acquisition unit 222 acquires an endoscopic image 38 captured using the endoscope body 100 shown in Fig. 1. Hereinafter, acquisition of the endoscopic image 38 may include acquisition of a moving image 38A, acquisition of a frame image 38B, and acquisition of a still image 39.

[0094] The endoscopic image acquiring section 222 can acquire a moving image 38A composed of time-series frame images 38B. The endoscopic image acquiring section 222 can acquire a still image 39 when a still image is captured during capturing of the moving image 38A. The endoscopic image acquiring section 222 stores the acquired endoscopic image 38. The RAM 212 can be used to store the endoscopic image 38.

[0095] Lesion area detection unit 224 automatically detects a lesion area, which is an area suspected to be a lesion, as an area of ​​interest from each of frame images 38B constituting moving image 38A or from still image 39. Lesion area detection unit 224 stores information about the detected lesion area in association with frame image 38B or still image 39. RAM 212 may be used to store the information about the lesion area.

[0096] The information on the lesion area may include information on the position of the lesion area. Examples of lesions include tumors, polyps, ulcers, and inflammation. The lesion area detection unit 224 may apply a method based on known image processing. The lesion area detection unit 224 may apply a learning model that has been trained.

[0097] Biopsy candidate region recognition unit 226 recognizes biopsy candidate regions that are candidates for the biopsy region, which is a region to be biopsied, from the lesion region detected from endoscopic image 38 using lesion region detection unit 224. When multiple lesion regions are detected, a biopsy candidate region is recognized for each lesion region. A biopsy candidate region may be recognized when the lesion region is equal to or larger than a specified area.

[0098] The certainty calculation section 228 calculates a biopsy certainty that indicates the likelihood of a biopsy candidate region being a biopsy region. The certainty calculation section 228 can calculate the biopsy certainty as a score and probability for each pixel that constitutes the biopsy candidate region.

[0099] The notification information generating unit 230 generates notification information for notifying the biopsy certainty. An example of the notification information is a notification image superimposed on an endoscopic image displayed on the display device 400. The notification information generating unit 230 generates notification images with different degrees of notification according to the biopsy certainty.

[0100] The notification information storage unit 232 stores notification information including a notification image. The notification information storage unit 232 may store the notification information in association with information on a lesion area and information on a biopsy candidate area.

[0101] [Procedure of image processing method according to the first embodiment] Fig. 4 is a flow chart showing the procedure of the image processing method according to the first embodiment. In the endoscopic image acquiring step S10, the endoscopic image acquiring unit 222 shown in Fig. 3 acquires the endoscopic image 38. After the endoscopic image acquiring step S10, the process proceeds to the lesion area detecting step S12.

[0102] In lesion area detection step S12, lesion area detection unit 224 detects a lesion area from each of frame images 38B constituting endoscopic image 38 or from still image 39. Lesion area detection unit 224 stores information on the detected lesion area. After lesion area detection step S12, the process proceeds to lesion area determination step S14.

[0103] In lesion area determination step S14, biopsy candidate area recognition unit 226 determines whether or not a lesion area has been detected in frame image 38B or still image 39. If biopsy candidate area recognition unit 226 determines in lesion area determination step S14 that no lesion area exists in frame image 38B or still image 39 being determined, the determination is No. If the determination is No, the process proceeds to end determination step S24.

[0104] On the other hand, in lesion area determination step S14, a Yes determination is made when biopsy candidate area recognition unit 226 determines that a lesion area exists in frame image 38B or still image 39 to be determined. If a Yes determination is made, the process proceeds to biopsy candidate area recognition step S16.

[0105] In the biopsy candidate region recognition step S16, the biopsy candidate region recognition unit 226 recognizes a biopsy candidate region for a lesion region. After the biopsy candidate region recognition step S16, the process proceeds to a certainty factor calculation step S18.

[0106] In the certainty calculation step S18, the certainty calculation section 228 calculates the biopsy certainty for the biopsy candidate region detected in the biopsy candidate region recognition step S16. After the certainty calculation step S18, the process proceeds to the notification information generation step S20.

[0107] In the notification information generating step S20, the notification information generating unit 230 generates a notification image to be superimposed on the endoscopic image as notification information based on the biopsy certainty generated in the certainty calculating step S18. The notification information generating unit 230 stores the notification image in the notification information storage unit 232. After the notification information generating step S20, the process proceeds to a notification image displaying step S22.

[0108] In the notification image display step S22, the notification information generation unit 230 transmits a video signal representing a notification image to the display device 400 via the video output unit 206. The display device 400 displays the notification image superimposed on the endoscopic image 38. After the notification image display step S22, the process proceeds to an end determination step S24.

[0109] In the end determination step S24, the endoscopic image acquisition unit 222 determines whether or not the acquisition of the endoscopic image has ended. The endoscopic image acquisition unit 222 can end the acquisition of the endoscopic image when it has acquired the final frame image 38B of the endoscopic image 38. The endoscopic image acquisition unit 222 can end the acquisition of the endoscopic image when a certain period of time has passed since the acquisition of any endoscopic image.

[0110] In the end determination step S24, if it is determined that the endoscopic image acquisition unit 222 has not finished acquiring the endoscopic image, the determination is No. In the case of a No determination, the process proceeds to the endoscopic image acquisition step S10, and each step from the endoscopic image acquisition step S10 to the end determination step S24 is repeatedly performed until the end determination step S24 is determined to be Yes.

[0111] On the other hand, if it is determined in the end determination step S24 that the endoscopic image acquisition section 222 has completed acquisition of the endoscopic image, the determination is Yes. If the determination is Yes, the processor device 240 ends the image processing method.

[0112] [Examples of notification images] Fig. 5 is a schematic diagram of a notification image according to the first example. Fig. 5 shows a schematic diagram of an arbitrary frame image 38B constituting an endoscopic image 38 acquired when an endoscopic examination of a large intestine is performed. The same applies to Figs. 6 and 9.

[0113] 5, an alarm image 502 is displayed superimposed on a lesion area 500. In the endoscopic image 38, a center position 506 of the lesion area 500 and a bounding box 508 surrounding the periphery of the lesion area 500 are displayed.

[0114] The notification image 502 shown in Fig. 5 has a size and shape that covers the entire biopsy candidate region 504. The notification image 502 has a different degree of notification depending on the biopsy certainty. 。 The notification image 502 includes a first area 510 , a second area 512 , and a third area 514 .

[0115] The first region 510 is a region where the biopsy confidence is maximum, and is a region including the center position of the lesion region 500. The second region 512 is a region where the biopsy confidence is lower than that of the first region 510, and is a region outside the first region 510. The third region 514 is a region where the biopsy confidence is lower than that of the second region 512, and is a region outside the second region 512.

[0116] The first region 510 is the region with the lowest degree of notification, and is displayed close to the normal image display of the endoscopic image 38. The second region 512 is the region with a higher degree of notification than the first region 510, and the first region 510 is emphasized. The third region 514 is the region with a higher degree of notification than the second region 512, and the first region 510 is emphasized in the same manner as the second region 512.

[0117] That is, the notification image 502 can improve the user's visibility of the biopsy candidate region 504, and can also suppress obstruction of the user's view of an area of ​​the biopsy candidate region where the biopsy certainty is relatively high.

[0118] An example of varying the degree of notification is to control image quality. Examples of image quality include saturation and brightness. Another example of image quality is resolution. That is, the saturation, brightness, or resolution of first region 510, second region 512, and third region 514 shown in FIG. 5 may be decreased as the distance from center position 506 of lesion region 500 increases.

[0119] The range in which the luminance and the like of the notification image 502 are changed is limited to a range that can be changed using the processor device 200 shown in Fig. 2. That is, the degree of notification of the notification image 502 is limited to a range that can be adjusted by the user who uses the endoscope system 10. Examples of parameters that can be set by the user include luminance, saturation, hue, and white balance.

[0120] As a result, the notification image 502 is little changed from the conventional one and does not deviate too far from the normal observation display of the endoscopic image 38, making it easy for the user to observe.

[0121] Fig. 6 is a schematic diagram of a notification image according to the second example. In the notification image 520 shown in the figure, a threshold value for the biopsy certainty is set, and the degree of notification is changed depending on whether the threshold is exceeded. The notification image 520 shown in Fig. 6 is composed of an area where the biopsy certainty exceeds the threshold. This makes it possible to provide a notification image 520 that allows the user to intuitively determine the biopsy area.

[0122] 3 includes a threshold setting unit that sets a threshold applied to the biopsy certainty factor. The threshold setting unit can set a threshold based on information input by a user.

[0123] [Modification of notification image display] The notification image 502 may be automatically hidden when a specified period of time has elapsed since the display start timing of the notification image 502. The specified period of time may be fixed or may be adjustable.

[0124] That is, the processor device 240 shown in FIG. 3 can measure the elapsed time from the display start timing of the notification image 502, and automatically hide the notification image 502 when a specified period has elapsed from the measurement start timing.

[0125] The user may determine that confirmation of the biopsy region has been completed after a certain period of time has elapsed since the start of display of the notification image 502. If the notification image 502 is automatically hidden, the user will not be bothered.

[0126] The notification image 502 may be hidden in response to a command signal for hiding the notification image 502. For example, a command signal for hiding the notification image 502 may be transmitted in response to a user's operation of a button or the like, and the notification image 502 may be hidden. Note that the display control of the notification image 502 according to the modified example is also applicable to the notification image 520 shown in FIG. 6.

[0127] [Specific examples of areas to be biopsied] The biopsy candidate region may be recognized based on a lesion certainty that indicates the likelihood of a lesion region in the lesion region 500. For example, a region having a lesion certainty that exceeds a specified threshold may be determined as a biopsy candidate region. The biopsy candidate region may be defined as a region that includes the center position 506 of the lesion region 500.

[0128] The biopsy candidate region may be recognized using a trained learning model. The learning model may apply CNN. CNN is an abbreviation of Convolutional Neural Network, which is an English abbreviation for convolutional neural network.

[0129] [Specific examples of biopsy confidence] The biopsy confidence level is calculated for each pixel that constitutes the biopsy candidate region 504. Each of a first region 510, a second region 512, and a third region 514 shown in Fig. 5 represents a set of multiple pixels having the same biopsy confidence level.

[0130] When a biopsy candidate region is recognized based on a lesion certainty, the lesion certainty in the biopsy candidate region may be taken as the biopsy certainty.

[0131] [Effects of the First Embodiment] According to the endoscope system and image processing method according to the first embodiment, the following advantageous effects can be obtained.

[0132] [1] A biopsy candidate region 504 is recognized in the lesion region 500, a biopsy certainty level is calculated for the biopsy candidate region 504, and a notification image 502 with a different notification level according to the biopsy certainty level is generated, and the notification image 502 is superimposed on the endoscopic image 38. In the notification image 502, the first region 510 with a relatively high biopsy certainty level has a lower notification level compared to a second region 512 with a relatively low biopsy certainty level, etc. This enables notification of the biopsy certainty level without interfering with the user's field of vision.

[0133] [2] The notification image 502 reduces the image quality of the second region 512 and the third region 514 as compared to the first region 510. This allows the first region 510 to be observed in the same state as in normal observation of the endoscopic image 38, and also makes it possible to emphasize the first region 510.

[0134] [3] The notification image 502 reduces the luminance and saturation of the second region 512 and the third region 514 compared to the first region 510. This allows the first region 510 to be observed in the same state as in normal observation of the endoscopic image 38, and also makes it possible to emphasize the first region 510.

[0135] [4] The range in which the luminance and the like are varied in the notification image 502 is limited to the range that can be adjusted in the endoscope system 10. As a result, the endoscopic image 38 on which the notification image 502 is superimposed does not deviate far from the normal display of the endoscopic image 38, and it is possible to display the endoscopic image 38 on which the notification image 502 is superimposed and which is easy to observe.

[0136] [5] The notification image 502 is automatically hidden after a specified period has elapsed from the display start timing of the notification image 502. This can prevent the user from being annoyed when the notification image 502 is hidden.

[0137] [Endoscope system according to the second embodiment] Fig. 7 is a functional block diagram of an endoscope system according to a second embodiment. The endoscope system 10A shown in the figure is different from the endoscope system 10 shown in Fig. 1 etc. in the configuration of a processor device 240A provided in an endoscope image processing unit 220A.

[0138] That is, in the endoscope system 10A, the processor device 240A is provided with a biopsy instrument detection unit 234. The biopsy instrument detection unit 234 detects whether or not the endoscopic image 38 includes a biopsy instrument such as forceps.

[0139] When the biopsy instrument detection unit 234 obtains a detection result that detects a biopsy instrument, the video output unit 206 stops the superimposition of the notification image on the endoscopic image 38. When the endoscopic image 38 includes a biopsy instrument, it is determined that the user has already recognized the biopsy area. When a biopsy instrument is detected, the superimposition of the notification image is automatically stopped, allowing the biopsy area to be observed in detail.

[0140] [Procedure of image processing method according to the second embodiment] 8 is a flowchart showing the procedure of an image processing method according to the second embodiment. In the flowchart shown in this figure, a biopsy instrument detection step S30 and a notification image non-display step S32 are added to the flowchart shown in FIG.

[0141] In the notification image display step S22, notification image 502 is superimposed on endoscopic image 38 on display device 400, and the process proceeds to biopsy instrument detection step S30. In biopsy instrument detection step S30, biopsy instrument detection unit 234 shown in FIG.

[0142] In the biopsy instrument detection step S30, if the biopsy instrument detection unit 234 detects a biopsy instrument from the endoscopic image 38, a Yes judgment is made. If the Yes judgment is made, the process proceeds to the notification image non-display step S32. On the other hand, in the biopsy instrument detection step S30, if the biopsy instrument detection unit 234 does not detect a biopsy instrument from the endoscopic image 38, a No judgment is made. If the No judgment is made, the process proceeds to the end judgment step S24.

[0143] In the notification image non-display step S32, the notification information generating unit 230 hides the notification image 502 superimposed on the endoscopic image 38 via the video output unit 206. After the notification image 502 is hidden in the notification image non-display step S32, the process proceeds to the end determination step S24.

[0144] [Examples of notification images] 9 is a schematic diagram of the transition of the notification image. Reference numeral 600 in the figure typically indicates the endoscopic image 38 when a biopsy instrument 602 is detected. Reference numeral 610 typically indicates the endoscopic image 38 in which the notification image 502 is not displayed. In the notification image 502 shown in the schematic diagram 600 of the endoscopic image, the first region 510 and the like shown in FIG. 5 are omitted.

[0145] In the endoscopic image 38 illustrated as the schematic diagram 610 of the endoscopic image, the bounding box that highlights the lesion area 500 is hidden, but the bounding box may be displayed. Note that the biopsy instrument 602 shown in Fig. 9 is an example of a treatment instrument.

[0146] [Modification of the second embodiment] During a biopsy, there may be cases where it is desired to recognize the position of a biopsy candidate region. In this case, when the biopsy instrument 602 shown in Fig. 9 is detected, the notification image 502 may be allowed to continue to be displayed, and when the biopsy instrument 602 is no longer detected, it may be determined that the biopsy has ended, and the notification image 502 may be automatically hidden.

[0147] In the image processing method according to the modified example, a biopsy instrument non-detection step is added between the biopsy instrument detection step S30 and the notification image non-display step S32 in the flowchart shown in Fig. 8. In the biopsy instrument non-detection step, biopsy instrument detection section 234 shown in Fig. 7 determines whether or not the biopsy instrument detected in the biopsy instrument detection step S30 becomes undetected.

[0148] In the biopsy instrument non-detection step, if biopsy instrument detection unit 234 detects a biopsy instrument, the verdict is No. If the verdict is No, the biopsy instrument non-detection step is repeatedly performed until the verdict is Yes in the biopsy instrument non-detection step.

[0149] On the other hand, in the biopsy instrument non-detection step, if the biopsy instrument detection unit 234 determines that the biopsy instrument has not been detected, a Yes determination is made. If the Yes determination is made, the process proceeds to the notification image non-display step S32. After the notification image non-display step S32 is performed, the end determination step S24 is performed.

[0150] [Effects of the second embodiment] No. two According to the endoscope system and image processing method according to the embodiment, the following advantageous effects can be obtained.

[0151] [1] When the biopsy instrument 602 is detected from the endoscopic image 38, the notification image 502 is hidden. This can prevent the notification image 502 from interfering with the user's view during a biopsy.

[0152] [2] When the biopsy instrument 602 is detected from the endoscopic image 38 and then the biopsy instrument 602 is not detected from the endoscopic image 38, the notification image 502 is hidden. This makes it possible to recognize the biopsy candidate region during the biopsy, and also prevents the notification image 502 from interfering with the user's view.

[0153] [Hardware configuration of the processing unit] Various processor devices may be applied to the various processing units shown in Figures 2, 3, 7, etc. The processing units may be called processing units. The various processor devices include a CPU (Central Processing Unit), a PLD (Programmable Logic Device), and an ASIC (Application Specific Integrated Circuit), etc.

[0154] A CPU is a general-purpose processor device that executes programs and functions as various processing units. A PLD is a processor device whose circuit configuration can be changed after manufacture. An example of a PLD is the Field Programmable Gate Array (FPGA). An ASIC is a dedicated electrical circuit with a circuit configuration designed specifically to perform a specific process.

[0155] A single processing unit may be configured with one of these various processor devices, or may be configured with two or more processor devices of the same or different types. For example, a single processing unit may be configured using multiple FPGAs, etc. A single processing unit may be configured by combining one or more FPGAs and one or more CPUs.

[0156] In addition, multiple processing units may be configured using one processor device. As an example of configuring multiple processing units using one processor device, one or more CPUs and software may be combined to form one processor. device In one embodiment, a single processor device functions as a plurality of processing units. This type of configuration is typified by computers such as client terminal devices and server devices.

[0157] As another configuration example 、 One example is a form that uses a processor device that realizes the functions of an entire system including multiple processing units using a single IC chip. This form is typified by system-on-chip. Note that IC is an abbreviation for integrated circuit. System-on-chip is sometimes abbreviated as SoC, which is an abbreviation for system-on-chip.

[0158] In this way, the various processing units are configured as a hardware structure using one or more of the various processor devices described above. More specifically, the hardware structure of the various processor devices is an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0159] [Modification of the endoscope system] [Modification of illumination light] The endoscope system 10 shown in FIG. etc. An example of a medical image that can be acquired using the above method is a normal light image obtained by irradiating a white band of light or light in a plurality of wavelength bands as the white band of light.

[0160] The endoscope system 10 shown in this embodiment etc. Another example of a medical image that can be obtained using the method is an image obtained by irradiating light of a specific wavelength band. The specific wavelength band can be a band narrower than the white band. The following modifications can be applied.

[0161] First Variation A first example of the particular wavelength band is the blue or green band of the visible range. The first example wavelength band includes a wavelength band from 390 nanometers to 450 nanometers or from 530 nanometers to 550 nanometers, and the first example light has a peak wavelength within the wavelength band from 390 nanometers to 450 nanometers or from 530 nanometers to 550 nanometers.

[0162] Second Modified Example A second example of the specific wavelength band is the red band of the visible range, the second example wavelength band includes a wavelength band from 585 nanometers to 615 nanometers or from 610 nanometers to 730 nanometers, and the second example light has a peak wavelength within the wavelength band from 585 nanometers to 615 nanometers or from 610 nanometers to 730 nanometers.

[0163] <Third Modification> A third example of the specific wavelength band includes a wavelength band where the absorption coefficients of oxygenated hemoglobin and reduced hemoglobin are different, and the light of the third example has a peak wavelength in the wavelength band where the absorption coefficients of oxygenated hemoglobin and reduced hemoglobin are different. The wavelength band of this third example includes a wavelength band of 400±10 nanometers, 440±10 nanometers, 470±10 nanometers, or a wavelength band of 600 nanometers or more and 750 nanometers or less, and the light of the third example has a peak wavelength in the wavelength band of 400±10 nanometers, 440±10 nanometers, 470±10 nanometers, or a wavelength band of 600 nanometers or more and 750 nanometers or less.

[0164] <Fourth modified example> A fourth example of a specific wavelength band is a wavelength band of excitation light that is used to observe the fluorescence emitted by fluorescent substances in living organisms and excites the fluorescent substances. For example, a wavelength band of 390 nanometers to 470 nanometers. Note that the observation of fluorescence is sometimes called fluorescence observation.

[0165] Fifth Modification A fifth example of the specific wavelength band is an infrared wavelength band, which includes a wavelength band from 790 nanometers to 820 nanometers or from 905 nanometers to 970 nanometers, and the fifth example light has a peak wavelength in the wavelength band from 790 nanometers to 820 nanometers or from 905 nanometers to 970 nanometers.

[0166] [Example of special light image generation] The processor device 200 may generate a special light image having information of a specific wavelength band based on a normal light image captured using white light. Note that the term "generate" here includes "acquire." In this case, the processor device 200 functions as a special light image acquirer. The processor device 200 acquires a signal of a specific wavelength band by performing a calculation based on color information of red, green, and blue, or cyan, magenta, and yellow, contained in the normal light image.

[0167] Cyan, magenta, and yellow are sometimes abbreviated as CMY, using the initials of their English names, Cyan, Magenta, and Yellow, respectively.

[0168] [Example of generating feature image] As medical images, feature images can be generated using calculations based on at least one of a normal light image obtained by irradiating white light or light of multiple wavelength bands as white light, and a special light image obtained by irradiating light of a specific wavelength band.

[0169] [Combinations of embodiments and modified examples] The components described in the above-mentioned embodiments can be used in appropriate combinations, and some of the components can be replaced.

[0170] [Example of application to image processing systems] The endoscopic image processing unit 220 shown in Fig. 3 and the endoscopic image processing unit 220A shown in Fig. 7 can be components of an image processing device having one or more processors and one or more memories. Such an image processing device can constitute a medical image processing system that superimposes and displays a notification image on not only endoscopic images but also medical images other than endoscopic images, such as CT images, MRI images, and ultrasound images, obtained from modalities other than endoscopic systems. Note that the medical image processing system represents a concept including medical image processing.

[0171] Here, medical images may include medical images. CT is an abbreviation for Computed Tomography. MRI is an abbreviation for Magnetic Resonance Imaging.

[0172] [Examples of application to programs] The above-mentioned endoscopic system and image processing method can be configured as a program that uses a computer to realize functions corresponding to each unit of the endoscopic image processing unit 220 and each step of the image processing method. Examples of functions that can be realized using a computer include a function to obtain a medical image, a function to detect a lesion area from a medical image, a function to recognize a biopsy candidate area from a lesion area, a function to calculate a biopsy certainty for the biopsy candidate area, a function to generate a notification image with a different degree of notification based on the biopsy certainty, and a function to transmit a signal to superimpose the notification image on the endoscopic image.

[0173] A program that causes a computer to realize the various functions described above can be stored in a computer-readable information storage medium, which is a tangible, non-transitory information storage medium, and the program can be provided through the information storage medium.

[0174] Further, instead of providing the program by storing it on a non-transitory information storage medium, a program signal may be provided via a communication network.

[0175] The above-described embodiment of the present invention may be modified, added, or deleted as appropriate within the scope of the gist of the present invention. The present invention is not limited to the above-described embodiment, and many modifications may be made by a person having ordinary knowledge in the relevant field within the technical concept of the present invention. [Explanation of symbols]

[0176] 10 Endoscope System 10A Endoscope System 38 Endoscopic Images 38A Video 38B Frame Image 39 Still Images 100 Endoscope body 102 Handheld operation unit 104 Insertion part 106 Universal Cable 108 Light guide connector 112 Soft part 114 Curved section 116 Hard tip 116A Tip side end surface 123 Lighting Department 123A Lighting Lens 123B Lighting Lens 126 Forceps mouth 130 Imaging unit 132 Photographic Lens 134 Image sensor 136 Drive circuit 138 Analog Front End 141 Air / water supply button 142 Suction button 143 Function button 144 Image capture button 170 Light Guide 200 Processor Unit 202 Image signal acquisition unit 204 Imaging signal processing section 205 Communication control section 206 Video output section 207 Memory section 208 Operation section 209 Audio Processing Unit 209A Speaker 210 System Control Unit 211 ROM 212 RAM 220 Endoscope image processing unit 220A Endoscope image processing unit 222 Endoscopic image acquisition unit 224 Lesion Area Detection Unit 226 Biopsy candidate area recognition unit 228 Confidence calculation part 230 Notification information generation unit 232 Notification information storage unit 240 Processor Device 240A Processor Device 300 Light source device 310 light source 310B Blue light source 310G green light source 310R red light source 330 Aperture 340 Condenser Lens 350 Light source control unit 400 Display Device 500 Lesion Areas 502 Notification Image 504 Biopsy candidate areas 506 Center position 508 Bounding Box 510 First area 512 Second area 514 Third Realm 520 Notification Image 600 Schematic diagram of endoscopic image 602 Biopsy Instruments 610 Schematic diagram of endoscopic image S10 to S32 Each step of the image processing method

Claims

1. 1. An image processing system comprising one or more processors, The processor, A color medical image is obtained by irradiating the image with illumination light from a white light source; Detecting an area of ​​interest from the medical image; Calculating a biopsy certainty for the region of interest, which indicates the certainty of the region being a biopsy target; generating an information image having a different degree of notification according to the calculated biopsy certainty degree, the information image being superimposed on the medical image; An image processing system that transmits a display signal representing the notification image to a display device.

2. The processor, setting a threshold for the biopsy confidence; The image processing system according to claim 1 , wherein a biopsy certainty factor exceeding the threshold value is displayed as an annunciation image.

3. The image processing system according to claim 1 , wherein the processor generates the notification image by varying image quality of the notification image according to the calculated biopsy certainty factor.

4. The image processing system according to claim 3 , wherein the processor generates the notification image in such a way that image quality of an area where the biopsy certainty factor is relatively low is reduced relative to an area where the biopsy certainty factor is relatively high.

5. The image processing system according to claim 3 , wherein the processor generates the notification image by varying a saturation of the notification image according to the calculated biopsy certainty factor.

6. The image processing system according to claim 5 , wherein the processor generates the notification image by lowering saturation in an area where the biopsy certainty factor is relatively small compared to an area where the biopsy certainty factor is relatively large.

7. The image processing system according to claim 3 , wherein the processor generates the notification image by varying a luminance of the notification image according to the calculated biopsy certainty degree.

8. The image processing system according to claim 7 , wherein the processor generates the notification image by lowering brightness in an area where the biopsy certainty factor is relatively small compared to an area where the biopsy certainty factor is relatively large.

9. The processor, Measure the elapsed time from the display start timing of the notification image, The image processing system according to claim 1 , wherein when a prescribed period of time has elapsed since a display start timing of the notification image has elapsed, transmission of a display signal representing the notification image is stopped.

10. The processor, Detecting whether a biopsy has been performed; The image processing system according to claim 1 , wherein when a detection result indicating the performance of a biopsy is obtained, transmission of the display signal representing the notification image is stopped.

11. The processor, Calculating a lesion certainty value representing a lesion likelihood for the region of interest; The image processing system according to claim 1 , wherein the biopsy certainty is calculated for the region of interest in which the lesion certainty exceeds a specified threshold.

12. A processor device for controlling an endoscope, comprising one or more processors, The processor, A color endoscopic image captured by irradiating the image with illumination light from a white light source is acquired from the endoscope; Detecting an area of ​​interest from the endoscopic image; Calculating a biopsy certainty for the region of interest, which indicates the certainty of the region being a biopsy target; generating a notification image having a different degree of notification according to the calculated biopsy certainty degree, the notification image being superimposed on the endoscopic image; A processor unit that transmits a display signal representing the notification image to a display device.

13. An endoscope and A processor device for controlling the endoscope; one or more processors; An endoscope system comprising: The processor, A color endoscopic image captured by irradiating the image with illumination light from a white light source is acquired from the endoscope; Detecting an area of ​​interest from the endoscopic image; Calculating a biopsy certainty for the region of interest, which indicates the certainty of the region being a biopsy target; generating a notification image having a different degree of notification according to the calculated biopsy certainty degree, the notification image being superimposed on the endoscopic image; An endoscope system that transmits a display signal representing the notification image to a display device.

14. A color medical image is obtained by irradiating illumination light from a white light source, Detecting an area of ​​interest from the medical image; Calculating a biopsy certainty for the region of interest, which indicates the certainty of the region being a biopsy target; generating an information image having a different degree of notification according to the calculated biopsy certainty degree, the information image being superimposed on the medical image; An image processing method for transmitting a display signal representing the notification image to a display device.

15. On the computer, A function for acquiring color medical images captured by irradiating the image with illumination light from a white light source; detecting an area of ​​interest from the medical image; A function of calculating a biopsy certainty representing the certainty of a biopsy region to be a target of a biopsy for the region of interest; a function of generating a notification image in which the degree of notification varies depending on the calculated biopsy certainty degree, and the notification image is displayed by being superimposed on the medical image; and A program for implementing a function of transmitting a display signal representing the notification image to a display device.

Images