Processor device and method for operating the processor device

Abstract

Provided is a processor device capable of obtaining accurate analysis results and a processor device operation method.　A processor device (16) in an endoscope system (10) is provided with a recognition processing unit (70), an image evaluation unit (72), and a warning processing unit (74). The recognition processing unit (70) analyzes endoscope images and performs a recognition process. The recognition process comprises a detection process for detecting areas of interest which include lesions or the like, and an identification process for identifying the lesion type and severity. The image evaluation unit (72) evaluates whether endoscope images fulfill prescribed criteria required for the recognition process. In situations in which endoscope images are evaluated as not fulfilling the prescribed criteria required for the recognition process, the warning processing unit (74) performs a warning process to provide warning thereof.

Description

[Technical field]

[0001] The present invention relates to a processor device and an operating method of the processor device for obtaining analysis results of an endoscopic image captured by an endoscope. [Background technology]

[0002] In the medical field, diagnosis using medical images is widely practiced. For example, an endoscope system including a light source device, an endoscope, and a processor device is widely known as an apparatus using medical images. In the endoscope system, an endoscopic image is obtained as a medical image by irradiating an observation target with illumination light and capturing an image of the observation target illuminated by the illumination light. The endoscopic image is used for diagnosis or the like by being displayed on a display or the like.

[0003] Furthermore, recent endoscope systems analyze endoscope images to detect and highlight areas of interest such as lesion areas, thereby assisting users in making diagnoses. Patent Document 1 below describes a configuration in which candidates for an area of ​​interest are detected, a reliability indicating the likelihood that each of the detected candidates is an area of ​​interest is calculated, and the manner of highlighting each candidate for an area of ​​interest is changed according to the calculated reliability. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5802364 Summary of the Invention [Problem to be solved by the invention]

[0005] However, there is a problem that the conventional devices may not be able to perform accurate analysis. In other words, the conventional devices perform analysis under the assumption that the endoscopic images to be analyzed are appropriate. However, some endoscopic images are blurred or out of focus, making them less appropriate for analysis, and accurate analysis results cannot be obtained by analyzing such endoscopic images.

[0006] The present invention has been made in view of the above background, and has an object to provide a processor device and a method of operating a processor device that can obtain more accurate analysis results. [Means for solving the problem]

[0007] In order to achieve the above-mentioned objective, the processor device of the present invention obtains analysis results of an endoscopic image captured by an endoscope, and in the processor device, an image processing processor determines whether the endoscopic image satisfies specified criteria required for analysis, and if it is determined that the specified criteria are not satisfied, issues a warning regarding the reliability of the analysis.

[0008] The amount of blurring of the endoscopic image may be set as the predetermined criterion, and if the amount of blurring is equal to or greater than the predetermined criterion, it may be determined that the predetermined criterion is not met.

[0009] The degree of focus of the endoscopic image may be set as the predetermined criterion, and if the degree of focus is equal to or lower than the predetermined focus degree, it may be determined that the predetermined criterion is not met.

[0010] The brightness of the endoscopic image may be set as the predetermined criterion, and if the brightness is equal to or lower than the predetermined criterion, it may be determined that the predetermined criterion is not met.

[0011] As the predetermined criterion, the ratio of the halation area with the maximum luminance gradation value may be set, and if the ratio of the halation area is equal to or greater than the predetermined ratio, it may be determined that the predetermined criterion is not met.

[0012] As the predetermined criterion, the proportion of the area where the image of the adhering matter adhering to the imaging optical system is captured may be set, and if the proportion of the area where the adhering matter is adhering is equal to or greater than the predetermined proportion, it may be determined that the predetermined criterion is not met.

[0013] When a feature amount relating to biometric information is calculated by analysis, if the feature amount is equal to or smaller than a predetermined amount, it may be determined that the predetermined criterion is not satisfied.

[0014] When detecting a region of interest from an endoscopic image by analysis, if the region of interest is away from the center of the endoscopic image by a predetermined distance or more, it may be determined that the region of interest does not satisfy a predetermined criterion.

[0015] When analyzing multiple endoscopic images capturing a common observation target, the specified criteria may consist of a primary criterion and a secondary criterion related to the proportion of endoscopic images that satisfy the primary criterion, and if the proportion of endoscopic images that satisfy the primary criterion is equal to or lower than the secondary criterion, it may be determined that the specified criteria are not satisfied.

[0016] When performing normal imaging in which normal light is irradiated onto the object to be observed and a normal image is captured as an endoscopic image, and special imaging in which special light having an emission spectrum different from that of normal light is irradiated onto the object to be observed and a special image is captured as an endoscopic image, the normal image and the special image may be compared to determine whether they satisfy a specified standard.

[0017] In the comparison, the analysis results of the normal image and the special image may be compared.

[0018] When calculating features related to biometric information through analysis, if the feature obtained by analysis of a normal image is equal to or greater than a predetermined standard amount and the feature obtained by analysis of a special image is less than the predetermined standard amount, it may be determined that the predetermined standard is not met.

[0019] In addition, in order to achieve the above-mentioned objective, in the processor device operation method of the present invention, which obtains analysis results of an endoscopic image captured by an endoscope, the image processing processor determines whether the endoscopic image satisfies specified criteria necessary for analysis, and if it is determined that the specified criteria are not satisfied, issues a warning regarding the reliability of the analysis. Effect of the Invention

[0020] According to the present invention, more accurate analysis results can be obtained. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is an external view of an endoscope system. [Diagram 2] 2 is a block diagram showing the functions of the endoscope system. FIG. [Diagram 3] 4 is a block diagram showing the functions of a detailed observation mode image processing unit. FIG. [Figure 4] 13 is a flowchart showing a procedure for performing a determination process and a warning process. [Diagram 5] 13 is a flowchart illustrating an example of a determination process. [Figure 6] 13 is a flowchart illustrating an example of a determination process. [Figure 7] 13 is a flowchart illustrating an example of a determination process. [Figure 8] 13 is a flowchart illustrating an example of a determination process. [Figure 9] 13 is a flowchart illustrating an example of a determination process. [Figure 10] 13 is a flowchart illustrating an example of a determination process. [Figure 11] 13 is a flowchart illustrating an example of a determination process. [Figure 12] 13 is a flowchart illustrating an example of a determination process. [Figure 13] 13 is a flowchart illustrating an example of a determination process. [Figure 14] 13 is a flowchart showing a flow of a determination process, a warning process, and an advice process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] 1, an endoscopic system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18, and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16.

[0023] The endoscope 12 has an insertion section 12a to be inserted into the body of an observation subject, an operation section 12b provided at the base end of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip side of the insertion section 12a. An angle knob 12e is provided on the operation section 12b, and by operating the angle knob 12e, the bending section 12c is bent so that the tip section 12d can be oriented in a desired direction.

[0024] The operation unit 12b is provided with a mode changeover switch 12f used to change the observation mode, a still image acquisition instruction unit 12g used to instruct acquisition of a still image of the observation target, and a zoom operation unit 12h used to operate the zoom lens 43 (see FIG. 2). The endoscope system 10 has, as observation modes, a normal observation mode and a detailed observation mode for observing more closely than in the normal observation mode, and the observation mode is switched by operating the mode changeover switch 12f.

[0025] The light source device 14 includes a light source section 20 (see FIG. 2) that emits illumination light for illuminating an observation target. The illumination light from the light source section 20 is guided by a light guide 25 (see FIG. 2) and irradiated from the tip section 12d toward the observation target. The observation target illuminated by the illumination light from the light source section 20 is imaged by an imaging sensor 44 (see FIG. 2) built into the tip section 12d.

[0026] The processor device 16 is electrically connected to a display 18 and a user interface 19. The display 18 outputs and displays an image of the observation target, information accompanying the image of the observation target, and the like. The user interface 19 is a keyboard, a mouse, and / or a touchpad, and has a function of accepting input operations such as function settings. An external recording unit (not shown) for recording images, image information, and the like may be connected to the processor device 16.

[0027] In FIG. 2, the light source device 14 includes the above-mentioned light source section 20 and a light source processor 21 that controls the light source section 20, and the emission timing and emission spectrum of the illumination light emitted from the light source section 20 are controlled by the light source processor 21.

[0028] In this embodiment, the light source device 14 emits normal light and special light having different emission spectra. The normal light is, for example, white light. The white light includes, for example, purple light in the wavelength band of 380 to 420 nm, blue light in the wavelength band of 420 to 500 nm, green light in the wavelength band of 480 to 600 nm, and red light in the wavelength band of 600 to 650 nm.

[0029] The special light, for example, emits a greater amount of violet light in the wavelength band of 380 nm to 420 nm, which has a high absorption coefficient for hemoglobin in blood vessels, than normal light. As a result, in a special image (an endoscopic image captured by irradiating the special light on an observation target) described below, the resolution of blood vessel structures and glandular duct structures is higher than that of other structures. Note that the type of special light is not limited to one, and a configuration may be adopted in which multiple types of special images are captured using multiple types of special light.

[0030] Illumination light from the light source unit 20 is incident on the above-mentioned light guide 25 via an optical path coupling unit 23 composed of a mirror, a lens, etc. The light guide 25 is built into the endoscope 12 and a universal cord (a cord that connects the endoscope 12 with the light source device 14 and the processor device 16). The light guide 25 propagates the light from the optical path coupling unit 23 to the tip 12d of the endoscope 12.

[0031] An illumination optical system 30a and an imaging optical system 30b are provided at the tip 12d of the endoscope 12. The illumination optical system 30a has an illumination lens 32, and illumination light propagated by the light guide 25 is irradiated onto the observation object via the illumination lens 32. The imaging optical system 30b has an objective lens 42, a zoom lens 43, and an imaging sensor 44. Light from the observation object irradiated with illumination light is incident on the imaging sensor 44 via the objective lens 42 and the zoom lens 43. As a result, an image of the observation object is formed on the imaging sensor 44. The zoom lens 43 is a lens for enlarging the observation object, and is moved between the telephoto end and the wide end by operating the zoom operation unit 12h.

[0032] The imaging sensor 44 is a color sensor, and in this embodiment, a primary color sensor having three types of pixels, namely, a B pixel having a B (blue) color filter, a G pixel having a G (green) color filter, and an R pixel having an R (red) color filter, is used. As such imaging sensor 44, a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) imaging sensor or the like can be used. The imaging sensor 44 is controlled by an imaging processor 45. Specifically, the imaging processor 45 reads out the signal of the imaging sensor 44, and an image signal is output from the imaging sensor 44.

[0033] The imaging processor 45 also controls the light source processor 21 described above to perform normal imaging, which irradiates the observation target with normal light to capture an image (signal readout), and special imaging, which irradiates the observation target with special light to capture an image. In normal imaging, a normal image is captured as an endoscopic image, and in special imaging, a special image is captured as an endoscopic image. The imaging cycle between normal imaging and special imaging can be freely set, but in this embodiment, normal imaging is performed five times (five frames) and special imaging is performed once (one frame) repeatedly. In this embodiment, imaging is performed 60 times (60 frames) per second. As a result, in this embodiment, 50 normal images (50 frames) and 10 special images (10 frames) are acquired per second.

[0034] Instead of the primary color imaging sensor 44 having RGB primary color filters, a complementary color imaging sensor having C (cyan), M (magenta), Y (yellow) and G (green) complementary color filters may be used. When a complementary color imaging sensor is used, image signals of four colors, CMYG, are output. Therefore, image signals of each of the RGB colors similar to those of the imaging sensor 38 can be obtained by converting the four color image signals of CMYG into image signals of three colors, RGB, by complementary color-primary color conversion. Also, instead of the imaging sensor 38, a monochrome sensor not having color filters may be used.

[0035] A CDS / AGC (Correlated Double Sampling / Automatic Gain Control) circuit 46 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the image sensor 44. The image signal passed through the CDS / AGC circuit 46 is converted into a digital image signal by an A / D (Analog / Digital) converter 48. The digital image signal after A / D conversion is input to the processor device 16.

[0036] The processor device 16 includes a central control unit 68 that constitutes the image processing processor of the present invention. The central control unit 68 is a hardware resource for executing program instructions stored in a memory 69, and drives and controls each unit of the processor device 16 to execute the program instructions. Due to the drive control of the central control unit 68 accompanying the execution of the program instructions, the processor device 16 functions as an image acquisition unit 50, a DSP (Digital Signal Processor) 52, a noise reduction unit 54, an image processing unit 58, and a display control unit 60.

[0037] The image acquisition unit 50 acquires an endoscopic image input from the endoscope 12. In this embodiment, the endoscopic image is a color image composed of a blue signal (B image signal), a green signal (G image signal), and a red signal (R image signal) output from the B pixel, G pixel, and R pixel of the imaging sensor 44. In addition, in this embodiment, normal imaging is performed by irradiating the observation target with normal light and special imaging is performed by irradiating the observation target with special light. As a result, in this embodiment, a normal image by normal imaging and a special image by special imaging are acquired as an endoscopic image, which is a color image. Furthermore, in this embodiment, five normal imaging sessions and one special imaging session are repeatedly performed. As a result, five normal images (frames) and one special image (frame) are repeatedly acquired.

[0038] The endoscopic image acquired by the image acquisition unit 50 is transmitted to the DSP 52. The DSP 52 performs various signal processing such as defect correction processing, offset processing, gain correction processing, matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing on the received endoscopic image.

[0039] In the defect correction process, signals from defective pixels of the imaging sensor 44 are corrected. In the offset process, dark current components are removed from the image signal that has been subjected to the defect correction process, and an accurate zero level is set. In the gain correction process, the signal level of the endoscopic image is adjusted by multiplying the image signal of each color after the offset process by a specific gain coefficient. Note that, when a monochrome sensor is used as the imaging sensor 44, it is preferable that the endoscopic image is captured by the monochrome sensor for each emission of light of a specific color, and a monochrome image of multiple colors is output from the monochrome sensor.

[0040] The image signals of each color after the gain correction process are subjected to matrix processing to improve color reproducibility. Then, the brightness and saturation of the endoscopic image are adjusted by gamma conversion. The endoscopic image after the matrix process is subjected to demosaic processing (also called isotropic processing or synchronization processing), and signals of the missing colors of each pixel are generated by interpolation. The demosaic processing ensures that all pixels have signals of each of the RGB colors. The DSP 52 performs YC conversion processing on the endoscopic image after the demosaic processing, and outputs the luminance signal Y and the color difference signals Cb and Cr to the noise reduction unit 54.

[0041] The noise reduction unit 54 performs noise reduction processing, for example, by a moving average method or a median filter method, on the endoscopic image that has been subjected to demosaic processing and the like by the DSP 56. The endoscopic image from which noise has been reduced is input to an image processing unit 58. Then, after various image processing is performed in the image processing unit 58, the image is input to a display control unit 60. The display control unit 60 controls the display of the display 18, and the endoscopic image that has been subjected to various image processing in the image processing unit 58 is converted into a video signal for display in the display control unit 60 and displayed on the display 18.

[0042] The image processing unit 58 includes a normal observation mode image processing unit 62 and a detailed observation mode image processing unit 64. The normal observation mode image processing unit 62 operates when the observation mode is the normal observation mode. On the other hand, the detailed observation mode image processing unit 64 operates when the observation mode is the detailed observation mode.

[0043] The normal observation mode image processing unit 62 performs color conversion processing, color enhancement processing, and structure enhancement processing on the endoscopic image input from the noise reduction unit 54. In the color conversion processing, color conversion processing is performed on the endoscopic image by 3×3 matrix processing, gradation conversion processing, 3D LUT (Look Up Table) processing, etc. Color enhancement processing is performed on the endoscopic image that has already been color converted. Structure enhancement processing is processing that emphasizes the structure of the observation target, and is performed on the endoscopic image after color enhancement processing. The endoscopic image that has been subjected to the above-mentioned image processing is then displayed on the display 18 via the display control unit 60.

[0044] The detailed observation mode image processing unit 64 also performs image processing similar to that performed by the normal observation mode processing unit, and the processed endoscopic image is displayed on the display 18. As shown in FIG. 3, the detailed observation mode image processing unit 64 is provided with a recognition processing unit 70, an image determination unit 72, and a warning processing unit 74. The recognition processing unit 70 analyzes the endoscopic image input from the noise reduction unit 54 and performs recognition processing. The recognition processing performed by the recognition processing unit 70 includes a detection processing for detecting an attention area from the recognition image (in this embodiment, the endoscopic image) and a discrimination processing for discriminating the type of lesion contained in the recognition image. The discrimination processing includes a processing performed on the attention area and a processing performed on the entire recognition image, and in this embodiment, a description will be given of an example in which discrimination processing is performed on the attention area.

[0045] In the recognition process, for example, an endoscopic image is divided into a plurality of small regions, and image features are calculated from the divided endoscopic image. In the detection process, it is determined whether each small region is a lesion based on the calculated features, a group of regions identified as the same type is extracted as one lesion, and a region including the extracted lesion is detected as a region of interest. In the discrimination process, the type and / or degree (stage) of the lesion is determined for the detected region of interest based on the features in the region of interest and the state (position, size, shape, etc.) of the region of interest. The judgment in the above-mentioned recognition process (detection process, discrimination process) is performed by machine learning algorithms such as a convolutional neural network and deep learning.

[0046] The feature amount calculated from the endoscopic image by the recognition processing unit 70 is preferably related to biological information and is the shape or color of a predetermined part of the observation target, or an index value obtained from the shape, color, etc. The feature amount is preferably at least any one of blood vessel density, blood vessel shape, number of blood vessel branches, blood vessel thickness, blood vessel length, blood vessel tortuosity, blood vessel depth, duct shape, duct opening shape, duct length, duct tortuosity, and color information, or a combination of two or more of them.

[0047] Then, the result of the above-mentioned recognition processing, that is, the analysis result obtained by the recognition processing unit 70 analyzing the endoscopic image, is displayed on the display 18 via the display control unit 60. Note that, in the present embodiment, an example in which the recognition processing unit 70 is provided in the endoscopic system 10 (more specifically, the processor device 16) has been described, but the present invention is not limited to this. The recognition processing unit may be provided in an external device other than the processor device 16 or the endoscopic system 10. In this case, an endoscopic image to be analyzed (subject of the recognition processing) may be transmitted to the external device, the recognition processing (analysis) may be performed in the external device, and the result of the recognition processing (analysis result) transmitted from the external device may be received (acquired) by the endoscopic system 10 (more specifically, the processor device 16).

[0048] 4, an endoscopic image that has been subjected to recognition processing by the recognition processing unit 70 is input to the image determination unit 72. The image determination unit 72 analyzes the input endoscopic image and performs a determination process to determine whether or not the input endoscopic image satisfies a predetermined standard required for the recognition processing (analysis performed by the recognition processing unit 70). Note that the determination process may be performed on the endoscopic image before the recognition processing by the recognition processing unit 70. In addition, the configuration is not limited to one in which the recognition processing and the determination processing are performed in series, and the same endoscopic image may be input to each of the recognition processing unit 70 and the image determination unit 72, and the recognition processing and the determination processing may be performed in parallel.

[0049] The warning processing unit 74 is activated when it is determined by the image determination unit 72 that the image does not satisfy the predetermined criteria required for the recognition processing, and performs warning processing. The warning processing is to warn that the reliability of the recognition processing has decreased due to the endoscopic image not satisfying the predetermined criteria required for the recognition processing. The warning is performed, for example, by displaying a message on the display 18 to the effect that the reliability of the recognition processing has decreased, and / or by outputting a warning sound from a speaker (not shown).

[0050] For example, the amount of image blur is set as the predetermined criterion used in the determination process. When the amount of image blur is set as the predetermined criterion, as shown in Fig. 5, the image determination unit 72 analyzes the endoscopic image to be determined and calculates an index value relating to the amount of image blur (the larger the value, the larger the amount of image blur). Then, the calculated index value is compared with a preset threshold value for image blur determination, and if it is equal to or greater than the threshold value, i.e., if the amount of image blur is equal to or greater than a predetermined amount of blur, it is determined that the predetermined criterion is not met. On the other hand, if the calculated index value is less than the threshold value for image blur determination, it is determined that the predetermined criterion is met.

[0051] The predetermined criterion is not limited to the amount of image blur. The predetermined criterion may be the degree of focus, image brightness, halation, attached matter, feature amount, or attention area position. When the degree of focus is set as the predetermined criterion, as shown in FIG. 6, the image determination unit 72 analyzes the endoscopic image to be determined and calculates an index value related to the degree of focus (the larger the value, the higher the degree of focus (the better the focus)). Then, the calculated index value is compared with a preset threshold value for determining the degree of focus, and if the calculated index value is equal to or less than the threshold value, that is, if the degree of focus is equal to or less than the predetermined degree of focus, it is determined that the predetermined criterion is not satisfied. On the other hand, if the calculated index value is greater than the threshold value for determining the degree of focus, it is determined that the predetermined criterion is satisfied.

[0052] When image luminance is set as the predetermined criterion, as shown in Fig. 7, the image determination unit 72 analyzes the endoscopic image to be determined and detects image luminance (for example, the average luminance gradation value of each pixel of the entire image or the region of interest). If the detected image luminance is equal to or less than a preset threshold value for luminance determination, that is, if the image luminance is equal to or less than the predetermined luminance, it is determined that the predetermined criterion is not satisfied. On the other hand, if the image luminance is greater than the threshold value for luminance determination, it is determined that the predetermined criterion is satisfied.

[0053] Furthermore, when halation is set as the predetermined criterion, as shown in Fig. 8, the image determination unit 72 analyzes the endoscopic image to be determined and detects a halation area in the entire image or in the region of interest. The halation area is a region where the luminance gradation value of the image is maximum, that is, a region where the image is blown out. Then, when the ratio of the halation area is equal to or greater than a preset threshold value for determining halation, that is, when the ratio of the halation area is equal to or greater than a predetermined ratio, the image determination unit 72 determines that the predetermined criterion is not satisfied. On the other hand, when the ratio of the halation area is less than the threshold value for determining halation, the image determination unit 72 determines that the predetermined criterion is satisfied.

[0054] Furthermore, when an attachment is set as the predetermined criterion, as shown in Fig. 9, the image determination unit 72 analyzes the endoscopic image to be determined and detects an attachment region in the entire image or in the region of interest. The attachment region is a region in which an image of an attachment such as a body fluid attached to an imaging optical system such as the objective lens 42 is captured. Then, when the proportion of the attachment region is equal to or greater than a preset threshold value for determining attachment, that is, when the proportion of the attachment region is equal to or greater than a predetermined proportion, the image determination unit 72 determines that the predetermined criterion is not satisfied. On the other hand, when the proportion of the attachment region is less than the threshold value for determining attachment, the image determination unit 72 determines that the predetermined criterion is satisfied.

[0055] Furthermore, when a feature amount is set as the predetermined criterion, as shown in Fig. 10, the image determination unit 72 determines that the predetermined criterion is not satisfied if the feature amount calculated in the recognition process is equal to or less than a preset threshold value for determining the feature amount, i.e., if the feature amount is equal to or less than the predetermined amount, on the other hand, the image determination unit 72 determines that the predetermined criterion is satisfied if the feature amount exceeds the threshold value for determining the feature amount.

[0056] Furthermore, when the attention area position is set as the predetermined criterion, as shown in Fig. 11, the image determination unit 72 detects the position (center position) of the attention area detected in the recognition process, and calculates the distance between the detected position and the center of the endoscopic image. If the calculated distance is equal to or greater than a preset threshold for determining the attention area position, that is, if the attention area is away from the center of the endoscopic image by a predetermined distance or more, it is determined that the predetermined criterion is not met. On the other hand, if the calculated distance is less than the threshold for determining the attention area position, it is determined that the predetermined criterion is met.

[0057] In the above embodiment, the example is described in which it is determined that the predetermined criterion is satisfied when one preset criterion is satisfied, but the present invention is not limited to this. It may be determined that the predetermined criterion is satisfied when all of a plurality of preset criteria are satisfied, or when a predetermined ratio or more of the plurality of preset criteria are satisfied.

[0058] 12, it may be determined that a predetermined criterion is met when the ratio of endoscopic images that satisfy the primary criterion exceeds the secondary criterion among a plurality of endoscopic images captured continuously (the predetermined criterion is not met when the ratio of endoscopic images that satisfy the primary criterion is equal to or less than the secondary criterion). Here, as the primary criterion, one or more of the above-mentioned image blur amount, focus degree, image brightness, halation, attachment, feature amount, and attention area position are set, and it is determined that the primary criterion is met when all of these are satisfied or a predetermined ratio or more is satisfied. Then, it is determined that the predetermined criterion is met when the ratio of endoscopic images that satisfy the primary criterion exceeds the secondary criterion (e.g., 10%) among the continuously captured endoscopic images.

[0059] It is preferable that the multiple endoscopic images captured consecutively are images of a common observation target. Whether or not the endoscopic images are images of a common observation target can be determined by analyzing the endoscopic images. Also, the endoscopic images captured during one detailed observation mode (from when the detailed observation mode is switched to until the detailed observation mode is ended) may be considered as images of a common observation target. Furthermore, for the multiple endoscopic images captured consecutively, the user may specify the imaging start time and imaging end time of the common observation target, and the endoscopic images captured during this period may be set as images of the common observation target.

[0060] In the above embodiment, the predetermined criteria are common to both normal and special images, but the predetermined criteria may be different depending on whether the endoscopic image to be judged is a normal or special image. In addition, the above embodiment is an example in which it is judged whether the predetermined criteria are satisfied for both normal and special images, but it may be a configuration in which it is judged whether the predetermined criteria are satisfied for only one of normal and special images.

[0061] Similarly, in the above embodiment, the contents of the recognition process are the same for normal images and special images, but the contents of the recognition process may be different for normal images and special images. Also, the recognition process may be performed only on one of the normal image and the special image.

[0062] Furthermore, whether or not a predetermined criterion is satisfied may be determined by comparing the normal image with the special image. In this case, for example, by comparing the feature amounts of a normal image and a special image captured of a common observation target, if the difference between the two is equal to or greater than a predetermined value, it may be determined that the predetermined criterion is not satisfied.

[0063] In addition, the feature amount of the normal image and the special image may be compared, and when one is equal to or greater than the reference amount but the other is less than the reference amount, it may be determined that the predetermined criterion is not met. For example, in the above embodiment, the resolution of the vascular structure and the duct structure of the special image is higher than that of the normal image, so when the feature amount related to the vascular structure and the duct structure is calculated in the recognition process, it is natural that the feature amount calculated from the special image has a larger amount of information than the feature amount calculated from the normal image. Nevertheless, when the feature amount calculated from the normal image is equal to or greater than the reference amount and the feature amount calculated from the special image is less than the reference amount, for example, when the attention area can be detected in the recognition process from the information image but cannot be detected in the special image, it is highly likely that the endoscopic image to be recognized is not captured under appropriate conditions. For this reason, in the example of FIG. 13, when the attention area is detected in the information image but not detected in the special image, it is determined that the predetermined criterion is not met.

[0064] In the above embodiment, an example is described in which a warning is issued when it is determined that an endoscopic image does not satisfy the predetermined criteria required for the recognition process (analysis), but the present invention is not limited to this. As shown in FIG. 14, when it is determined that an endoscopic image does not satisfy the predetermined criteria required for the recognition process (analysis), advice processing may be performed to advise an imaging method so that an endoscopic image that satisfies the predetermined criteria required for the recognition process is captured. The advice processing is performed, for example, by the warning processing unit 74 (see FIG. 3), and in the advice processing, advice on an imaging method is given by displaying a message on the display 18, as in the above-mentioned warning processing. In the example of FIG. 14, an example is described in which advice processing is performed in addition to warning processing, but a configuration in which advice processing is performed instead of warning processing may be used.

[0065] The contents of advice in the advice process (the contents of the message for advice) are preferably determined with reference to the cause of the judgment that the predetermined criterion is not met. In this case, for example, if it is judged that the predetermined criterion is not met with respect to the amount of image blur, a message is displayed urging the user to "move the camera (the tip 12d of the insertion section 12a) slowly." If it is judged that the predetermined criterion is not met with respect to the degree of focus (if the high-frequency components of the image are below the criterion), a message is displayed urging the user to "move the camera (the tip 12d of the insertion section 12a) back and forth" to focus. If it is judged that the predetermined criterion is not met with respect to image brightness, a message is displayed urging the user to "increase the amount of illumination light" or "accurately apply illumination light to the object of observation." If it is judged that the predetermined criterion is not met with respect to halation, a message is displayed urging the user to "change the shooting angle." If it is judged that the predetermined criterion is not met with respect to adhesions, a message is displayed urging the user to "clean the objective lens (the tip 12d of the insertion section 12a)." Also, if it is determined that the feature amount or the attention area position does not meet the predetermined criteria, or if it is determined that the attention area is not detected from the special image despite being detected from the general image, a message urging the user to "capture the object of observation from the front (center of the image)" is displayed.

[0066] In addition, even if the image is re-imaged according to the advice described above, there are cases where it is determined that the predetermined criteria are not met. In such cases, it is not only determined that the predetermined criteria are not met due to an inappropriate operation of the user, such as an imaging method, but also includes cases where the predetermined criteria are not met due to a malfunction on the device side, such as an error in recognition processing. And, even in the latter case, repeated warnings and / or advice are problematic, as they may hinder diagnosis. For this reason, when warnings and / or advice are given for endoscopic images of a common observation target that do not meet the predetermined criteria, it is preferable to set an upper limit on the number of warnings and / or advice. Of course, when the endoscopic images are endoscopic images of a common observation target and the cause of the judgment that the predetermined criteria are not met is also common, an upper limit may be set on the number of warnings and / or advice.

[0067] In the above embodiment, the example in which the general observation mode and the detailed observation mode are switched by the user's operation (operation of the mode changeover switch 12f) has been described, but the general observation mode and the detailed observation mode may be automatically switched without the user's operation. In this case, for example, it is possible to perform a detection process in the general observation mode, and when a region of interest is detected, to switch to the detailed observation mode and perform a discrimination process. In addition, when it is determined in the detailed observation mode that the endoscopic image meets a predetermined standard, that is, when it is considered that an accurate analysis (recognition process) has been performed, it is possible to display the analysis result (the result of the recognition process) and return to the general observation mode.

[0068] In the above embodiment, the hardware structure of the processing units that execute various processes, such as the light source processor 21, the imaging processor 45, the image acquisition unit 50, the DPS 52, the noise reduction unit 54, the image processing unit 58, the normal observation mode image processing unit 62, the detailed observation mode image processing unit 64, the central control unit 68, the recognition processing unit 70, the image determination unit 72, and the warning processing unit 74, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing various processes.

[0069] One processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same type or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by a computer such as a client or server, one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0070] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements, and the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). [Explanation of symbols]

[0071] 10 Endoscope System 12 Endoscopy 12a Insertion part 12b Operation section 12c Curved section 12d Tip 12e Angle knob 12f Mode Switch 12g Still image acquisition instruction section 12h Zoom operation section 14 Light source device 16 Processor Unit 18 Display 19 User Interface 20 Light source section 21 Light source processor 23 Optical path coupling section 25 Light Guide 30a illumination optical system 30b Imaging optical system 32 Lighting Lens 42 Objective Lens 43 Zoom Lens 44 Image sensor 45 Imaging Processor 46 CDS / AGC circuit 48 A / D Converter 50 Image acquisition unit 52 DSP 54 Noise reduction section 58 Image Processing Section 60 Display control unit 62 Normal observation mode image processing section 64 Detailed observation mode image processing section 68 Central Control Unit 69 Memory 70 Recognition processing section 72 Image Judgment Section 74 Warning Processing Section

Claims

1. A processor device that acquires an analysis result of an endoscopic image captured by an endoscope, The image processor determining whether the endoscopic image satisfies a predetermined criterion required for the analysis; If it is determined that the predetermined criterion is not satisfied, a warning is issued regarding the reliability of the analysis; and a normal imaging process for irradiating an observation target with normal light and capturing an endoscopic image as the normal image; and a special imaging process for irradiating an observation target with at least one type of special light having an emission spectrum different from that of the normal light and capturing at least one type of special image as the endoscopic image, each of which is automatically and repeatedly performed in a predetermined imaging cycle; the normal light is white light, and the special light has a higher ratio of purple light than the normal light, the region of interest is a region including a lesion, A processor device that determines that the predetermined criterion is not met when the attention area is detected from the normal image and the attention area is not detected from the special image.

2. The amount of blur of the endoscopic image is set as the predetermined criterion, 2. The processor device according to claim 1, wherein when the amount of shaking is equal to or greater than a predetermined amount of shaking, it is determined that the predetermined criterion is not satisfied.

3. As the predetermined criterion, a focus degree of the endoscopic image is set, 3. The processor device according to claim 1, wherein when the degree of focus is equal to or lower than a predetermined degree of focus, it is determined that the predetermined criterion is not satisfied.

4. The luminance of the endoscopic image is set as the predetermined criterion, The processor device according to claim 1 , wherein when the luminance is equal to or lower than a predetermined luminance, it is determined that the predetermined criterion is not satisfied.

5. 5. The processor device according to claim 1, wherein the predetermined criterion is set to a ratio of a halation area having a maximum brightness gradation value, and when the ratio of the halation area is equal to or greater than the predetermined ratio, it is determined that the predetermined criterion is not satisfied.

6. The processor device according to claim 1 , wherein the predetermined criterion is set to a percentage of an area where an image of an object adhering to an imaging optical system is captured, and when the percentage of the area is equal to or greater than a predetermined percentage, it is determined that the predetermined criterion is not met.

7. In the case where a feature amount related to biometric information is calculated by the analysis, The processor device according to claim 1 , wherein when the feature amount is equal to or smaller than a predetermined amount, it is determined that the predetermined criterion is not satisfied.

8. When detecting a region of interest from the endoscopic image by the analysis, The processor device according to claim 1 , wherein when the region of interest is away from the center of the endoscopic image by a predetermined distance or more, it is determined that the predetermined criterion is not satisfied.

9. In the case of analyzing a plurality of endoscopic images of a common observation target, the predetermined criteria include a primary criterion and a secondary criterion related to a ratio of the endoscopic images that satisfy the primary criterion, The processor device according to claim 1 , wherein when a ratio of the endoscopic images that satisfy the primary criterion is equal to or less than the secondary criterion, it is determined that the predetermined criterion is not satisfied.

10. In the case of performing normal imaging in which a target object is irradiated with normal light and a normal image is captured as the endoscopic image, and special imaging in which a target object is irradiated with special light having an emission spectrum different from that of the normal light and a special image is captured as the endoscopic image, The processor device according to claim 1 , further comprising: a processor that compares the normal image with the special image to determine whether or not the normal image satisfies the predetermined criterion.

11. The processor device according to claim 10 , wherein the comparison comprises comparing the analysis results of the normal image and the special image.

12. In the case where a feature amount related to biometric information is calculated by the analysis, The processor device according to claim 11, further comprising: a processor that determines that the specified standard is not met if the feature amount obtained by analysis of the normal image is equal to or greater than a specified standard amount and the feature amount obtained by analysis of the special image is less than the specified standard amount.

13. A method for operating a processor device to obtain an analysis result of an endoscopic image captured by an endoscope, comprising: The image processor determining whether the endoscopic image satisfies a predetermined criterion required for the analysis; if it is determined that the predetermined criterion is not satisfied, a warning is issued regarding the reliability of the analysis. Along with a normal imaging process for irradiating an observation target with normal light and capturing an endoscopic image as the normal image; and a special imaging process for irradiating an observation target with at least one type of special light having an emission spectrum different from that of the normal light and capturing at least one type of special image as the endoscopic image, each of which is automatically and repeatedly performed in a predetermined imaging cycle; the normal light is white light, and the special light has a higher ratio of purple light than the normal light, the region of interest is a region including a lesion, A method for operating a processor device, the method including determining that the predetermined criterion is not met when the attention area is detected from the normal image and the attention area is not detected from the special image.

Images