Image processing device and its operating method

Abstract

To provide an image processing device which can store a high-quality image in which the shake is suppressed even in a case where a plurality of high luminance portions and the like exist, and provide an operation method of the same.SOLUTION: An evaluation image generation unit 61 generates a plurality of frequency images with different spatial frequencies from an endoscopic image and generates an evaluation image used for image evaluation on the basis of the plurality of frequency images. An evaluation value calculation unit 62 calculates a computed evaluation value with computation processing for evaluation value based on a specific pixel whose evaluation value for each pixel of the evaluation image is within a specific range. An image storage control unit 63 stores the specific computed evaluation value satisfying a specific condition of the computed evaluation values and the endoscopic image corresponding to the specific computed evaluation value in an image storage memory 52 in association with each other.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to an image processing device and its operating method for controlling the storage of an image of an object to be observed. [Background technology]

[0002] In an endoscopic examination, when a lesion or the like is detected, a still image acquisition command switch or the like is operated to store an image of the lesion in an image storage memory. In this case, it is required to store high-quality images with reduced blurring. In Patent Document 1, to store high-quality images, the signal differences between an evaluation image group (four pixels) and horizontally and vertically adjacent pixel groups are calculated, and the sum of all pixels based on the signal differences between these image groups is calculated as an evaluation value. In Patent Document 1, images with high evaluation values ​​are then selected for storage. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5043595 Summary of the Invention [Problem to be solved by the invention]

[0004] When using the signal difference between adjacent pixels, as in the method of Patent Document 1, the evaluation value tends to be high if there are many areas with clear contrast between light and dark. Therefore, if the image to be saved contains multiple high-brightness areas (a glaring image) even though blurring and other issues have not been suppressed, the evaluation value may be high and the image may be selected for saving.

[0005] An object of the present invention is to provide an image processing device and an operating method thereof that can store high-quality images with reduced blur even when multiple high-brightness areas exist. [Means for solving the problem]

[0006] The image processing device of the present invention includes a processor, which acquires an endoscopic image, generates a plurality of frequency images with different spatial frequencies from the endoscopic image, generates an evaluation image to be used for image evaluation based on the plurality of frequency images, calculates a calculated evaluation value by evaluation value calculation processing based on specific pixels whose pixel-by-pixel evaluation values ​​are within a specific range in the evaluation image, and performs image storage control to associate a specific calculated evaluation value that satisfies a specific condition among the calculated evaluation values ​​with an endoscopic image corresponding to the specific calculated evaluation value and store them in an image storage memory, or, when the maximum calculated evaluation value among the calculated evaluation values ​​is calculated, associates the maximum calculated evaluation value with the endoscopic image corresponding to the maximum calculated evaluation value and store them in an image storage memory.

[0007] It is preferable that the processor generates at least a high-frequency image and a low-frequency image having a lower spatial frequency than the high-frequency image, and that the processor generates an evaluation edge image from at least one of the high-frequency image or the low-frequency image as the evaluation image.

[0008] The processor acquires, as the endoscopic image, at least one of a first illumination light image obtained by imaging an object to be observed illuminated with a first illumination light and a second illumination light image obtained by imaging an object to be observed illuminated with a second illumination light in which the intensity of the illumination light at a short wavelength is greater than the emission spectrum of the first illumination light, and when the processor generates an evaluation edge image as an evaluation image by a first weighting calculation based on at least a first high-frequency edge image and a first low-frequency edge image generated from the first illumination light image, it is preferable to perform the first weighting calculation by making the low-frequency weighting coefficient for the first low-frequency edge image larger than the high-frequency weighting coefficient for the first high-frequency edge image.

[0009] The processor acquires, as the endoscopic image, at least one of a first illumination light image obtained by imaging an object to be observed illuminated with a first illumination light and a second illumination light image obtained by imaging an object to be observed illuminated with a second illumination light in which the intensity of the short-wavelength illumination light is greater than the emission spectrum of the first illumination light, and when the processor generates an evaluation edge image as an evaluation image by a second weighting calculation based on at least a second high-frequency edge image and a second low-frequency edge image generated from the second illumination light image, it is preferable to perform the second weighting calculation by making the high-frequency weighting coefficient for the second high-frequency edge image larger than the low-frequency weighting coefficient for the second low-frequency edge image.

[0010] The pixel evaluation value is preferably a pixel value of the edge image, and the calculated evaluation value is preferably obtained by an evaluation value calculation process based on specific pixels whose pixel values ​​are in a specific range of the edge image. The evaluation value calculation process preferably calculates at least one of the sum of the number of specific pixels, the maximum edge length among the edge lengths formed by the specific pixels, or the number of horizontal lines, vertical lines, or diagonal lines based on the specific pixels.

[0011] The processor acquires, according to a specific acquisition cycle, a group of endoscopic images including a first illumination light image obtained by imaging an object to be observed illuminated with a first illumination light and a second illumination light image obtained by imaging an object to be observed illuminated with a second illumination light in which the intensity of the illumination light at a short wavelength is higher than the emission spectrum of the first illumination light, and the processor calculates a calculated evaluation value based on only one of the first illumination light image or the second illumination light image, and preferably stores, in the image storage memory, an endoscopic image included in the group of endoscopic images corresponding to an evaluation image whose calculated evaluation value satisfies a specific condition, in association with the calculated evaluation value.

[0012] The processor acquires, according to a specific acquisition cycle, a group of endoscopic images including a first illumination light image obtained by imaging an object to be observed illuminated with a first illumination light and a second illumination light image obtained by imaging an object to be observed illuminated with a second illumination light in which the intensity of the illumination light at a short wavelength is higher than the emission spectrum of the first illumination light, and the processor calculates a calculated evaluation value based on each of the first illumination light image or the second illumination light image, and preferably stores, in the image storage memory, the first illumination light image corresponding to an evaluation image whose calculated evaluation value satisfies a specific condition and the second illumination light image corresponding to an evaluation image whose calculated evaluation value satisfies the specific condition, in association with the calculated evaluation value.

[0013] The image saving control is preferably performed during a still image acquisition period when a still image acquisition instruction is issued.

[0014] The method of operating the image processing device of the present invention includes the steps of: a processor acquiring an endoscopic image; generating a plurality of frequency images with different spatial frequencies from the endoscopic image; and generating an evaluation image to be used for image evaluation based on the plurality of frequency images; a processor calculating a calculated evaluation value by evaluation value calculation processing based on specific pixels whose pixel-by-pixel evaluation values ​​are within a specific range in the evaluation image; and a processor performing image storage control to associate a specific calculated evaluation value that satisfies a specific condition among the calculated evaluation values ​​with an endoscopic image corresponding to the specific calculated evaluation value and store them in the image storage memory; or, when the maximum calculated evaluation value among the calculated evaluation values ​​is calculated, to associate the maximum calculated evaluation value with the endoscopic image corresponding to the maximum calculated evaluation value and store them in the image storage memory. [Effects of the Invention]

[0015] According to the present invention, even when there are multiple high-brightness areas, it is possible to store a high-quality image with reduced blur. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is an explanatory diagram of a configuration of an endoscope system. [Figure 2] FIG. 2 is a block diagram showing the functions of the endoscope system. [Figure 3] FIG. 4 is an explanatory diagram showing a first light emission pattern. [Figure 4] FIG. 10 is an explanatory diagram showing a second light emission pattern. [Figure 5] FIG. 10 is an explanatory diagram showing a third light emission pattern. [Figure 6] FIG. 10 is an explanatory diagram showing a fourth light emission pattern. [Figure 7] FIG. 2 is a block diagram showing the functions of the processor device 14. [Figure 8] FIG. 10 is an explanatory diagram showing a multi-resolution conversion process. [Figure 9] FIG. 10 is an explanatory diagram illustrating a multi-resolution inverse conversion process. [Figure 10] FIG. 10 is an explanatory diagram showing a method for generating an edge image for evaluation. [Figure 11] FIG. 1 is a schematic diagram showing an image sensor with a Bayer array. [Figure 12] 10 is a schematic diagram showing pixel values ​​included in each pixel of an evaluation edge image. FIG. [Figure 13] FIG. 10 is an explanatory diagram showing an example of image storage control. [Figure 14] FIG. 14 is an explanatory diagram showing an example of image storage control different from that shown in FIG. 13. [Figure 15] FIG. 10 is an explanatory diagram showing the flow of image acquisition in a multi-flash mode. [Figure 16] 10A and 10B are explanatory diagrams showing an example of image saving control in a multi-flash mode. [Figure 17] 17 is an explanatory diagram showing an example of image saving control different from that shown in FIG. 16 in the multi-flash mode. [Figure 18] 10 is a flowchart showing a series of steps in image saving control. DETAILED DESCRIPTION OF THE INVENTION

[0017] 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14.

[0018] The endoscope 12 has an insertion section 12a, an operating section 12b, a bending section 12c, and a tip section 12d. The insertion section 12a is inserted into the body of the subject. The operating section 12b is provided at the base end of the insertion section 12a. The bending section 12c and the tip section 12d are provided on the tip side of the insertion section 12a. The bending section 12c is bent by operating an angle knob 12e of the operating section 12b. The tip section 12d is directed in a desired direction by the bending of the bending section 12c. A forceps channel (not shown) is provided from the insertion section 12a to the tip section 12d, through which a treatment tool or the like can be inserted. The treatment tool is inserted into the forceps channel from the forceps port 12j.

[0019] The endoscope 12 is provided inside with an optical system for forming an image of the subject and an optical system for irradiating the subject with illumination light. The operation unit 12b is provided with an angle knob 12e, an observation mode selector switch 12f, a still image acquisition instruction switch 12h, and a zoom operation unit 12i. The observation mode selector switch 12f is used to switch the observation mode. The still image acquisition instruction switch 12h is used to instruct acquisition of a still image of the subject. The zoom operation unit 12i is used to operate the zoom lens 42 (see FIG. 2).

[0020] The light source device 13 generates illumination light. The display 15 displays endoscopic images. The user interface 16 has a keyboard, mouse, microphone, tablet, touch pen, etc., and accepts input operations such as function settings. The processor device 14 performs system control of the endoscopic system 10 and also performs image processing on image signals transmitted from the endoscope 12.

[0021] 2, the light source device 13 includes a light source unit 20, a light source processor 21 that controls the light source unit 20, and an optical path coupling unit 22. The light source unit 20 has multiple semiconductor light sources, each of which is turned on or off. When the multiple semiconductor light sources are turned on, the light emission amount of each semiconductor light source is controlled to emit illumination light that illuminates the subject. The light source unit 20 has four color LEDs: a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d. The light source unit 20 or the light source processor 21 may be built into the endoscope 12. The light source processor 21 may be built into the processor device 14.

[0022] The light emitted by each of the LEDs 20a to 20d is incident on the light guide 23 via an optical path combining unit 22 configured with a mirror, a lens, etc. The light guide 23 propagates the light from the optical path combining unit 22 to the tip 12d of the endoscope 12.

[0023] An illumination optical system 30 and an imaging optical system 32 are provided at the tip 12d of the endoscope 12. The illumination optical system 30 has an illumination lens 31, and illumination light propagated by the light guide 23 is irradiated onto the subject via the illumination lens 31. On the other hand, when the light source unit 20 is built into the tip 12d of the endoscope 12, light is emitted toward the subject via the illumination lens of the illumination optical system without passing through a light guide.

[0024] The imaging optical system 32 has an objective lens 35, a zoom lens 36, and an imaging sensor 37. Light from the subject irradiated with illumination light is incident on the imaging sensor 37 via the objective lens 35 and the zoom lens 36. As a result, an image of the subject is formed on the imaging sensor 37. The zoom lens 36 is a lens for enlarging the subject, and is moved between the telephoto end and the wide-angle end by operating the zoom operation unit 12i.

[0025] The imaging sensor 43 is a primary color sensor and includes three types of pixels: B pixels (blue pixels) with blue color filters, G pixels (green pixels) with green color filters, and R pixels (red pixels) with red color filters.

[0026] Furthermore, the imaging sensor 43 is preferably a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The imaging processor 44 controls the imaging sensor 43. Specifically, the imaging processor 44 reads out a signal from the imaging sensor 43, thereby outputting an image signal from the imaging sensor 43. The output image signal is transmitted to the processor unit 14.

[0027] The processor device 14 corresponds to the image processing device of the present invention and is provided with a central control unit 50 consisting of a processor that is an image control processor such as a CPU (Central Processing Unit), and a program memory 51 that stores programs for executing various processes. It is also provided with an image storage memory 52 that stores still images and the like obtained when the still image acquisition instruction switch 12h is operated. The processor device 14 performs various types of image processing based on image signals from the endoscope 12. Images that have undergone various types of image processing are displayed on the display 15. The processor device 14 has a function to accept operations on the observation mode selector switch 12f and the still image acquisition instruction switch 12h, as well as a function to accept operations on the user interface 16.

[0028] The endoscope system 10 has a mono-emission mode and a multi-emission mode, and the modes can be switched via the central controller 50 by operating the observation mode selector switch 12f. The mono-emission mode is a mode in which illumination light of the same spectrum is continuously irradiated to illuminate the object to be observed. The multi-emission mode is a mode in which multiple illumination lights of different spectrums are irradiated while being switched according to a specific pattern to illuminate the object. The illumination light includes a first illumination light and a second illumination light having a spectrum different from that of the first illumination light. The first illumination light is normal light (broadband light such as white light) used to provide brightness to the entire object for screening observation. The second illumination light is multiple types of special light used to highlight specific structures, such as glandular ducts and blood vessels in the mucous membrane of the object. In the mono-emission mode, it is preferable that the first illumination light and the second illumination light can be switched.

[0029] The light source processor 21 controls the light intensity of the four colors of violet light V, blue light B, green light G, and red light R independently, and emits the first illumination light and the second illumination light while changing the light intensity.

[0030] In the mono illumination mode, illumination light of the same spectrum is continuously emitted for each frame. For example, by illuminating a subject with the first illumination light for each frame and capturing an image, a first illumination light image with natural coloring is displayed on the display 15. The first illumination light is preferably broadband light such as white light including four colors: purple light V, blue light B, green light G, and red light R. Furthermore, by illuminating a subject with the second illumination light for each frame and capturing an image, a second illumination light image that highlights a specific structure is displayed on the display 15. The second illumination light is preferably special light with a high intensity of short-wavelength light such as purple light, and the intensity of short-wavelength illumination light (e.g., purple light V) is preferably higher than the emission spectrum of the first illumination light. Note that a frame refers to a unit of time that includes at least the period from the timing of light emission to the completion of image signal readout by the image sensor 43.

[0031] In the multi-flash mode, control is performed to change the light intensities of the purple light V, blue light B, green light G, and red light R for each specific frame F according to a specific light emission pattern. In this embodiment, in the multi-flash mode, only the first illumination light image MPx is displayed on the display 15, and the second illumination light image MPy is not displayed on the display 15 (see FIGS. 16 and 17).

[0032] Examples of light emission patterns in the multi-flash mode are listed below. For example, in the first light emission pattern, as shown in FIG. 3, two frames of first illumination light L1 (L1SK) are emitted during a first illumination period Pe1 in which the subject is illuminated with the first illumination light L1. After the first illumination period Pe1, one frame of second illumination light L2 (L2SP) is emitted sequentially during a second illumination period Pe2 in which the subject is illuminated with the second illumination light L2. This pattern is repeated. In the figure, the arrow indicates the direction of time progression. In the first light emission pattern, the emission spectrum of the first illumination light L1 (L1SK) emitted during each first illumination period Pe1 is the same, and the emission spectrum of the second illumination light L2 (L2SP) emitted during each second illumination period Pe2 is also the same.

[0033] 4, in the second light emission pattern, the first illumination light L1 (L1SK) is emitted for two frames during the first illumination period Pe1, and then the second illumination light L2 (L2SP) is emitted for one frame during the second illumination period Pe2 after the first illumination period Pe1. Next, the first illumination light L1 (L1SK) is emitted for two frames during the first illumination period Pe1, and then the second illumination light L2 (L2SQ) is emitted for one frame during the second illumination period Pe1 after the first illumination period Pe1. In the second light emission pattern, the first illumination light L1 (L1SK) emitted during each first illumination period Pe1 has the same emission spectrum, while the second illumination light L2 (L2SP) and the second illumination light L2 (L2SQ), which have different emission spectra, are alternately emitted during the second illumination period Pe2.

[0034] 5, in the third light emission pattern, the first illumination light L1 (L1SK) is emitted for two frames in the first illumination period Pe1, and after the first illumination period Pe1, the second illumination light L2 (L2SP) is emitted for one frame in the second illumination period Pe2. Next, the first illumination light L1 (L1SL) is emitted for two frames in the first illumination period Pe1, and after the first illumination period Pe1, the second illumination light L2 (L2SP) is emitted for one frame in the second illumination period Pe2. e In the third emission pattern, the second illumination light L2 (L2SP) is emitted for one frame in the first illumination period Pe1. In the third emission pattern, the first illumination light L1 (L1SK) and the first illumination light L1 (L2SL), each having a different emission spectrum, are emitted alternately in the first illumination period Pe1, while the emission spectrum of the second illumination light L2 (L2SP) emitted in each second illumination period Pe2 is the same.

[0035] 6, in the fourth light emission pattern, the first illumination light L1 (L1SK) is emitted for two frames in the first illumination period Pe1, and after the first illumination period Pe1, the second illumination light L2 (L2SP) is emitted for one frame in the second illumination period Pe2. Next, the first illumination light L1 (L1SL) is emitted for two frames in the first illumination period Pe1, and after the first illumination period Pe1, the second illumination light L2 (L2SP) is emitted for one frame in the second illumination period Pe2. e In the fourth emission pattern, the second illumination light L2 (L2SQ) is emitted for one frame in the first illumination period Pe1. In the fourth emission pattern, the first illumination light L1 (L1SK) and the first illumination light L1 (L1SL), each having a different emission spectrum, are emitted alternately in the first illumination period Pe1. In the second illumination period Pe2, the second illumination light L2 (L2SP) and the second illumination light L2 (L2SQ), each having a different emission spectrum, are emitted alternately.

[0036] The processor device 14 performs various types of processing or control based on the image signal from the endoscope 12. In this embodiment, one of the various types of processing or control is image storage control for storing blur-free endoscopic images. In the processor device 14, the central control unit 50 executes an image storage control program stored in the program memory 51, thereby realizing the functions of an image acquisition unit 60, an evaluation image generation unit 61, an evaluation value calculation unit 62, and an image storage control unit 63, as shown in FIG. 7.

[0037] The image acquisition unit 60 acquires an endoscopic image as an image signal from the endoscope 12. The evaluation image generation unit 61 generates a plurality of frequency images with different spatial frequencies from the endoscopic image, and generates an evaluation image to be used for image evaluation based on the plurality of frequency images. Specifically, as shown in FIG. 8, the evaluation image generation unit 61 performs a multi-resolution conversion process on the endoscopic image MP to generate at least a high-frequency image HFP and a low-frequency image LFP having a lower spatial frequency than the high-frequency image. The size of the low-frequency image LFP is half the size of the high-frequency image HFP. The low-frequency image LFP is subjected to a multi-resolution conversion process to generate a frequency image with an even lower spatial frequency (half the size of the low-frequency image LFP). You can get ) Note that the multi-resolution conversion process includes, for example, wavelet conversion.

[0038] The evaluation image generation unit 61 generates an evaluation edge image from at least one of a high-frequency image or a low-frequency image as an evaluation image. Specifically, the evaluation edge image is generated by performing a multi-resolution inverse transform process on at least one of the high-frequency image or the low-frequency image. For example, as shown in FIG. 9, the evaluation image generation unit 61 generates a high-frequency edge image HEP by performing a multi-resolution inverse transform process on the high-frequency image HFP, and generates a low-frequency edge image HEP by performing a multi-resolution inverse transform process on the low-frequency image LFP. The size of the low-frequency edge image LEP is the same as the size of the high-frequency edge image HEP. Note that the multi-resolution inverse transform process is, for example, a wavelet inverse transform process.

[0039] In this embodiment, an edge image is used as the evaluation image, but a contour-extracted edge image in which the contour of the object to be observed is extracted by various contour extraction processes other than the multi-resolution conversion process may also be used as the evaluation image. Alternatively, a change amount edge image in which an area in which the change amount of spatial frequency is equal to or greater than a certain value may also be used as the evaluation image.

[0040] When the evaluation image generating unit 61 generates either a high-frequency edge image or a low-frequency edge image, it uses one of the images as the evaluation edge image. On the other hand, when the evaluation image generating unit 61 generates both a plurality of high-frequency edge images and a plurality of low-frequency edge images, it generates the evaluation edge image by performing a weighting operation based on the plurality of high-frequency edge images and the low-frequency edge images. By using such a weighting operation, it becomes easier to obtain edges, which are structural information, while avoiding the influence of disturbances such as noise.

[0041] Specifically, in the weighting calculation, a high-frequency weighting coefficient kH for the high-frequency edge image and a low-frequency weighting coefficient kL for the low-frequency edge image are determined, and as shown in Figure 10, the high-frequency edge image HEP multiplied by the high-frequency weighting coefficient kH and the low-frequency edge image LEP multiplied by the low-frequency weighting coefficient kL are added together to generate the evaluation edge image EEP.

[0042] When the image acquisition unit 60 acquires at least one of a first illumination light image and a second illumination light image as an endoscopic image, the first weighting calculation based on at least the first high-frequency edge image and the first low-frequency edge image generated from the first illumination light image is different from the second weighting calculation based on at least the second high-frequency edge image and the second low-frequency edge image generated from the second illumination light image. In the first weighting calculation, the low-frequency weighting coefficient kL1 for the first low-frequency edge image is made larger than the high-frequency weighting coefficient kH1 for the first high-frequency edge image. For example, a ratio of kH1:kL1=3:6 is preferred. In contrast, in the second weighting calculation, the high-frequency weighting coefficient kH2 for the second high-frequency edge image is made larger than the low-frequency weighting coefficient kL2 for the second low-frequency edge image. For example, a ratio of kH1:kL1=6:3 is preferred.

[0043] As described above, the ratio between the high-frequency weighting coefficients and the low-frequency weighting coefficients is changed between the first weighting calculation and the second weighting calculation for the following reason. In order for the evaluation value calculation unit 62 to accurately calculate the evaluation value, the evaluation edge image needs to contain more high-frequency components, which are structural information. The extent to which high-frequency components are contained in an endoscopic image is determined by the ratio of the number of B pixels, G pixels, and R pixels of the image sensor 37. For example, if the image sensor 37 has a Bayer array in which the ratio of the number of pixels is B pixels (B):G pixels (G):R pixels (R) = 1:2:1 as shown in FIG. 11, the G pixels contain more high-frequency components than the B pixels or R pixels.

[0044] Therefore, when the image sensor 37 has a Bayer array, if the first illumination light image used in the first weighting calculation is an image based on a first illumination light such as white light, the low-frequency weighting coefficient kL1 is made larger than the high-frequency weighting coefficient kH1, taking into consideration that the G pixels, which have sufficient brightness compared to the B pixels, which lack brightness, contain more high-frequency components. However, if the second illumination light image used in the second weighting calculation is an image based on a second illumination light in which the intensity of short-wavelength illumination light is greater than the emission spectrum of the first illumination light, the G pixels do not contain many high-frequency components, so the high-frequency weighting coefficient kH2 is made larger than the low-frequency weighting coefficient kL2 to include more high-frequency components.

[0045] The evaluation value calculation unit 62 calculates a calculated evaluation value by evaluation value calculation processing based on specific pixels whose pixel-by-pixel evaluation values ​​of the evaluation image are within a specific range. Specifically, as shown in FIG. 12, the evaluation value calculation unit 62 uses the pixel values ​​PV included in each pixel Px of the evaluation edge image EEP as the pixel evaluation value. The specific range is preferably a range in which the pixel values ​​of the evaluation edge image are equal to or less than a certain value in order to exclude high luminance values. Therefore, the specific pixels whose pixel values ​​PV of the evaluation edge image are within a specific range exclude pixels other than those with high luminance values, while including pixels of structural information such as blood vessels.

[0046] The evaluation value calculation process calculates at least one of the sum of the number of specific pixels, the maximum edge length of the edge lengths formed by the specific pixels, or the number of horizontal lines, vertical lines, or diagonal lines based on the specific pixels. Specifically, the sum of the number of specific pixels is preferably the sum of the number of specific pixels included in the evaluation edge image. The maximum edge length of the edge lengths formed by the specific pixels is preferably the edge length with the maximum length of the multiple edges included in the evaluation edge image. The number of horizontal lines, vertical lines, or diagonal lines based on the specific pixels is preferably the number of horizontal lines, vertical lines, or diagonal lines of the edges included in the evaluation edge image.

[0047] The image storage control unit 63 stores a specific calculated evaluation value that satisfies a specific condition and an endoscopic image that corresponds to the specific calculated evaluation value. of Alternatively, when the maximum calculated evaluation value among the calculated evaluation values ​​is calculated, the maximum calculated evaluation value is associated with the endoscopic image corresponding to the maximum calculated evaluation value and stored in the image storage memory 52. ​​As described above, by performing image storage control using multiple frequency images, it is possible to store an image in which not only is blur suppressed but also structural information such as blood vessels is clearly displayed even if multiple high-brightness areas are present. Note that in this embodiment, image storage control is performed by the image storage control unit 63 during the still image acquisition period SPP when a still image acquisition instruction is issued, but image storage control may also be performed automatically or manually during other periods.

[0048] Specifically, of the calculated evaluation values ​​calculated during the still image acquisition period SPP, it is preferable to set the calculated evaluation values ​​that are equal to or greater than a certain threshold as specific calculated evaluation values. For example, as shown in Fig. 13, when five calculated evaluation values ​​EV1 to EV5 are calculated during the still image acquisition period SPP, if the calculated evaluation values ​​EV1 and EV3 are equal to or greater than a certain threshold, the calculated evaluation values ​​EV1 and EV1 corresponding to the calculated evaluation value EV1, and the calculated evaluation values ​​EV3 and EV3 corresponding to the calculated evaluation value EV3 are stored in the image storage memory 52.

[0049] 14 , when image saving control is performed during the still image acquisition period SPP, if the calculated evaluation value EV1 is calculated first during the still image acquisition period SPP, the image saving control unit 63 sets the calculated evaluation value EV1 as the largest calculated evaluation value and associates it with the endoscopic image MP1 corresponding to the calculated evaluation value EV1 and saves it in the image saving memory 52. ​​Next, if the calculated calculated evaluation values ​​EV2, EV3, and EV4 are all smaller than the calculated evaluation value EV1, the image saving control unit 63 does not perform image saving control. Then, if the calculated evaluation value EV5 calculated after the calculated evaluation value EV4 is larger than the calculated evaluation value EV1, the image saving control unit 63 deletes the already-saved calculated evaluation value EV1 and endoscopic image MP1, and newly saves the calculated evaluation value EV5 and the endoscopic image MP5 corresponding to the calculated evaluation value EV5 in the image saving memory 52.

[0050] In the multi-light-emission mode, the image saving control unit 63 selects and performs one of two patterns of image saving control. In the multi-light-emission mode, as shown in Fig. 15, the image acquisition unit 60 acquires an endoscopic image group MPG including a first illumination light image MPx and a second illumination light image MPy as an endoscopic image in accordance with a specific acquisition cycle. In the case of Fig. 15, the specific acquisition cycle is a cycle in which an endoscopic image group MPG, each set of which includes two frames of a first illumination light image MPx and one frame of a second illumination light image MPy, is repeatedly acquired.

[0051] The first pattern of image storage control in the multi-flash mode is a pattern in which the evaluation value calculation unit 62 calculates a calculated evaluation value based on only either the first illumination light image or the second illumination light image. In this case, as shown in Fig. 16 , when an endoscopic image group MPG1 (first illumination light images MPx1, 2 and second illumination light image MPy1), an endoscopic image group MPG2 (first illumination light images MPx3, 4 and second illumination light image MPy2), and an endoscopic image group MPG3 (first illumination light images MPx5, 6 and second illumination light image MPy3) are acquired, calculated evaluation values ​​EVx1-6 are calculated for the first illumination light images MPx1-6 in each endoscopic image group. Then, the image storage control unit 63 stores in the image storage memory 52 an endoscopic image included in the endoscopic image group MPG1 that corresponds to a calculated evaluation value EVx1 equal to or greater than a certain threshold value among the calculated calculated evaluation values ​​EVx1-6, in association with the calculated evaluation value EVx1. In this case, it is preferable that the endoscopic image group MPG1 includes at least one frame each of the first illumination light image and the second illumination light image. Note that, although in Fig. 16 only the first illumination light image is displayed on the display 15 and the second illumination light image is not displayed on the display 15, this is not limitative (the same applies to Fig. 17).

[0052] A second pattern of image saving control in the multi-light emission mode is a pattern in which the evaluation value calculation unit 62 calculates calculated evaluation values ​​based on each of the first illumination light images and the second illumination light images. In this case, as shown in Fig. 17, when an endoscopic image group MPG1 (first illumination light images MPx1, 2 and second illumination light image MPy1), an endoscopic image group MPG2 (first illumination light images MPx3, 4 and second illumination light image MPy2), and an endoscopic image group MPG3 (first illumination light images MPx5, 6 and second illumination light image MPy3) are acquired, calculated evaluation values ​​EVx1-6 are calculated for the first illumination light images MPx1-6, and calculated evaluation values ​​EVy1-3 are calculated for the second illumination light images MPy1-3.

[0053] Then, the image saving control unit 63 saves the first illumination light image MPx1 corresponding to the calculated evaluation value EVx1 that is equal to or greater than a certain threshold value among the calculated calculated evaluation values ​​EVx1 to EVx6 in association with the calculated evaluation value EVx1 in the image saving memory 52. ​​In addition, the image saving control unit 63 saves the second illumination light image MPy2 corresponding to the calculated evaluation value EVy2 that is equal to or greater than a certain threshold value among the calculated calculated evaluation values ​​EVy1 to EVy3 in association with the calculated evaluation value EVy2 in the image saving memory 52.

[0054] Next, a series of steps in the image saving control of the present invention will be described with reference to the flowchart in Fig. 18. When the still image acquisition instruction switch 12h or the like is operated, image saving control is performed. In the image saving control, the image acquisition unit 60 acquires an endoscopic image. The evaluation image generation unit 61 generates a plurality of frequency images with different spatial frequencies from the endoscopic image. Furthermore, the evaluation image generation unit 61 generates an evaluation image to be used for image evaluation based on the plurality of frequency images.

[0055] The evaluation value calculation unit 62 calculates a calculated evaluation value by evaluation value calculation processing based on specific pixels whose pixel-by-pixel evaluation values ​​are within a specific range of the evaluation image. The image saving control unit 63 associates a specific calculated evaluation value that satisfies a specific condition among the calculated evaluation values ​​with an endoscopic image corresponding to the specific calculated evaluation value and saves them in the image saving memory 52. ​​Furthermore, when the maximum calculated evaluation value is calculated among the calculated evaluation values, the image saving control unit 63 associates the maximum calculated evaluation value with the endoscopic image corresponding to the maximum calculated evaluation value and saves them in the image saving memory 52.

[0056] In this embodiment, an example has been described in which the processor device 14 is connected to the endoscope system 10, but the present invention is not limited to this, and other medical devices such as an ultrasound imaging device or a radiography device may also be used. Furthermore, the endoscope 12 may be a rigid endoscope or a flexible endoscope.

[0057] In this embodiment, the hardware structure of processing units that perform various processes, such as the image acquisition unit 60, the evaluation image generation unit 61, the evaluation value calculation unit 62, and the image storage control unit 63, is made up of various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) to function as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, which is a processor with a circuit configuration designed specifically for performing various processes.

[0058] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor. Examples of multiple processing units configured with a single processor include, first, a configuration in which a single processor is configured with a combination of one or more CPUs and software, as typified by client or server computers, and this processor functions as multiple processing units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by a System on Chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0059] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit formed by combining circuit elements such as semiconductor elements, and the hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). [Explanation of symbols]

[0060] 10 Endoscopy System 12 Endoscopy 12a Insertion part 12b Operation section 12c curved section 12d Tip 12e Angle Knob 12f Observation mode switch 12h Still image acquisition command switch 12i Zoom control 12j forceps mouth 13 Light source device 14 Processor unit 15 Display 16 User Interface 20 Light source section 20a V-LED 20b B-LED 20c G-LED 20d R-LED 21 Light Source Processor 22 Optical path coupling section 23 Light Guide 30 Illumination optical system 31 Lighting lens 32 Imaging optical system 35 objective lens 36 Zoom Lens 37 Image sensor 44 Imaging processor 50 Central Control Unit 51 Program Memory 52 Image storage memory 60 Image acquisition unit 61 Evaluation image generation unit 62 Evaluation value calculation unit 63 Image storage control unit HFP High Frequency Image HEP High Frequency Edge Image LFP Low Frequency Image LEP Low Frequency Edge Image EEP evaluation edge image MP endoscopic image MPG endoscopic images MPG1~MPG3 Endoscopic image group MPx 1st illumination light image MPy 2nd illumination light image PV pixel value Px pixels

Claims

1. a processor; The processor: Obtaining endoscopic images, generating a plurality of frequency images having different spatial frequencies from the endoscopic image, and generating an evaluation image to be used for image evaluation based on the plurality of frequency images; calculating a calculated evaluation value by an evaluation value calculation process based on specific pixels whose pixel-by-pixel evaluation value of the evaluation image is in a specific range; performing image storage control to associate a specific calculated evaluation value that satisfies a specific condition among the calculated evaluation values ​​with an endoscopic image corresponding to the specific calculated evaluation value and store them in an image storage memory, or, when a maximum calculated evaluation value among the calculated evaluation values ​​is calculated, to associate the maximum calculated evaluation value with an endoscopic image corresponding to the maximum calculated evaluation value and store them in an image storage memory; The processor generates at least a high frequency image and a low frequency image, the spatial frequency of which is lower than that of the high frequency image; the processor generates an evaluation edge image from at least one of the high frequency image and the low frequency image as the evaluation image; the pixel evaluation value is a pixel value of the evaluation edge image, the specific range is a range in which pixel values ​​of the evaluation edge image are equal to or less than a certain value, The specific condition is that the calculated evaluation value is equal to or greater than a certain threshold value.

2. the processor acquires, as the endoscopic image, at least one of a first illumination light image obtained by imaging an observation target illuminated with first illumination light and a second illumination light image obtained by imaging an observation target illuminated with second illumination light in which the intensity of illumination light at a short wavelength is higher than the intensity of the emission spectrum of the first illumination light; 2. The image processing device according to claim 1, wherein when acquiring the first illumination light image, the processor performs a first weighting calculation based on at least a first high-frequency edge image and a first low-frequency edge image generated from the first illumination light image, and when generating an evaluation edge image as the evaluation image, the processor performs the first weighting calculation by making the low-frequency weighting coefficient for the first low-frequency edge image larger than the high-frequency weighting coefficient for the first high-frequency edge image.

3. the processor acquires, as the endoscopic image, at least one of a first illumination light image obtained by imaging an observation target illuminated with first illumination light and a second illumination light image obtained by imaging an observation target illuminated with second illumination light in which the intensity of illumination light at a short wavelength is higher than the intensity of the emission spectrum of the first illumination light; 2. The image processing device according to claim 1, wherein when acquiring the second illumination light image, the processor performs a second weighting operation based on at least a second high-frequency edge image and a second low-frequency edge image generated from the second illumination light image, and when generating an evaluation edge image as the evaluation image, the processor performs the second weighting operation by making the high-frequency weighting coefficient for the second high-frequency edge image larger than the low-frequency weighting coefficient for the second low-frequency edge image.

4. An image processing device as described in claim 1, wherein the calculated evaluation value is obtained by an evaluation value calculation process based on specific pixels whose pixel values ​​of the evaluation edge image are within a specific range.

5. 5. The image processing device according to claim 4, wherein the evaluation value calculation process calculates at least one of the sum of the number of the specific pixels, the maximum edge length among the edge lengths formed by the specific pixels, or the number of horizontal lines, vertical lines, or diagonal lines based on the specific pixels.

6. the processor acquires, as the endoscopic images, a group of endoscopic images including a first illumination light image obtained by capturing an image of an observation target illuminated with first illumination light and a second illumination light image obtained by capturing an image of an observation target illuminated with second illumination light having a short wavelength and a higher intensity than the emission spectrum of the first illumination light, in accordance with a specific acquisition cycle; The processor calculates the calculated evaluation value based on only one of the first illumination light image or the second illumination light image; 4. An image processing device according to claim 1, wherein the processor stores in the image storage memory an endoscopic image included in an endoscopic image group corresponding to an evaluation image whose calculated evaluation value satisfies the specific condition, in association with the calculated evaluation value.

7. the processor acquires, as the endoscopic images, a group of endoscopic images including a first illumination light image obtained by capturing an image of an observation target illuminated with first illumination light and a second illumination light image obtained by capturing an image of an observation target illuminated with second illumination light having a short wavelength and a higher intensity than the emission spectrum of the first illumination light, in accordance with a specific acquisition cycle; The processor calculates the calculated evaluation value based on each of the first illumination light image and the second illumination light image; 4. The image processing device according to claim 1, wherein the processor stores in the image storage memory a first illumination light image corresponding to an evaluation image whose calculated evaluation value satisfies the specific condition, and a second illumination light image corresponding to an evaluation image whose calculated evaluation value satisfies the specific condition, in association with the calculated evaluation value.

8. 8. The image processing apparatus according to claim 1, wherein the image saving control is performed during a still image acquisition period when a still image acquisition instruction is issued.

9. The processor: acquiring an endoscopic image; generating a plurality of frequency images having different spatial frequencies from the endoscopic image, and generating an evaluation image to be used for image evaluation based on the plurality of frequency images; calculating a calculated evaluation value by evaluation value calculation processing based on specific pixels whose pixel-by-pixel evaluation values ​​of the evaluation image are in a specific range; and performing an image storage control step of storing in an image storage memory a specific calculated evaluation value that satisfies a specific condition among the calculated evaluation values ​​and an endoscopic image corresponding to the specific calculated evaluation value, in association with each other, or, when a maximum calculated evaluation value among the calculated evaluation values ​​is calculated, storing in an image storage memory the maximum calculated evaluation value and the endoscopic image corresponding to the maximum calculated evaluation value, in association with each other, The processor generates at least a high frequency image and a low frequency image, the spatial frequency of which is lower than that of the high frequency image; the processor generates an evaluation edge image from at least one of the high frequency image and the low frequency image as the evaluation image; the pixel evaluation value is a pixel value of the evaluation edge image, the specific range is a range in which pixel values ​​of the evaluation edge image are equal to or less than a certain value, The specific condition is that the calculated evaluation value is equal to or greater than a certain threshold.

Images