Endoscope processor and endoscope system

Abstract

The image processing unit of the processor for an endoscope includes: an enhancement processing calculation unit that takes each pixel of a camera image as a pixel of interest and performs enhancement processing by performing nonlinear grayscale conversion on the pixel value of the pixel of interest; and a preprocessing unit that sets a reference upper limit characteristic line and a reference lower limit characteristic line in order to adjust the output pixel value after the nonlinear grayscale conversion, and calculates the degree of deviation of the pixel value in a partial set area determined around the pixel of interest. The enhancement processing calculation unit calculates the output ratio of the output pixel value after the nonlinear grayscale conversion of the pixel value relative to the maximum pixel value that the camera image can take, and uses the reference upper limit characteristic line and the reference lower limit characteristic line to calculate the adjusted upper limit value and the lower limit value according to the degree of deviation of the pixel value. The pixel value after the enhancement processing is calculated using the adjustment upper limit value, the adjustment lower limit value and the output ratio.

Description

Technical Field

[0001] The present invention relates to an endoscope processor and an endoscope system, which include an image processing unit for performing grayscale conversion on a captured image of living tissue in a body cavity. Background Art

[0002] Endoscopes are used in the observation and treatment of living tissue inside the human body. Images of living tissue that are photographed using an endoscope and displayed on a display are often images that have been processed so that the living tissue or a portion of the living tissue of interest can be easily visually identified compared to other portions. Image processing includes processing for enhancing contrast, processing for brightening the entire image, processing for changing the color tone, and enhancement processing for grayscale conversion to expand the grayscale value of a predetermined grayscale range.

[0003] For example, in the enhancement process, a tone curve is often provided for changing and outputting the grayscale of an input image. Such a tone curve is stored and maintained in advance, and during the enhancement process, the stored and maintained tone curve is called to perform enhancement processing on the image obtained through the endoscope. Therefore, how to perform the enhancement process is determined by the tone curve.

[0004] For example, there is known an electronic endoscope device technology (see Japanese Patent No. 5006759) that can obtain an image with blood vessels sufficiently enhanced while using white light.

[0005] In this technology, there are: an RGB conversion unit for converting an image signal into an RGB signal; an R signal amplification unit for increasing or decreasing the R signal in the RGB signal with a certain gain; and a GB signal amplification unit for nonlinearly increasing or decreasing the G signal and the B signal in the RGB signal. At this time, the gains in the R signal amplification unit and the GB signal amplification unit are stored as values ​​specific to each part of the observation object, and the gains are selected according to the part of the observation object.

[0006] Furthermore, there is known an image processing technique (see Japanese Patent No. 5006759) that can convert the brightness of each pixel with high accuracy even in an image with continuous brightness.

[0007] In this technology, an upper limit conversion function and a lower limit return function that determine the output upper limit and output lower limit of the brightness conversion are determined based on the brightness of each pixel in a local area in a camera image, and the upper limit value and the lower limit value of the brightness are calculated using these two conversion functions. Then, based on the brightness of each pixel, the ratio of the value used to set the upper limit value and the lower limit value is calculated using the upper limit value and the lower limit value, and the calculated upper limit value, lower limit value and ratio are used to calculate the value after the brightness conversion (refer to Japanese Patent Gazette No. 5966603). SUMMARY OF THE INVENTION

[0008] Problems to be solved by the invention

[0009] In the above-mentioned electronic endoscope device technology, the contrast of the image is reduced in the portion where the tone curve changes slowly. Therefore, for example, the image that should be recognizable is damaged in the dark portion, which causes difficulty in recognition.

[0010] On the other hand, in the above-mentioned image processing technology, during the brightness conversion, the noise component in the image is enhanced in the portion where the brightness value changes sharply, so in the converted image, black spots are sometimes scattered in the flat portion (for example, a white area with high brightness).

[0011] Therefore, the object of the present invention is to provide an endoscope processor and an endoscope system, which can avoid destroying the image of parts that are easily damaged in the past, such as dark parts, when enhancing the image by grayscale conversion. It can also suppress the enhancement of noise in the flat parts within the image.

[0012] Technical solutions to solve problems

[0013] One embodiment of the present invention is an endoscope processor including an image processing unit that performs grayscale conversion on a captured image of living tissue in a body cavity. The image processing unit of the endoscope processor includes:

[0014] an enhancement processing calculation unit that performs enhancement processing of the input captured image by performing nonlinear grayscale conversion on a pixel value of a pixel of interest of the input captured image;

[0015] and a preprocessing unit, which sets a baseline upper limit characteristic line and a baseline lower limit characteristic line for the input pixel value, sets a partial set area including the focus pixel in the camera image and calculates the degree of deviation of the pixel value in the partial set area, wherein the baseline upper limit characteristic line represents a baseline adjustment upper limit value of the output pixel value output by the nonlinear grayscale conversion, and the baseline lower limit characteristic line represents a baseline adjustment lower limit value of the output pixel value for adjusting the output pixel value.

[0016] The enhanced processing calculation unit,

[0017] By inputting the pixel value and performing the nonlinear grayscale conversion, an output ratio of the output pixel value to the maximum pixel value that the camera image can take is calculated,

[0018] The reference adjustment upper limit value and the reference adjustment lower limit value obtained by inputting the pixel value and using the reference upper limit characteristic line and the reference lower limit characteristic line are adjusted according to the degree of deviation of the pixel value, thereby calculating the adjustment upper limit value and the adjustment lower limit value,

[0019] A value of a dividing point between the adjustment upper limit value and the adjustment lower limit value is calculated according to the output ratio, and the calculated value of the dividing point is output as a pixel value after the enhancement process.

[0020] In addition, another embodiment of the present invention is also an endoscope processor, which includes an image processing unit that performs grayscale conversion on a captured image of living tissue in a body cavity. The image processing unit of the endoscope processor includes:

[0021] an enhancement processing calculation unit that performs enhancement processing of the input captured image by performing nonlinear grayscale conversion on pixel values ​​of pixels of interest within a set partial area of ​​the input captured image;

[0022] and a preprocessing unit for calculating a degree of deviation of the pixel values ​​in the partially set area of ​​the captured image.

[0023] The enhanced processing calculation unit,

[0024] An upper characteristic line and a lower characteristic line are set for the input pixel value according to the degree of deviation of the pixel value in the partially set area, wherein the upper characteristic line represents an adjustment upper limit value of the output pixel value output by the nonlinear grayscale conversion, and the lower characteristic line represents an adjustment lower limit value of the output pixel value for adjusting the output pixel value,

[0025] By inputting the pixel value and performing the nonlinear grayscale conversion, an output ratio of the output pixel value to the maximum pixel value that the camera image can take is calculated,

[0026] By inputting the pixel value and using the upper limit characteristic line and the lower limit characteristic line, the adjustment upper limit value and the adjustment lower limit value are calculated.

[0027] A value of a dividing point between the adjustment upper limit value and the adjustment lower limit value is calculated according to the output ratio, and the calculated value of the dividing point is output as a pixel value after the enhancement process.

[0028] Preferably, the upper limit characteristic line and the lower limit characteristic line are lines obtained by scaling a reference upper limit characteristic line and a reference lower limit characteristic line as references toward the output side according to the degree of deviation.

[0029] Preferably, each of a plurality of predetermined block areas in the captured image is set as the partial set area.

[0030] Preferably, the value of the division point is a value obtained by weightedly averaging the adjustment upper limit value and the adjustment lower limit value using the output ratio and a ratio remaining after subtracting the output ratio from 1 as weighting coefficients.

[0031] Preferably, the larger the deviation is, the larger the difference between the adjustment upper limit value and the adjustment lower limit value is.

[0032] Preferably, the larger the deviation is, the smaller the adjustment lower limit is.

[0033] Preferably, the larger the deviation is, the larger the adjustment upper limit is.

[0034] Preferably, the rate of change of the adjustment upper limit value relative to the change of the input pixel value increases as the deviation increases.

[0035] A rate of change of the adjustment lower limit value with respect to a change of the input pixel value decreases as the deviation increases.

[0036] Preferably, the nonlinear grayscale conversion is set based on the distribution of pixel values ​​within the partially set area.

[0037] Preferably, the camera image is composed of pixel values ​​of multiple color components.

[0038] The processing and calculation in the preprocessing section and the enhancement processing calculation section are performed for each of the color components.

[0039] Another embodiment of the present invention is an endoscope system including: the endoscope processor; and an endoscope connected to the endoscope processor and including an imaging element for imaging the living tissue.

[0040] Effects of the Invention

[0041] According to the endoscope processor and endoscope system of the present invention, when the grayscale conversion of the captured image is performed for enhancement processing, the image of the dark portion, which is a portion that is easily damaged in the past, can be preserved, and enhancement of noise in the flat portion within the image can be suppressed. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1This is a diagram for explaining nonlinear grayscale conversion performed by an endoscope processor of an endoscope system in one embodiment.

[0043] Figure 2 It is a diagram for explaining a partial setting area, a reference upper limit characteristic line, and a reference lower limit characteristic line set by an endoscope processor of an endoscope system in one embodiment.

[0044] Figure 3 It is used to illustrate and Figure 2 The diagram shows different methods for calculating the adjustment upper limit value and the adjustment lower limit value.

[0045] Figure 4 This is a diagram for explaining calculation processing of enhancement processing executed by an endoscope processor of an endoscope system in one embodiment.

[0046] Figure 5 This is a perspective view of the appearance of an endoscope system according to one embodiment.

[0047] Figure 6 This is a block diagram showing the main structure of an endoscope system according to one embodiment.

[0048] Figure 7 This is a diagram showing an example of the flow of enhancement processing executed by the endoscope processor according to one embodiment. DETAILED DESCRIPTION

[0049] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Figure 1 This is a diagram for explaining nonlinear grayscale conversion performed by an endoscope processor of an endoscope system in one embodiment. Figure 2 It is a diagram for explaining a partial setting area, a reference upper limit characteristic line, and a reference lower limit characteristic line set by an endoscope processor of an endoscope system in one embodiment. Figure 3 It is used to illustrate and Figure 2 The diagram shows different methods for calculating the adjustment upper limit value and the adjustment lower limit value.

[0050] Figure 4 This is a diagram for explaining calculation processing of enhancement processing executed by an endoscope processor of an endoscope system in one embodiment.

[0051] The endoscope processor in this embodiment uses nonlinear grayscale conversion to enhance the camera image captured by the endoscope. The enhancement process using nonlinear grayscale conversion is described below. As described later, the enhancement process using nonlinear grayscale conversion is performed by the image processing unit of the endoscope processor. More specifically, the calculation of the deviation of the pixel value in the partial area described below is performed by the preprocessing unit of the endoscope processor (see Figure 5 ), the calculation of the adjustment upper limit value and the adjustment lower limit value, and the enhancement processing using the output ratio, the adjustment upper limit value and the adjustment lower limit value described later obtained by the nonlinear grayscale conversion are performed by the enhancement processing calculation unit (refer to Figure 5 )conduct.

[0052] (Enhancement using non-linear grayscale conversion)

[0053] like Figure 1 As shown in FIG. 1 , the enhancement processing calculation unit performs nonlinear grayscale conversion on the pixel values ​​in the captured image of the living tissue in the body cavity captured by the endoscope. Figure 1 In the example shown, the conversion curve of the nonlinear grayscale conversion is expressed as an output ratio relative to an input ratio. The input ratio and the output ratio are the ratios of the input pixel value or the output pixel value after the nonlinear conversion relative to the maximum pixel value that can be obtained by the pixel value of the camera image. For example, when the 8-bit AD conversion is performed, the pixel value of the camera image is expressed as a grayscale value of 0 to 255. Figure 1 In the example shown, the output ratio relative to the input ratio is expressed as a conversion curve of nonlinear grayscale conversion, but the conversion curve may be expressed as an output pixel value curve relative to the input pixel value.

[0054] On the other hand, in one embodiment, Figure 2 As shown, each pixel in the input image is set as a focus pixel, and a partial setting area is set for the focus pixel. The partial setting area is, for example, an area located within a predetermined frame with the focus pixel of the captured image as the center pixel, such as a rectangular frame area. Therefore, a partial setting area is set for each focus pixel. The focus pixel is moved so as to scan pixel by pixel in the captured image.

[0055] In another embodiment, the partial setting area may be set in advance as a pre-fixed area instead of being set according to the pixel of interest. For example, a user observing a camera image displayed on a display connected to the processor for the endoscope may input and indicate the area to be enhanced by using an input operation device such as a mouse, thereby setting the partial setting area in the preprocessing unit. In addition to being set by the user's input, the partial setting area may also be configured such that, by analyzing the pixel values ​​of the camera image, the preprocessing unit automatically sets as the partial setting area a portion of the image that is easily damaged due to nonlinear grayscale conversion or a portion where noise contained in the pixel value is easily apparent.

[0056] Furthermore, the preprocessing unit calculates the degree of deviation of the pixel values ​​in the set partial setting area. As the degree of deviation of the pixel values, for example, the variance, standard deviation, absolute deviation, and the difference between the maximum pixel value and the minimum pixel value of the pixel values ​​can be listed. The degree of deviation of the pixel values ​​is used for calculating the adjustment upper limit value and the adjustment lower limit value described below.

[0057] Figure 2 The reference upper limit characteristic line shown is a line indicating a reference adjustment upper limit value of an output pixel value, which adjusts the output pixel value to be output by nonlinear grayscale conversion for the input pixel value. The reference lower limit characteristic line is a line indicating a reference adjustment lower limit value of an output pixel value, which is used to adjust the output pixel value. Figure 2 As shown, the reference upper limit characteristic line and the reference lower limit characteristic line are represented by the output pixel value relative to the input pixel value. The form of the reference upper limit characteristic line and the reference lower limit characteristic line is preferably a form in which the output pixel value remains constant or increases as the input pixel value increases, and is also preferably a so-called monotonous increase in which the output pixel value increases as the input pixel value increases. The reference upper limit characteristic line is set so that when the pixel value of the camera image is represented by a grayscale value of 0 to 255, when the input pixel value is 255, the output pixel value is 255, that is, when the input pixel value is the maximum grayscale value, the output pixel value is also the maximum grayscale value. The reference lower limit characteristic line is set so that when the pixel value of the camera image is represented by a grayscale value of 0 to 255, when the input pixel value is 0, the output pixel value is 0, that is, when the input pixel value is the minimum grayscale value, the output pixel value is also the minimum grayscale value. Furthermore, the reference lower limit characteristic line is set to be located on the side where the output pixel value is smaller than the reference upper limit characteristic line. In this case, when the input pixel value is close to the minimum grayscale value, preferably, the slope of the reference lower limit characteristic line is gentler than the slope of the reference upper limit characteristic line. When the input pixel value of the reference upper limit characteristic line is the minimum grayscale value, the output pixel value does not need to pass through the minimum grayscale value, and when the input pixel value of the reference lower limit characteristic line is the maximum grayscale value, the output pixel value does not need to pass through the maximum grayscale value.

[0058] Such a reference upper limit characteristic line and reference lower limit characteristic line may be a straight line or a curve, or a combination of a straight line and a curve. The reference upper limit characteristic line and reference lower limit characteristic line may also be represented by a preset function.

[0059] Such a reference upper limit characteristic line and reference lower limit characteristic line are set in advance by the preprocessing unit.

[0060] like Figure 2 As shown in , the benchmark upper limit characteristic line and the benchmark lower limit characteristic line are used to calculate the benchmark adjustment upper limit value and the benchmark adjustment lower limit value. Figure 2 As shown, when the pixel value of the pixel of interest is input as the input pixel value, the reference adjustment upper limit value and the reference adjustment lower limit value corresponding to the input pixel value (input pixel value) are calculated based on the reference upper limit characteristic line and the reference lower limit characteristic line.

[0061] Furthermore, the adjustment upper limit value and the adjustment lower limit value are calculated by adjusting the calculated reference adjustment upper limit value and the reference adjustment lower limit value according to the degree of deviation of the pixel value in the partial set area calculated by the preprocessing unit. For example, the adjustment upper limit value is calculated so that the larger the deviation of the pixel value is, the larger the pixel value is relative to the reference adjustment upper limit value, and the adjustment lower limit value is calculated so that the pixel value is smaller relative to the reference adjustment lower limit value.

[0062] In the above-mentioned embodiment, the benchmark upper limit characteristic line and the benchmark lower limit characteristic line are predetermined, and the input pixel value of the focus pixel is used to calculate the benchmark adjustment upper limit value corresponding to the input pixel value through the benchmark upper limit characteristic line, and the benchmark adjustment lower limit value corresponding to the input pixel value is calculated through the benchmark lower limit characteristic line, and the benchmark adjustment upper limit value and the benchmark adjustment lower limit value are adjusted according to the degree of deviation of the pixel value in the partial set area, thereby calculating the adjustment upper limit value and the adjustment lower limit value. However, as described above, in the case where the partial set area is not set according to the focus pixel, but a predetermined fixed area is set in advance, such as Figure 3 As shown, the upper limit adjustment value and the lower limit adjustment value can be calculated by fixing the upper limit characteristic line and the lower limit characteristic line set according to the deviation of the pixel values ​​in the partially set area, so as to calculate the upper limit adjustment value and the lower limit adjustment value corresponding to the input pixel value. In the case where the degree of deviation of the pixel values ​​in the partially set area is large, for example, the upper limit characteristic line is adjusted to increase the upper limit adjustment value; in the case where the degree of deviation of the pixel values ​​in the partially set area is large, for example, the lower limit characteristic line is adjusted to decrease the lower limit adjustment value. In the case where the upper limit characteristic line and the lower limit characteristic line are represented by a set function, it is preferred that the numerical value representing the deviation of the pixel value is used as a parameter for changing the above function. In this case, it is preferred that the above parameter is incorporated into the above function according to the numerical value representing the degree of deviation, so that when the degree of deviation is large, for example, the upper limit adjustment value becomes larger and the lower limit adjustment value becomes smaller.

[0063] The upper limit characteristic line and the lower limit characteristic line are preferably in a form in which the output pixel value remains constant or increases as the input pixel value increases. The upper limit characteristic line is set so that when the input pixel value is the maximum grayscale value, the output pixel value is also the maximum grayscale value. The lower limit characteristic line is set so that when the input pixel value is the minimum grayscale value, the output pixel value is also the minimum grayscale value. Further, the lower limit characteristic line is set so that it is located on the side where the output pixel value is smaller than the upper limit characteristic line. In this case, when the input pixel value is close to the minimum grayscale value, preferably, the slope of the lower limit characteristic line is gentler than the slope of the upper limit characteristic line.

[0064] In the enhancement processing calculation unit, the output ratio calculated by inputting pixel values ​​and performing nonlinear grayscale conversion and the output ratio calculated by using Figure 2 or Figure 3 The method shown in the figure calculates the adjustment upper limit value and the adjustment lower limit value, calculates the value of the division point between the adjustment upper limit value and the adjustment lower limit value according to the output ratio, and outputs the calculated value of the division point as the pixel value after the enhancement process.

[0065] like Figure 4 As shown, when the adjustment upper limit value is set to 1 and the adjustment lower limit value is set to 0, the output ratio of the values ​​between 0 and 1 is used as the dividing point between 0 and 1, the value of the dividing point between the adjustment upper limit value and the adjustment lower limit value is calculated, and the value of the dividing point is output as the pixel value after enhancement processing.

[0066] That is, the acceptable range of pixel values ​​is determined by using the upper and lower adjustment limits determined according to the deviation of the pixel values ​​in the partially set area, and within the acceptable range, the value corresponding to the above-mentioned output ratio is used as the pixel value after the enhancement process. For example, when the upper adjustment limit is 150, the lower adjustment limit is 50, and the output ratio is 0.4, the value corresponding to the output ratio of 0.4 is 90 (= (150-50) · 0.4 + 50). This value is used as the pixel value after the enhancement process.

[0067] Therefore, when the range of possible grayscale values ​​of the pixel value is, for example, 0 to 255, when the upper adjustment limit value is set to 255 and the lower adjustment limit value is set to 0, the pixel value after the enhancement processing is the same as the pixel value obtained by the nonlinear grayscale conversion, but the upper adjustment limit value and the lower adjustment limit value vary according to the degree of deviation of the pixel value in the partially set area, so the value is different from the pixel value obtained by the nonlinear grayscale conversion. That is, according to the degree of deviation of the pixel value in the partially set area, the pixel value after the enhancement processing varies according to the change of the upper adjustment limit value and the lower adjustment limit value.

[0068] Therefore, when grayscale conversion is performed on a captured image and enhancement processing is performed, a portion of the image that is easily damaged, such as a dark portion, is set as a partial setting area, and a portion where noise is enhanced by nonlinear grayscale conversion is set as a partial setting area, thereby determining an upper adjustment limit value and a lower adjustment limit value that vary according to the degree of deviation of the pixel value in the partial setting area. The pixel value after enhancement processing is calculated using the upper adjustment limit value and the lower adjustment limit value and the output ratio calculated by nonlinear grayscale conversion. Therefore, without destroying the image of the portion of the image that is easily damaged, such as a dark portion, it is possible to perform enhancement processing that is difficult to enhance noise in a flat portion within the image.

[0069] (Endoscope System)

[0070] Figure 5 is a perspective view of the appearance of the endoscope system 1 according to the present embodiment. Figure 6 2 is a block diagram showing a main structure of the endoscope system 1 .

[0071] The endoscope system 1 mainly includes an endoscope processor (hereinafter referred to as a processor) 2, a light source device 3, an endoscope 4, and a display 5. The light source device 3, the endoscope 4, and the display 5 are respectively connected to the processor 2. In addition, although the light source device 3 and the processor 2 are configured separately, the light source device 3 may be provided in the processor 2.

[0072] The light source device 3 emits white light or light of multiple wavelengths. The light source device 3, for example, has a white light source such as an LED (Light Emitting Diode) light source, a halogen lamp, a metal halide lamp or other high-brightness lamp. In addition, the light source device 3 may also have the following structure: by rotating a rotating plate equipped with a plurality of filters, the filters on the light path of the outgoing light from the white light source are switched, so that the light filtered by each filter is output in sequence. In this case, the light source device 3, for example, has a lamp power driver, a light source, a focusing lens, a filter turntable, a motor and a motor driver, which are not shown in the figure. In addition, the light source device 3 may also have a plurality of semiconductor light-emitting elements such as light-emitting diodes or laser diodes that emit light of various wavelengths.

[0073] like Figure 5 As shown, the front end of the endoscope 4 is provided with an insertion portion 42, which is flexible and is used for insertion into the human body. A bending portion 45 is provided near the front end of the insertion portion 42, and is bent by remote operation of a manual operation portion 44 connected to the base end of the insertion portion 42. The bending mechanism of the bending portion 45 is a well-known mechanism assembled in a general endoscope. The bending mechanism bends the bending portion 45 by pulling an operating wire, and the operating wire is linked to the rotation operation of the bending operation knob provided on the manual operation portion 44. A camera element 46A (see FIG. 1 ) is connected to the front end of the bending portion 45. Figure 6) The distal end portion 46. According to the bending action of the bending portion 45 caused by the rotation operation of the bending operation knob, the orientation of the distal end portion 46 changes accordingly, thereby moving the imaging area of ​​the endoscope 4.

[0074] The endoscope 4 includes a light guide 48 arranged over substantially the entire length from the connector portion 49 to the distal end portion 46 (see Figure 6 The light guide 48 is an optical fiber bundle, and guides the irradiation light supplied from the light source device 3 to the front end portion 46 of the electronic endoscope 4 .

[0075] Although not shown, the front end portion 46 of the electronic endoscope 4 includes: a light distribution lens disposed in front of the front end of the light guide 48; an objective lens 46B (see Figure 6 ), which images the image of the living tissue; the imaging element 46A (refer to Figure 6 ) which receives the imaged image; and an amplifier (not shown) which amplifies the image signal output from the imaging element 46A.

[0076] The light distributing lens is disposed opposite to the front end surface of the light guide 48 , and diverges the irradiation light emitted from the front end surface of the light guide 48 to illuminate the living tissue as the object.

[0077] The objective lens 46B collects scattered light or reflected light from the living tissue and forms an image of the subject on the light receiving surface of the imaging element 46A.

[0078] As the imaging element 46A, a CCD imaging element or a CMOS imaging element is preferably used.

[0079] The imaging signal output from the imaging element 46A is amplified by an amplifier (not shown) and then sequentially transmitted to the connector portion 49 .

[0080] The connector portion 49 is connected to the processor 2. The connector portion 49 may also include a circuit having the following functions. For example, the connector portion 49 may also include a circuit that drives the imaging element 46A by supplying a driving signal to the imaging element 46A, converts the analog signal AD output from the imaging element 46A into a digital signal, and transmits the signal as a captured image to the processor 2. In addition, the circuit may also have the following function: accessing a memory (not shown) provided on the connector portion 49 to read the inherent information of the endoscope 4, and outputting the information to the processor 2.

[0081] The processor 2 is a device that further processes the signal of the image captured by the imaging element 46A of the endoscope, and supplies the signal to the display 5 .

[0082] The processor 2 is provided with a connector unit (not shown) for connecting the endoscope 4. The connector unit is mechanically connected to the connector unit 49, so that the endoscope 4 and the processor 2 are electrically connected, and the light source device 3 and the endoscope 4 are optically connected.

[0083] The processor 2 includes an image input processing unit 21, an image memory 22, a result display unit 23, a system controller 24, and an image processing unit 25. Figure 6 , a structure in which a replaceable light source device 3 is provided in a housing in which a processor 2 is provided is shown.

[0084] The system controller 24 manages and controls the operations of the image input processing unit 21, the image memory 22, the result display unit 23, and the image processing unit 25, and controls the operations of the light source device 3 and the endoscope 4. The processor 2 includes a memory (not shown) for storing information and data in addition to the image memory 22.

[0085] The image input processing unit 21 performs predetermined signal processing such as color correction and matrix operation on the signal of the living tissue image. The video signal after the signal processing is sent to the image memory 22 as a frame memory for each pixel and temporarily stored.

[0086] Further, the image stored in the image memory 22 is called to perform enhancement processing, and is sent to the image processing section 25 via the system controller 24 according to an instruction of the system controller 24. The processing in the image processing section 25 will be described later.

[0087] The image processed by the image processing unit 25 is sent to the image memory 22 again and stored.

[0088] The result display unit 23 calls the image stored in the image memory 22, performs signal processing (gamma correction, etc.) on the image signal to generate an image signal for monitor display, and converts the generated image signal into a predetermined video format signal. The converted video format signal is output to the display 5. Thus, the captured image of the living tissue is displayed on the display screen of the display 5.

[0089] In such a processor 2, the image processing unit 25 is configured to perform image enhancement processing on the captured image of the living tissue. Figure 2 The following describes how to set the upper and lower limit values.

[0090] like Figure 6 As shown, the image processing unit 25 includes a preprocessing unit 25A and an enhancement processing calculation unit 25B. The preprocessing unit 25A sets the following for each pixel of interest: Figure 2The partial set area shown is used to perform the enhancement process using the above-mentioned nonlinear grayscale conversion, and the deviation of the pixel value in the partial set area is calculated. Further, the preprocessing unit 25A sets a reference upper limit characteristic line and a reference lower limit characteristic line.

[0091] like Figure 1 As shown, the enhancement processing calculation unit 25B inputs a pixel value and performs nonlinear grayscale conversion, thereby calculating an output ratio of an output pixel value relative to the maximum pixel value that the captured image can take. Figure 1 In the illustrated example, the output ratio is calculated using the input ratio which is the ratio of the input pixel value to the maximum pixel value.

[0092] Furthermore, if Figure 2 As shown, the enhancement processing calculation unit 25B calculates the reference adjustment upper limit value and the reference adjustment lower limit value by using the reference upper limit characteristic line and the reference lower limit characteristic line by inputting pixel values, and then calculates the adjustment upper limit value and the adjustment lower limit value after adjusting the reference adjustment upper limit value and the reference adjustment lower limit value according to the degree of deviation of the pixel values ​​in the partial set area. Figure 4 As shown, the enhancement processing calculation unit 25B calculates the value of the dividing point between the adjustment upper limit value and the adjustment lower limit value according to the output ratio by using the calculated input ratio as the range value of 0 to 1, the calculated adjustment upper limit value and the calculated adjustment lower limit value, and outputs the calculated dividing point value as the pixel value after the enhancement processing. Such enhancement processing is performed on each pixel.

[0093] Figure 7 This is a diagram showing an example of the flow of enhancement processing executed by a processor according to one embodiment.

[0094] Figure 7 It is shown that by calculating Figure 2 The above is an example of a flow of performing enhancement processing by adjusting the upper limit value and the lower limit value.

[0095] like Figure 6 As shown in FIG. 1 , first, the camera image of the current frame is input to the image processing unit 25 (step S10). Then, the preprocessing unit 25A sets each pixel as a focus pixel and sets a partial setting area around the focus pixel including the focus pixel (step S12). For example, the partial setting area is set so that the focus pixel is set as the center pixel. The partial setting area is, for example, as follows: Figure 2 The rectangular area shown.

[0096] Next, the preprocessing unit 25A calculates the deviation of the pixel values ​​of the set partial area in the set captured image (step S14). As described above, the deviation may be, for example, the variance, standard deviation, absolute deviation, or the difference between the maximum pixel value and the minimum pixel value.

[0097] The enhancement processing calculation unit 25B uses the set Figure 1 The nonlinear grayscale conversion function shown calculates an output ratio by performing a nonlinear grayscale conversion on the input ratio of the pixel value of the pixel of interest (step S16).

[0098] Furthermore, if Figure 2 As shown, the enhancement processing calculation unit 25B uses the reference upper limit characteristic line and the reference lower limit characteristic line that are set in advance or according to the setting of the partial setting area, takes the pixel value in the partial area as the input pixel value, and calculates the output pixel value of the reference upper limit characteristic line and the output pixel value of the reference lower limit characteristic line corresponding to the input pixel value as the reference adjustment upper limit value and the reference adjustment lower limit value (step S18). In addition, if the reference upper limit characteristic line and the reference lower limit characteristic line are set, the reference adjustment upper limit value and the reference adjustment lower limit value can be calculated before setting the partial setting area, so step S18 can also be performed between step S10 and step S12.

[0099] The enhancement processing calculation unit 25B adjusts the calculated reference adjustment upper limit value and reference adjustment lower limit value according to the degree of deviation of the pixel value calculated in step S14, so that Figure 2 The upper adjustment limit value and the lower adjustment limit value are calculated as shown in step S20. The upper adjustment limit value is, for example, a value obtained by multiplying the quantized value of the deviation of the pixel value (for example, the standard deviation) by the positive coefficient α as an adjustment value and added to the reference upper adjustment limit value, and the lower adjustment limit value is, for example, a value obtained by multiplying the quantized value of the deviation of the pixel value (for example, the standard deviation) by the negative coefficient β as an adjustment value and added to the reference lower adjustment limit value. In this case, the greater the degree of deviation, the greater the upper adjustment limit value, and the smaller the lower adjustment limit value. The coefficients α and β are set according to the set reference upper characteristic line and reference lower characteristic line.

[0100] The enhancement processing calculation unit 25B uses the calculated adjustment upper limit value and adjustment lower limit value, and the calculated output ratio to calculate the pixel value after the enhancement processing (step S22). Specifically, the enhancement processing calculation unit 25B calculates (adjustment upper limit value - adjustment lower limit value) · output ratio + adjustment lower limit value as the pixel value after the enhancement processing. Figure 4 As shown in FIG. 1 , the calculated value is the calculated dividing point value, which is calculated by dividing the dividing point value between the adjustment upper limit value and the adjustment lower limit value by the output ratio. In addition, (adjustment upper limit value-adjustment lower limit value)·output ratio+adjustment lower limit value can also be expressed as output ratio·adjustment upper limit value+(1-output ratio)·adjustment lower limit value, so the dividing point value can also be said to be a value obtained by weighted averaging the adjustment upper limit value and the adjustment lower limit value using the output ratio and the remaining ratio after subtracting the output ratio from 1 as weighting coefficients.

[0101] In the image processing unit 25, it is determined whether all pixels of the camera image have been enhanced (step S24). When the determination result is negative, the pixel of interest is moved by one pixel, and the process returns to step S12, and the partial setting area is reset around the pixel of interest, and steps S14 to S24 are repeated. In this way, when the determination result is positive, in the image processing unit 25, the image after image enhancement processing is displayed on the display 5 via the result display unit 23 (step S26). In addition, when the partial setting area is set according to an input instruction, or the partial setting area is automatically extracted and fixed according to the analysis result of the camera image, although not shown in the figure, steps S14 to S22 are repeated for each partial setting area.

[0102] Furthermore, the image processing unit 25 determines whether the input of the frame image continues (step S28). If the determination is positive, the process returns to step S10 and steps S10 to S28 are repeated. In this way, the enhancement process is repeated until the determination result is negative.

[0103] In this way, the pixel value after enhancement processing is calculated using the output ratio obtained by nonlinear grayscale conversion and the upper and lower adjustment limits set according to the degree of deviation of the pixel values ​​in the partially set area. Therefore, although there is a deviation in the pixel value, the dark damaged part is set as the partially set area by nonlinear grayscale conversion of the image. Due to the relatively large deviation of the pixel values ​​in the dark part, the upper adjustment limit becomes larger and the lower adjustment limit becomes smaller, so the pixel value obtained by the output ratio is Figure 4 The values ​​of the distribution points shown tend to change relatively greatly with respect to changes in the input pixel values. Therefore, damage to dark areas is suppressed. On the other hand, by setting the portion with small pixel value deviation as the partial setting area, since the deviation of the pixel values ​​in the partial setting area is small, the adjustment upper limit value and the adjustment lower limit value become small, and the difference between them also becomes small. In the portion where the pixel value changes sharply due to nonlinear conversion, adjustment is performed so that Figure 4 The value at the distribution point shown has a smaller change relative to the input pixel value, thereby suppressing the enhancement of noise.

[0104] like Figure 2 As shown, in the above embodiment, the reference upper limit characteristic line and the reference lower limit characteristic line which are set in advance are used to calculate the reference adjustment upper limit value and the reference adjustment lower limit value, and the adjustment upper limit value and the adjustment lower limit value are calculated according to the degree of deviation of the pixel values ​​in the partially set area. However, in other embodiments, such as Figure 3As shown, the enhancement processing calculation unit 25B can also set the upper limit characteristic line and the lower limit characteristic line according to the degree of deviation of the pixel value in the partial setting area. The upper limit characteristic line is a characteristic line representing the adjusted upper limit value of the output pixel value, and the adjusted upper limit value of the output pixel value is used to adjust the output pixel value output after the nonlinear grayscale conversion of the input pixel value, and the lower limit characteristic line is a characteristic line representing the adjusted lower limit value of the output pixel value, and the adjusted lower limit value of the output pixel value is used to adjust the output pixel value. In this case, by inputting the pixel value into the enhancement processing calculation unit 25B, the upper limit characteristic line and the lower limit characteristic line are used to directly calculate the adjusted upper limit value and the adjusted lower limit value. In such an embodiment, since the setting and fixing of the partial setting area is irrelevant to the pixel of interest, the upper limit characteristic line and the lower limit characteristic line are preferably lines that are scaled to the output side by the reference upper limit characteristic line and the reference lower limit characteristic line as the reference according to the degree of deviation of the pixel value in the partial setting area. Thus, it is possible to easily set the upper limit characteristic line and the lower limit characteristic line whose adjusted upper limit value and adjusted lower limit value change according to the degree of deviation of the pixel value. Regarding scaling of the reference upper limit characteristic line and the reference lower limit characteristic line toward the output side according to the degree of deviation, for example, the reference upper limit characteristic line and the reference lower limit characteristic line are scaled using a scaling factor that changes according to a value indicating the degree of deviation.

[0105] According to one embodiment, preferably, the greater the deviation of the pixel value in the partially set area, the greater the difference between the upper adjustment limit and the lower adjustment limit. As a result, the pixel value deviation becomes larger, and therefore the change of the pixel value after the enhancement processing also becomes larger according to the change of the output ratio, so that the damage of the dark part image such as image damage can be suppressed by nonlinear grayscale conversion. In addition, in the part with a smaller pixel value deviation, the difference between the upper adjustment limit and the lower adjustment limit becomes smaller, and therefore the change of the pixel value after the enhancement processing becomes smaller. Therefore, the enhancement of noise is suppressed.

[0106] According to one embodiment, preferably, the greater the deviation of the pixel values ​​in the partially set area, the smaller the adjustment lower limit value is. In addition, preferably, the greater the deviation of the pixel values ​​in the partially set area, the greater the adjustment upper limit value is.

[0107] According to one embodiment, preferably, the rate of change of the upper limit value of the adjustment relative to the change of the input pixel value is not constant, but increases as the deviation of the pixel value in the partially set area increases, and the rate of change of the lower limit value of the adjustment relative to the change of the input pixel value is not constant, but decreases as the deviation of the pixel value in the partially set area increases. Thus, image damage can be more effectively suppressed, and noise enhancement can be more effectively suppressed.

[0108] In the above embodiment, Figure 1The conversion characteristic curve representing the change of output relative to input in the nonlinear grayscale conversion shown is not limited to a pre-set curve. According to one embodiment, it is preferably set according to the distribution of pixel values ​​in a partially set area. That is, the nonlinear grayscale conversion preferably changes with the change of the distribution of pixel values ​​in the partially set area. Thus, the nonlinear grayscale conversion is characterized according to the characteristics of the pixel values ​​in the partially set area, so that preferred enhancement processing can be achieved. For example, a histogram curve represented by a cumulative histogram of pixel values ​​in a partially set area after normalization within the range of 0 to 1 can be used as a conversion characteristic curve. In addition, Figure 1 The output ratio of the pixel subjected to the enhancement processing may be a value obtained by dividing the number of pixels having a smaller pixel value than the pixel value of the pixel by the number of pixels in the partial set area.

[0109] In addition, the camera image captured by the endoscope 4 is composed of pixel values ​​of multiple color components. In this case, it is preferable to perform processing and calculation by the preprocessing unit 25A and the enhancement processing calculation unit 25B for each color component. Since the degree of enhancement of image damage and noise varies depending on the color component, at least one of the nonlinear grayscale conversion, the reference upper limit characteristic line, and the reference lower limit characteristic line can be set differently for each color component.

[0110] In the above embodiment, the partial setting area to be set is set for each pixel of interest, but the partial setting area may be a block area that is set in advance and fixed in the captured image. In this case, each of the plurality of block areas set in the captured image may be fixed in advance as a partial setting area. Since the upper adjustment limit value and the lower adjustment limit value in the enhancement process can be changed by parameters other than the pixel value deviation (for example, the coefficients α, β), the reference upper limit characteristic line, or the reference lower limit characteristic line for each partial setting area, it is possible to perform enhancement processes with different characteristics for each partial setting area.

[0111] The endoscope processor and the endoscope system of the present invention are not limited to the above-described embodiments, and various improvements and changes can be made without departing from the gist of the present invention.

[0112] The present invention relates to a patent application of Japanese Patent Application No. 2020-154912 filed with the Japan Patent Office on September 15, 2020, and the entire contents of the application are incorporated into this specification by reference.

Claims

1. An endoscope processor, comprising an image processing unit for performing grayscale conversion on a captured image of living tissue in a body cavity, It is characterized in that The image processing unit comprises: an enhancement processing calculation unit that performs enhancement processing of the input captured image by performing nonlinear grayscale conversion on a pixel value of a pixel of interest of the input captured image; and a preprocessing unit, which sets a reference upper limit characteristic line and a reference lower limit characteristic line for the input pixel value, sets a partial set area including the focus pixel in the camera image, and calculates the degree of deviation of the pixel value in the partial set area, wherein the reference upper limit characteristic line represents a reference adjustment upper limit value of the output pixel value for adjusting the output pixel value output by the nonlinear grayscale conversion, and the reference lower limit characteristic line represents a reference adjustment lower limit value of the output pixel value for adjusting the output pixel value; Wherein, the enhanced processing calculation unit, By inputting the pixel value and performing the nonlinear grayscale conversion, an output ratio of the output pixel value to the maximum pixel value that the camera image can take is calculated, The reference adjustment upper limit value and the reference adjustment lower limit value obtained by inputting the pixel value and using the reference upper limit characteristic line and the reference lower limit characteristic line are adjusted according to the degree of deviation of the pixel value, thereby calculating the adjustment upper limit value and the adjustment lower limit value, A value of a dividing point between the adjustment upper limit value and the adjustment lower limit value is calculated according to the output ratio, and the calculated value of the dividing point is output as a pixel value after the enhancement process.

2. An endoscope processor, comprising an image processing unit for performing grayscale conversion on a captured image of living tissue in a body cavity, It is characterized in that The image processing unit comprises: an enhancement processing calculation unit that performs enhancement processing of the input captured image by performing nonlinear grayscale conversion on pixel values ​​of pixels of interest within a set partial area of ​​the input captured image; and a preprocessing unit for calculating a degree of deviation of the pixel values ​​in the partially set area of ​​the camera image; The enhanced processing calculation unit, An upper characteristic line and a lower characteristic line are set for the input pixel value according to the degree of deviation of the pixel value in the partially set area, wherein the upper characteristic line represents an adjustment upper limit value of the output pixel value output by the nonlinear grayscale conversion, and the lower characteristic line represents an adjustment lower limit value of the output pixel value for adjusting the output pixel value, By inputting the pixel value and performing the nonlinear grayscale conversion, an output ratio of the output pixel value to the maximum pixel value that the camera image can take is calculated, By inputting the pixel value and using the upper limit characteristic line and the lower limit characteristic line, the adjustment upper limit value and the adjustment lower limit value are calculated. A value of a dividing point between the adjustment upper limit value and the adjustment lower limit value is calculated according to the output ratio, and the calculated value of the dividing point is output as a pixel value after the enhancement process.

3. The endoscope processor according to claim 2, wherein: The upper limit characteristic line and the lower limit characteristic line are lines obtained by scaling a reference upper limit characteristic line and a reference lower limit characteristic line, which are references, toward the output side according to the degree of deviation.

4. The endoscope processor according to claim 2 or 3, wherein: Each of a plurality of predetermined block areas in the captured image is set as the partial setting area.

5. The endoscope processor according to any one of claims 1 to 3, wherein: The value of the division point is a value obtained by weightedly averaging the adjustment upper limit value and the adjustment lower limit value using the output ratio and a ratio remaining after subtracting the output ratio from 1 as weighting coefficients.

6. The endoscope processor according to any one of claims 1 to 3, wherein: The larger the deviation is, the larger the difference between the adjustment upper limit value and the adjustment lower limit value is.

7. The endoscope processor according to any one of claims 1 to 3, wherein: The larger the deviation is, the smaller the adjustment lower limit value is.

8. The endoscope processor according to any one of claims 1 to 3, wherein: The larger the deviation is, the larger the adjustment upper limit is.

9. The endoscope processor according to any one of claims 1 to 3, wherein: The rate of change of the adjustment upper limit value relative to the change of the input pixel value increases as the deviation increases. A rate of change of the adjustment lower limit value with respect to a change of the input pixel value decreases as the deviation increases.

10. The endoscope processor according to any one of claims 1 to 3, wherein: The nonlinear grayscale conversion is set based on the distribution of pixel values ​​within the partially set area.

11. The endoscope processor according to any one of claims 1 to 3, wherein: The camera image is composed of pixel values ​​of multiple color components. The processing and calculation in the preprocessing section and the enhancement processing calculation section are performed for each of the color components.

12. An endoscope system comprising: an endoscope processor according to any one of claims 1 to 11; An endoscope is connected to the endoscope processor and includes an imaging element for imaging the living tissue.

Images