Image processing methods

Abstract

An image processing method provides a method for partially eliminating prominent brightness while maintaining overall brightness and contrast for the image. The pixel values ​​(brightness) of reference pixels (P1) through (P8) are sequentially compared with the pixel value of a pixel of interest (P0). The histogram value of the reference pixel with a value less than or equal to the pixel value of the pixel of interest (P0) and equal to +1 is then compared with the slope limit value. The number of reference pixels with a value less than or equal to the slope limit value is counted as the true count value for both pixels, and the brightness of the pixel of interest is output based on these true count values.

Description

Technical Field

[0001] The first invention relates to an image processing method for converting a camera image in which a mixture of extremely bright and dark parts such as a window in a room is present in the same frame, or an image in which a mixture of parts with extremely low contrast such as water or fog is present in the same frame, into an image that is easy to see.

[0002] The second invention relates to an image processing method for automatically correcting blocked upshadows in an input image.

[0003] The third invention relates to an image processing method that combines an image that has been subjected to a sharpening process to transform an image containing mixed portions of extremely low contrast into an easily visible image with an image that has been subjected to a correction process to correct underexposure of dark portions of the image. Background Art

[0004] As an image processing method for converting a picture in which extremely bright and dark parts are mixed into a picture that is easy to see, Patent Document 1 discloses an imaging device comprising: an imaging element that performs photoelectric conversion on an optical image and outputs an electrical signal; a signal processing unit that processes the electrical signal output from the imaging element to generate an image signal; a histogram circuit that generates a histogram based on the image signal output from the signal processing unit; and an exposure control unit that performs exposure control using the histogram detected by the histogram circuit.

[0005] In addition, patent document 2 discloses an image processing method, which includes: a process of reading light from an original and generating image data; a process of creating a histogram of concentration distribution based on the image data; a process of generating a concentration correction curve based on the ratio of the number of data close to the light and dark ends of the concentration distribution to the total number of data of the image data; and a process of performing concentration correction on the image data using the concentration correction curve.

[0006] Patent document 3 discloses a camera device comprising: a camera unit that captures a subject and obtains image data of the captured image; and a grayscale correction unit that performs grayscale correction on a pixel portion consisting of pixels whose brightness levels are within a given range in a given pixel portion of the captured image constituted by the image data obtained by the camera unit. In this grayscale correction, the brightness interval between the pixels is increased while maintaining the upper and lower relationship of the relative brightness between the pixels.

[0007] In addition, patent document 2 discloses an image processing method, which includes: a process of reading light from an original and generating image data; a process of creating a histogram of concentration distribution based on the image data; a process of generating a concentration correction curve based on the ratio of the number of data close to the light and dark ends of the concentration distribution to the total number of data of the image data; and a process of performing concentration correction on the image data using the concentration correction curve.

[0008] Furthermore, patent document 3 discloses a camera device comprising: a camera unit that captures a subject and obtains image data of the captured image; and a grayscale correction unit that performs grayscale correction on a pixel portion consisting of pixels whose brightness levels are within a given range in a given pixel portion of the captured image constituted by the image data obtained by the camera unit. In this grayscale correction, the brightness interval between the pixels is increased while maintaining the upper and lower relationship of the relative brightness between the pixels.

[0009] The image processing methods disclosed in Patent Documents 1 to 3 above require a long processing time unless a high-performance computer or device is used, and are therefore not suitable for real-time processing of high-definition or higher-resolution moving images.

[0010] The inventors proposed in Patent Document 4 a method for shortening processing time and enabling real-time processing of moving images. Patent Document 4 describes a method that captures pixel-by-pixel image data from a captured image on a single channel basis, decomposes the captured image data pixel by pixel in a specific color space, generates a brightness histogram, reads this image brightness information using a predetermined reading pattern, and sets the brightness of pixels at a specific position (center) in the reading pattern based on the average histogram, excluding pixels at a specific position (center) in the reading pattern.

[0011] Patent document 5 proposed by the same inventor discloses the following: when a tone mapping diagram is obtained by locally dividing an input signal, and when the contrast of the input signal is improved by correcting the tone mapping diagram, for areas with little brightness change, a gradient limit is set when converting the input brightness signal of the tone mapping diagram into an output brightness signal to suppress the brightness change so that the output brightness signal does not change more than a certain amount, and then the overall brightness of the area reduced by the gradient limit is adjusted as a whole.

[0012] Underexposed images, where dark areas cannot be visually identified, are a phenomenon that can be seen in underexposed images or backlit images. For example, this phenomenon occurs when shooting at night using a video camera that cannot achieve a shutter speed longer than 1 / 30 second, or when the exposure, gain / aperture are limited by the automatic exposure function in backlighting. This phenomenon is unavoidable due to the structure of general cameras.

[0013] Patent Document 6 describes a method for restoring an image captured under conditions with large brightness differences, such as backlighting. Specifically, as described in paragraphs (0038) and (0039), a histogram is created. If the intensity at zero brightness is greater than a threshold, it is determined that the dark portion is underexposed, and the brightness of that portion is corrected.

[0014] Patent document 7 states that when the brightness difference between the dark and bright parts of an image becomes larger, the normal camera mode is switched to the synthetic camera mode, and dark part underexposure correction is performed to reduce the brightness difference. Therefore, automatic exposure control is performed on the long-exposure image signal and the short-exposure image signal respectively.

[0015] Patent document 8 describes the following: a long-exposure image signal with a long exposure time and a short-exposure image with a short exposure time are synthesized, a brightness cumulative value and a histogram are generated for the synthesized image signal, dark underexposure in the synthesized image signal is detected based on the brightness histogram, a target brightness cumulative value is set based on the detection result, and the target brightness cumulative value is used to perform exposure correction control of the above-mentioned camera unit.

[0016] Patent Document 9 discloses an imaging device capable of obtaining the remaining battery capacity. The document describes a device comprising a determination unit that displays a battery segment when the remaining battery capacity is low or a battery failure occurs. This determination unit determines when the number of pixels with underexposed dark areas or overexposed bright areas exceeds a predetermined value, thereby displaying a histogram.

[0017] Prior art literature

[0018] Patent Literature

[0019] Patent Document 1: Japanese Patent Application Laid-Open No. 2002-142150

[0020] Patent Document 2: Japanese Patent Application Laid-Open No. 2003-051944

[0021] Patent Document 3: Japanese Patent Application Laid-Open No. 2007-124087

[0022] Patent Document 4: JP Patent No. 4386959

[0023] Patent Document 5: JP Patent No. 6126054

[0024] Patent Document 6: Japanese Patent Application Laid-Open No. 2011-223173

[0025] Patent Document 7: Japanese Patent Application Laid-Open No. 2002-084449

[0026] Patent Document 8: Japanese Patent Application Laid-Open No. 2008-228058

[0027] Patent Document 9: Japanese Patent Application Publication No. 2016-092798 Summary of the Invention

[0028] -Problems to be solved by the invention-

[0029] According to the method disclosed in Patent Document 4, the processing time is significantly shortened compared to Patent Documents 1 to 3, and real-time processing of moving images is possible.

[0030] However, the method described in Patent Document 4 may partially produce a portion where the brightness fluctuates excessively, or may fail to remove noise.

[0031] According to the content disclosed in Patent Document 5, an image obtained by reading or importing image data such as AHE (Adaptive Histogram Equalization) or CLAHE (Contract Limited AHE) in a given pattern is divided into multiple blocks, a histogram is generated for each small area, and a tone map is created for each of the areas.

[0032] In this way, if a tone map is created for each small area, the boundary lines between the small areas may not be clearly visible.

[0033] Furthermore, since AHE and CLAHE scan an image and create a histogram for each area to create a tone map, they require a large amount of high-speed memory for real-time processing and cannot perform independent processing on a pixel-by-pixel basis. Therefore, they are not suitable for GPUs and FPGAs.

[0034] In Patent Documents 6 to 9, when performing backlight and shadow correction (dark underexposure correction), a brightness histogram is created and then flattened. Flattening the histogram requires counting the histogram for the entire image, which requires a large memory and increases processing time.

[0035] Furthermore, there is a problem that the contrast in dark areas becomes excessive in the sharpening process, and the contrast in bright areas is often insufficient in the dark area underexposure correction process. No conventional technology has proposed a processing method that satisfies both of these problems.

[0036] -Methods for solving the problem-

[0037] The premise of the first invention is that the acquired image data is not divided into small regions but is processed independently in pixel units, that is, a histogram is not generated for each region.

[0038] That is, in the first invention, when the image data taken in is processed in units of pixels, n (n is an integer) reference pixels are set around the pixel of interest which becomes the scanning position of the image scan, the pixel value (brightness) of the pixel of interest and the pixel value (brightness) of each reference pixel are compared in sequence, the number of reference pixels having pixel values ​​lower than the value of the pixel of interest is counted, and when the brightness comparison with the n reference pixels is completed, the count value is proportionally distributed as the output brightness.

[0039] In addition, in the first invention, when the image data taken in is processed in units of pixels, n (n is an integer) reference pixels are set around the focus pixel which becomes the scanning position of the image scan, the pixel value (brightness) of the focus pixel and the pixel value (brightness) of each reference pixel are compared in sequence, and the number of reference pixels which are below the focus pixel value and below the tilt limit value by comparing the histogram value + 1 of each reference pixel with a pre-set tilt limit value is counted to obtain a count value taking the tilt limit into account and output it.

[0040] In the first invention, instead of actually using a lot of memory to create a histogram and a tone map for each region as in the past, the same result as creating a tone map can be obtained by executing logic.

[0041] In order to correct the brightness of the darkened image due to the gradient limit in the image processing method, the number of reference pixels that are not counted in the comparison with the gradient limit value is counted, and the number of reference pixels that are not counted can be added to the output as an offset value.

[0042] In order to automatically calculate the fixed offset value for each area, the average value of the reference pixel values ​​is calculated, and the number of reference pixels that are not counted in the comparison with the slope limit value is counted. The output can be performed taking into account the average value of the reference pixel values ​​and the number of reference pixels that are not counted.

[0043] In order to improve the overall contrast of the image processed as described above, the number of reference pixels not counted in the comparison with the inclination limit value is counted, and the output can be performed taking into account the number of reference pixels not counted.

[0044] In addition, in order to improve the overall brightness and contrast, the average value of the reference pixel value is calculated, and the number of reference pixels that are not counted in the comparison with the slope limit value is counted, and the output can be performed taking into account these average values ​​and the number of reference pixels that are not counted.

[0045] Furthermore, in an overall dark image, when pixel values ​​tend toward 0, the entire image may appear whitish. To eliminate this, the pixel value of interest is compared with each reference pixel value, and the number of reference pixels with pixel values ​​equal to the pixel value of interest and the number of reference pixels with pixel values ​​smaller than the pixel value of interest are counted separately. The former count value is added to the latter count value in proportion to the value of the pixel of interest, and the average is then taken into account for output.

[0046] The second invention assumes that the input image is not divided into small areas, but rather processed independently on a pixel-by-pixel basis. Specifically, rather than creating a histogram for each area, brightness distribution information is obtained from the image data, and brightness adjustment is performed only on dark areas according to the darkness.

[0047] That is, in the second invention, a Blur plane for luminance blur is created from the Y plane (luminance plane) of the input image, dark areas of the Blur plane are corrected, and the input image is divided by the corrected Blur plane.

[0048] In the second invention, a flat field frame is not created as a frame buffer (memory), but a Gaussian blur value (Blur value) of brightness is calculated with reference to the surroundings of the pixel of interest, and a level transformation (correction of distribution information) is performed on a pixel-by-pixel basis, and the pixel of interest is divided by the level transformation value.

[0049] More specifically, a Blur value obtained by performing Gaussian Blur processing on the brightness of each pixel in the Y-plane (luminance plane) memory of the input image is obtained, and this Blur value is normalized according to the bit depth to set distribution information having a value between 0 and 1.0. A threshold is then set between the normalized values ​​of 0 and 1.0, and all pixels with values ​​greater than this threshold are set to 1.0. For dark pixels with values ​​less than the threshold, the brightness magnification (n) of the darkest pixel is determined, and the distribution information is corrected so that its reciprocal (1 / n) is the minimum value of the distribution information. The brightness of the input image is then divided by the corrected dark brightness distribution information (1 / n to 1.0) as the denominator.

[0050] In the above-described processing, by performing the processing for calculating the Blur value and the processing for correcting the luminance distribution information in parallel, it is possible to reduce delay.

[0051] In the case of a row buffer (FPGA), parallel processing can be performed at the stage where the level transformation (correction of distribution information) of the number of rows equal to the diameter of the Gaussian filter is completed. In the case of a kernel filter (GPU, CPU), since the above-mentioned process can be directly installed, the output can be performed just after the kernel filter operation is completed.

[0052] In the present invention, the image is not scanned to create a histogram, but the brightness distribution information is obtained from the image data, and only the brightness of the dark parts is adjusted according to the darkness. Therefore, the processing speed is greatly improved and the corrected image can be displayed in real time.

[0053] Unlike in the past, it is not necessary to actually use a lot of memory to create a histogram and tone map for each region, but the same result as creating a histogram and tone map can be obtained by executing logic.

[0054] Memory for histograms and tone maps is no longer required, and they can be implemented using only line buffers. Since the delay is only the amount of the line buffer, there is no need for full-screen statistics such as histogram acquisition, so there is no frame-by-frame lag.

[0055] Since the logic itself is simple, it can be installed in the empty areas of small-scale FPGAs and other imaging circuits. In addition, since the logic and memory such as histogram arrangement are proportional to the pixel depth (number of bits), the circuit scale is almost unchanged even with high bit depths such as 36 bits and 48 bits.

[0056] Correction can be performed based on a single image without referencing previous or subsequent frames, and saturation can be emphasized by processing RGB individually.

[0057] In the image processing method involved in the third invention, the image processing combines an image that has been subjected to a sharpening process and an image that has been subjected to a dark portion underexposure process, and utilizes the principle of a flat field frame used in the dark portion underexposure process to increase the dark portion underexposure process ratio for the dark portion of the image and to increase the sharpening process ratio for the bright portion.

[0058] In a specific example of the sharpening process, when the image data taken in is processed in units of pixels, n (n is an integer) reference pixels are set around the pixel of interest that becomes the scanning position of the image scan, the pixel value (brightness) of the pixel of interest and the pixel value (brightness) of each reference pixel are compared in sequence, the number of reference pixels having a pixel value below the pixel of interest is counted, and in parallel, the histogram value of each reference pixel is increased by 1 and compared with a pre-set inclination limit value, and the conditions that the reference pixel value is below the pixel of interest and the condition that the histogram value of the pixel is below the inclination limit value are satisfied. By counting the number of reference pixels that meet both conditions, the true count values ​​of both sides are obtained, and the true count values ​​of both sides are output in proportion.

[0059] In a specific example of the dark portion underexposure processing, a Blur value obtained by performing Gaussian Blur processing on the brightness of each pixel in the Y-plane (luminance plane) memory of the input image is obtained, and the Blur value is normalized to set distribution information having a value between 0 and 1.0. A threshold is further set between the normalized values ​​of 0 and 1.0, and all pixels with values ​​greater than the threshold are set to 1.0. For dark portion pixels with values ​​less than the threshold, a brightness magnification (n) of the darkest pixel is determined, and the distribution information is corrected so that the reciprocal (1 / n) is the minimum value of the distribution information. The brightness of the input image is divided by the corrected dark portion brightness distribution information (1 / n to 1.0) as the denominator.

[0060] -Effects of the Invention-

[0061] According to the first invention, problems that remain after the sharpening process and the dark portion underexposure correction process cannot be solved can be solved simultaneously.

[0062] The first invention is based on the premise that the captured image data is not divided into small regions, but is processed independently on a pixel-by-pixel basis. In other words, since a histogram is not generated for each region, the processing speed is greatly improved, and the corrected image can be displayed in real time.

[0063] Instead of actually using a lot of memory to create histograms and tone maps for each region as in the past, the same result as creating histograms and tone maps can be obtained by executing logic.

[0064] Memory for histograms and tone maps is no longer required; they can be stored solely in the line buffer. The latency is limited to the line buffer, and full-screen statistics such as histogram acquisition are unnecessary, resulting in no frame-by-frame lag.

[0065] Because the logic itself is simple, it can be installed in the empty areas of small-scale FPGAs and other imaging circuits. In addition, since there is no logic or memory proportional to the pixel depth (number of bits), such as histogram arrangement, the circuit scale is almost unchanged even at high bit depths such as 36 bits and 48 bits. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a basic diagram for explaining the logical image processing method according to the first invention.

[0067] Figure 2 is with Figure 1 The corresponding hypothetical tone map is obtained by averaging the histograms of the reference pixels.

[0068] Figure 3 This is a diagram showing an implementation example of the image processing method according to the first invention.

[0069] Figure 4 is with Figure 3 The corresponding hypothetical tone map.

[0070] Figure 5 It is a diagram showing another implementation example of the image processing method according to the first invention.

[0071] Figure 6 is with Figure 5 The corresponding hypothetical tone map.

[0072] Figure 7 It is a diagram showing another implementation example of the image processing method according to the first invention.

[0073] Figure 8 is with Figure 7 The corresponding hypothetical tone map.

[0074] Figure 9 It is a diagram showing another implementation example of the image processing method according to the first invention.

[0075] Figure 10 is with Figure 9 The corresponding hypothetical tone map.

[0076] Figure 11 It is a diagram showing another implementation example of the image processing method according to the first invention.

[0077] Figure 12 is with Figure 11 The corresponding hypothetical tone map.

[0078] Figure 13 It is a diagram showing another implementation example of the image processing method according to the first invention.

[0079] Figure 14 is with Figure 13 The corresponding hypothetical tone map.

[0080] Figure 15 This is a basic block diagram for explaining the logical image processing method according to the second invention.

[0081] Figure 16 (a) is a diagram illustrating horizontal Blur processing, and (b) is a diagram illustrating vertical Blur processing.

[0082] Figure 17 (a) is the source image, (b) is the image after Gaussian Blur processing, and (c) is the image after flat field processing.

[0083] Figure 18 This is a basic block diagram for explaining the logical image processing method according to the third invention.

[0084] Figure 19 (a) is a diagram illustrating horizontal blur processing in dark portion underexposure processing, and (b) is a diagram illustrating vertical blur processing.

[0085] Figure 20 This is a basic diagram for explaining the sharpening process.

[0086] Figure 21 Is the description and Figure 20 The corresponding hypothetical tone map is obtained by averaging the histograms of the reference pixels.

[0087] Figure 22 This is a diagram showing an example of implementation of the clarity processing.

[0088] Figure 23 is with Figure 22 The corresponding hypothetical tone map.

[0089] Figure 24 This is a diagram showing another implementation example of the clarity processing.

[0090] Figure 25 is with Figure 24 The corresponding hypothetical tone map.

[0091] Figure 26 This is a diagram showing another implementation example of the clarity processing.

[0092] Figure 27 is with Figure 26 The corresponding hypothetical tone map.

[0093] Figure 28 This is a diagram showing another implementation example of the clarity processing.

[0094] Figure 29 is with Figure 28 The corresponding hypothetical tone map.

[0095] Figure 30 This is a diagram showing another implementation example of the clarity processing.

[0096] Figure 31 is with Figure 30 The corresponding hypothetical tone map.

[0097] Figure 32 This is a diagram showing another implementation example of the clarity processing.

[0098] Figure 33 is with Figure 32 The corresponding hypothetical tone map.

[0099] Figure 34(a) is the source image, (b) is the image after dark portion underexposure processing, (c) is the image after sharpening processing, and (d) is the image after image synthesis (of the present invention). DETAILED DESCRIPTION

[0100] Figure 1 This is an explanatory diagram of the case where the image processing method according to the first invention is implemented. In the figure, P0 is a pixel of interest, and the brightness (luminance) of the pixel of interest P0 is adjusted. Figure 2 It will be based on Figure 1 The reference pixel when processing the method (in Figure 1 The execution results obtained by calculating the luminance of the reference pixel are 8 pixels (P1 to P8). The luminance of the reference pixel is represented as a virtual tone map.

[0101] The first invention obtains a single transformation result for the pixel of interest based on the pixel of interest and the reference pixels, and does not output a tone map.

[0102] In the processing steps, the brightness of the focus pixel P0 is first compared with the reference pixels P1 to P8 around it, and the number of reference pixels with brightness smaller than the brightness of the focus pixel P0 is counted. Corresponding to the count value, the brightness of the focus pixel P0 is corrected according to a given algorithm.

[0103] For example, if the number of reference pixels with a brightness lower than that of the pixel of interest P0 is one and the number of reference pixels with a brightness higher than that of P0 is seven, the maximum brightness that can be output is set to 1, and the reference pixel values ​​are corrected to 1 / 8 of the brightness. The correction algorithm is not limited to this.

[0104] When the above operation is implemented in an FPGA or CPU, the pixels of interest are moved one by one in the row direction, and the processing is performed in parallel for each row, thereby correcting the brightness of all pixels and smoothing the brightness.

[0105] Furthermore, when implemented on a GPU, since these operations are performed independently for each pixel, the brightness of all pixels is corrected and brightness is smoothed by performing parallel processing simultaneously using multiple cores.

[0106] on the other hand, Figure 2 The hypothetical tone map shown is not actually generated algorithmically, but rather illustrates the result of executing the logic for all input luminances for ease of understanding.

[0107] There is room for improvement in the above installation example. Figure 2In the virtual tone mapping diagram, there are three locations where the output increases sharply relative to the input, specifically, locations with an inclination of 45 degrees or more. These locations are locations where the brightness increases significantly, such as noise.

[0108] Figure 3 1 is a diagram showing an improved installation example of the above. Figure 4 is with Figure 3 The corresponding hypothetical tone map, Figure 4 The dotted line is the execution Figure 2 A hypothetical tone map for the logical case.

[0109] In order to eliminate such a portion with prominent brightness, the pixel values ​​(luminance) of the reference pixels P1 to P8 are sequentially compared with the pixel value of the attention pixel P0 to determine whether they are equal to or lower than the pixel value of the attention pixel P0.

[0110] In parallel with this, the histogram value +1 of the reference pixel is compared with the tilt limit value (45°) to determine whether it is equal to or less than the tilt limit value.

[0111] Then, the number of reference pixels for which both of the above two judgments are satisfied is counted as both true count values, and the brightness of the pixel of interest is output based on the both true count values.

[0112] Through the above processing, such as Figure 4 As shown by the solid line, an easily visible image can be obtained without any portion where the brightness changes significantly.

[0113] The slope limit is taken into account by counting pixel values ​​below the focus pixel value and comparing the histogram value of each reference pixel +1 with a preset slope limit value. The number of reference pixels below the slope limit value is set as the true count value of both.

[0114] On the other hand, in this state, the entire image may become dark. Figure 5 and Figure 6 The structure to correct this is shown in the hypothetical tone map of .

[0115] exist Figure 5 In the installation example shown, the number of reference pixels that satisfies both the conditions of being less than the pixel value of the focus pixel P0 and less than the inclination limit value is set as the actual count value of both parties, and then the number of reference pixels (non-count value) of the histogram with a larger inclination limit value is counted, and the brightness offset value is set to an external parameter with a value of 0 to n, and (non-count value × offset value / n) is added to the actual count value of both parties and output.

[0116] Here, the offset value is determined by how much of the brightness (a+b) of the end point value is reduced by raising the gradient limit.

[0117] Output = actual count value of both sides + (non-count value × offset value / n)

[0118] (n: reference pixel number, such as 128 or 256)

[0119] (Offset value is 0 to n)

[0120] Through the above, such as Figure 6 As shown, although there is no change in the oversparse tone mapping characteristics of the image, the entire image becomes brighter by the amount of the shift.

[0121] Figure 7 This is an example of an installation example in which the offset amount is automatically calculated for each area. In this installation example, the read reference pixel value and the focus pixel value are compared, the histogram value of the reference pixel is increased by 1 and compared with a pre-set tilt limit value. If the histogram value of the reference pixel is larger than the tilt limit value, it is counted as a non-count value.

[0122] In parallel with the above, the reference pixel values ​​below the preset tilt limit are simply added together and divided by the number of reference pixels to calculate the average value. The resulting value, divided by the maximum brightness of the pixel (256 for 8-bit), is then proportionally distributed between 0 and n to set as the offset value. In other words, by directly using the average brightness of the reference pixels as the offset value, the brightness of the pixel of interest, P0, can be aligned with the brightness of the surrounding pixels.

[0123] Then, as shown below, (non-count value×offset value / n) is added to both true count values ​​and output.

[0124] Output = actual count value of both sides + (non-count value × offset value / n)

[0125] Figure 8 This is a hypothetical tone map based on the above-mentioned processing. In this implementation example, by directly setting the average brightness of the reference pixel as the offset value, a tone map that matches the surrounding brightness can be automatically generated.

[0126] The above-mentioned image processing eliminates areas where the brightness is extremely prominent compared to the surrounding area, and eliminates the overall darkening of the image. However, the overall contrast of the image may be insufficient. Figure 9 An example of installation in which this is eliminated is shown.

[0127] In this installation example, the number of reference pixels that satisfy both the conditions of being below the pixel value of the focus pixel P0 and below the inclination limit value is set as the actual count value of both parties, and then the number of reference pixels (non-count value) of the histogram with a larger inclination limit value is counted.

[0128] Then, the intensity value of contrast is set as an external parameter having a value of 0 to n, and the brightness of the pixel of interest P0 is output by multiplying the two real count values ​​by {n / (n-non-count value×intensity value / n)}.

[0129] Output = actual count value of both parties × n / (n - non-count value × intensity value / n)

[0130] (n: reference pixel number, such as 128 or 256)

[0131] (Intensity value is 0~n)

[0132] Figure 10 This is a hypothetical tone mapping diagram of the image obtained by the above-mentioned processing. In the above-mentioned installation example, the end point value (maximum brightness) reduced by the slope limit is uniformly raised by offsetting, but in this installation example, a multiplication process is performed, so the slope also changes.

[0133] Figure 11 This example shows an implementation that simultaneously utilizes both the offset and contrast functions. In this implementation, the number of reference pixels that satisfy both the pixel value of the pixel of interest P0 and the tilt limit value is used as the true count value for both. In parallel with the above, the reference pixel values ​​below the preset tilt limit value are simply added, and the average value is calculated by dividing by the number of reference pixels, which is then used as the offset value.

[0134] Then, the true count values ​​of both are multiplied by {n / (n-uncounted value × intensity value / n)}, and then {uncounted value × (n-intensity value) / n × offset value / n} is added to the output. In other words, the output is {n / (n-uncounted value × intensity value / n)} + {uncounted value × (n-intensity value) / n × offset value / n}.

[0135] (n: reference pixel number, such as 128 or 256)

[0136] (Offset value and intensity value are 0 to n)

[0137] The above processing eliminates areas with extreme brightness that differ from surrounding areas (such as noise) and corrects overall image contrast. However, this processing tends to bias the image toward a specific brightness in areas where a majority of pixels have the same value. For example, in a dark image, if pixel values ​​tend toward 0, the image may appear washed out.

[0138] Figure 13 Is further improvement Figure 3 In the figure of the portion shown, in this installation example, two comparators for comparing reference pixel values ​​and focus pixel values ​​are prepared, and the number of reference pixels having a brightness smaller than the focus pixel value and being below the tilt limit value, and the number of reference pixels having a brightness equal to the focus pixel value and being below the tilt limit value are counted separately.

[0139] To the count value of luminance smaller than the pixel value of interest obtained in the previous item, a proportion of the count value of luminance equal to the pixel value of interest is added according to the luminance of interest.

[0140] As a result, if Figure 14 As shown, the end point (maximum brightness) is fixed, and a tone mapping diagram can be obtained in which the starting point is moved to the origin (0, 0).

[0141] The second invention is as follows Figure 15 As shown, the input image is color-decomposed to extract the color scheme (CbCr), luminance (Y), and the three primary colors (RGB). In the present invention, the luminance (Y) is blurred and flat-field corrected, and then combined with the color scheme (CbCr) and the three primary colors (RGB) to create the output image. Note that this invention does not create a flat-field frame (an image with uniform brightness); it simply utilizes the principle of a flat-field frame.

[0142] In the Blur process, an image with a general change in brightness is an image with blurred brightness. In this embodiment, a Gaussian Blur process is performed to blur the image using a Gaussian function.

[0143] Gaussian Blur processing is performed, for example Figure 16 The horizontal Blur process shown in (a) is performed on the line buffer after the horizontal Blur process is completed. Figure 16 The vertical Blur process shown in (b) is used to create a blurred plane.

[0144] During the horizontal blur process, the minimum value of the input image (Ymin) is obtained by averaging the minimum value of the input image over the past four frames.

[0145] In addition, Gaussian Blur processing is performed while obtaining the minimum value (Bmin) and maximum value (Bmax) of the Blur image.

[0146] When implementing the logic of the present invention, a Gaussian table with radius R is used, and Gaussian calculation is not performed. In addition, the radius R is set to a maximum of 30 pixels, and 62 line buffers are used (30×2 + 1 for the center + 1 for horizontal calculation).

[0147] In the example shown in the figure, since the kernel size is 61×61, the Gaussian Blur process is separated into horizontal Blur and vertical Blur to reduce the processing area. However, the two processes may be performed simply as a kernel filter without separation.

[0148] Once the Blur plane is created by the Gaussian Blur process described above, the Blur plane is corrected. The Blur plane mentioned here is not a flat frame, but a line buffer that lists Blur values.

[0149] First, the Blur plane is normalized to set distribution information with values ​​between 0 and 1.0. A threshold is further set between the normalized values ​​of 0 and 1.0. All pixels with values ​​greater than this threshold are set to 1.0. For dark pixels with values ​​less than the threshold, the brightness multiplier (n) of the darkest pixel is determined, and the distribution information is corrected so that the reciprocal (1 / n) is the minimum value of the distribution information.

[0150] For example, if 0.5 is set as the threshold value, among the normalized values ​​0 to 1.0, 0 to 0.5 becomes 0 to 1.0, and all values ​​greater than 0.5 become 1.0.

[0151] By performing this correction, pixels in the input image with a brightness of 0.5 or higher are converted to 1.0 in the distribution information. This ensures that the original image remains unchanged during subsequent processing (dividing the input image by the brightness distribution information as the denominator). On the other hand, since the denominator gradually decreases for darker areas, correction is performed so that the darker the original image, the higher the magnification.

[0152] Following Blur plane correction (distribution information correction), flattening is performed. This flattening process processes the current frame based on the Blur plane and the input image minimum value (Ymin). Specifically, the input image's luminance (Y) and three primary colors (RGB) are divided by the Blur plane.

[0153] In the above, at the time when the vertical Blur value of the target pixel is determined, the processing from the creation of the Blur plane to the acquisition and correction of the distribution information can be performed. Therefore, real-time processing can be performed without using a frame buffer and with only a delay equivalent to the line buffer.

[0154] Flat field processing includes normal processing and color burst processing. In normal processing, only luminance (Y) is processed and combined with the color combination (CbCr) before output. In color burst processing, instead of luminance (Y), the same calculation is performed on the three primary color (RGB) planes.

[0155] The general calculation formula for the flat field processing is as follows.

[0156] Output image F(x, y) = input image Y(x, y) * 256 / flat field frame Blur(x, y)

[0157] 256 is the case when the bit depth is 8 bits.

[0158] When the equation for blur plane correction (distribution information correction) is applied to the above equation, the output image becomes as shown below.

[0159] Output image F(x,y)={(Y(x,y)-Y(min)>0?(Y(x,y)-Y(min):0)*256 / {Blur(x,y)*(255-Bmin) / Bmax+Bmin<255?Blur(x,y)*(255-Bmin) / Bmax+Bmin:255}

[0160] Figure 17 (a) is the original image, showing underexposure in dark areas. (b) is the image after Gaussian Blur processing, resulting in an overall blurred image with blurred brightness. (c) is the image after flat field processing, showing the underexposure in (a) clearly visible.

[0161] The third invention is as follows Figure 18 As shown, the input image is simultaneously subjected to underexposure processing (FC: flat-field corrector) and sharpening processing. The underexposure processed image and the sharpening processed image are synthesized using the flat-field frame principle used in the underexposure processing and output.

[0162] In the dark underexposure processing, Figure 18 As shown, the input image is color-decomposed to extract the color scheme (CbCr), luminance (Y), and the three primary colors (RGB). In the present invention, the luminance (Y) is blurred and flattened, and then combined with the color scheme (CbCr) and the three primary colors (RGB) to create the output image. Note that this invention does not create a flat-field frame (an image with uniform brightness); rather, it simply utilizes the principle of a flat-field frame.

[0163] In the Blur process, a brightness-blurred image is created, which is a generally fluctuating image of brightness. In this embodiment, a Gaussian Blur process is performed to blur the image using a Gaussian function.

[0164] Gaussian Blur processing is performed, for example Figure 19 The horizontal Blur process shown in (a) is performed on the line buffer after the horizontal Blur process is completed. Figure 19 The vertical Blur process shown in (b) is used to create a blurred plane.

[0165] During the horizontal blur process, the minimum value of the input image (Ymin) is obtained by averaging the minimum values ​​of the input image over the past four frames.

[0166] In addition, the minimum value (Bmin) and maximum value (Bmax) of the Blur image are obtained while performing Gaussian Blur processing.

[0167] When implementing the logic of the present invention, a Gaussian table for radius R is used, and Gaussian calculation is not performed. Furthermore, the radius R is set to a maximum of 30 pixels, and 62 line buffers are used (30×2 + 1 for the center + 1 for horizontal calculation).

[0168] In the example shown in the figure, since the kernel size is 61×61, the Gaussian Blur process is separated into horizontal Blur and vertical Blur to reduce the processing area. However, the two processes may be performed simply as a kernel filter without separation.

[0169] Once the Blur plane is created through the aforementioned Gaussian Blur process, a flat-field frame is generated. This is generated based on the Blur plane and the minimum value (Ymin) of the input image. As mentioned above, the present invention does not actually create a flat-field frame; instead, it simply utilizes the principle of the flat-field frame.

[0170] When generating a flat-field frame, the Blur plane is normalized to create distribution information with values ​​between 0 and 1.0. Furthermore, a threshold is set between the normalized values ​​of 0 and 1.0. All pixels with values ​​greater than this threshold are set to 1.0. For dark pixels with values ​​less than the threshold, the brightness multiplier (n) for the darkest pixel is determined, and the distribution information is corrected so that the reciprocal of this multiplier (1 / n) is the minimum value of the distribution information.

[0171] For example, if 0.5 is set as the threshold value, among the normalized values ​​0 to 1.0, 0 to 0.5 becomes 0 to 1.0, and all values ​​greater than 0.5 become 1.0.

[0172] By performing this correction, pixels in the input image with a brightness of 0.5 or higher are assigned a value of 1.0 in the distribution information, and the original image remains unchanged in subsequent processing (dividing the input image by the brightness distribution information as the denominator). On the other hand, since the denominator gradually decreases for darker areas, correction is performed so that the darker the original image, the higher the magnification.

[0173] Following the flat frame generation, flattening processing is performed. This flattening process processes the current frame based on the Blur plane and the input image minimum value (Ymin). Specifically, the input image's luminance (Y) and three primary colors (RGB) are divided by the Blur plane.

[0174] In the above, at the time when the vertical Blur value of the target pixel is determined, the processing from the creation of the Blur plane to the acquisition and correction of the distribution information can be performed, so real-time processing can be performed without using a frame buffer and with only a delay equivalent to the line buffer.

[0175] Flat field processing includes normal processing and color burst processing. In normal processing, only luminance (Y) is processed and combined with the color combination (CbCr) before output. In color burst processing, instead of luminance (Y), the same calculation is performed on the three primary color (RGB) planes.

[0176] The general calculation formula for the flat field processing is as follows.

[0177] Output image F(x, y) = input image Y(x, y) * 256 / flat field frame Blur(x, y)

[0178] 256 is the case when the bit depth is 8 bits.

[0179] When the equation for blur plane correction (distribution information correction) is applied to the above equation, the output image becomes as shown below.

[0180] Output image F(x,y)={(Y(x,y)-Y(min)>0?(Y(x,y)-Y(min):0)*256 / {Blur(x,y)*(255-Bmin) / Bmax+Bmin<255?Blur(x,y)*(255-Bmin) / Bmax+Bmin:255}

[0181] Next, based on Figure 20 The clarification process will be explained later. Figure 20 In the example, P0 is the pixel of interest, and the brightness (luminance) of the pixel of interest P0 is adjusted. Figure 21 It will be based on Figure 20 The reference pixel when the method is processed ( Figure 20 The execution results obtained are 8 (P1 to P8) and a virtual tone mapping diagram is graphed using all the brightness that the reference pixels can have.

[0182] The present invention obtains a unique transformation result for the pixel of interest based on the pixel of interest and the reference pixel, rather than outputting a tone mapping image.

[0183] In the processing steps, first compare the brightness of the focus pixel P0 and the reference pixels P1 to P8 around it, count the number of reference pixels with brightness smaller than the brightness of the focus pixel P0, and correct the brightness of the focus pixel P0 according to the given algorithm based on the count value.

[0184] For example, if the number of reference pixels with a brightness lower than that of the pixel of interest P0 is one and the number of reference pixels with a brightness higher than that of P0 is seven, the maximum brightness that can be output is set to 1, and the reference pixel values ​​are corrected to 1 / 8 of the brightness. The correction algorithm is not limited to this.

[0185] When the above operation is implemented in an FPGA or CPU, the pixels of interest are moved one by one in the row direction, and this processing is performed in parallel for each row, thereby correcting the luminance of all pixels and smoothing the brightness.

[0186] Furthermore, when implemented on a GPU, since these operations are performed independently for each pixel, the brightness of all pixels is corrected and brightness is smoothed by performing parallel processing simultaneously using multiple cores.

[0187] on the other hand, Figure 21 The hypothetical tone map shown is not actually generated algorithmically, but rather illustrates the results of the execution of the logic for all input luminances for ease of understanding.

[0188] There is room for improvement in the above installation example. Figure 21 In the virtual tone mapping diagram, there are three locations where the output increases sharply relative to the input, specifically, where the slope is 45 degrees or more. These locations are where the brightness, such as noise, becomes prominent and large.

[0189] Figure 22 This is a diagram showing an example of installation in which the above-mentioned improvement is applied. Figure 23 is with Figure 22 The corresponding hypothetical tone map, Figure 23 The dotted line is the execution Figure 18 A hypothetical tone map for the logical case.

[0190] In order to eliminate such a portion with prominent brightness, the pixel values ​​(luminance) of the reference pixels P1 to P8 are sequentially compared with the pixel value of the focus pixel P0 to determine whether the pixel value is equal to or lower than the pixel value of the focus pixel P0.

[0191] In parallel with this, the histogram value +1 of the reference pixel is compared with the tilt limit value (45°) to determine whether it is equal to or less than the tilt limit value.

[0192] Then, the number of reference pixels for which both of the above two judgments are satisfied is counted as both true count values, and the brightness of the pixel of interest is output based on the both true count values.

[0193] Through the above processing, such as Figure 23 As shown by the solid line, an easily visible image can be obtained without any portion where the brightness changes significantly.

[0194] The slope limit is taken into account by comparing the pixel values ​​below the focus pixel value and the histogram value +1 of each reference pixel with a preset slope limit value and counting the number of reference pixels below the slope limit value as the true count value of both.

[0195] On the other hand, in this state, the entire image may become dark. Figure 24 and Figure 25 A hypothetical tone map of φ shows the structure that would correct this.

[0196] exist Figure 24 In the installation example shown, the number of reference pixels that satisfies both the conditions of being less than the pixel value of the focus pixel P0 and less than the inclination limit value is set as the actual count value of both parties, and then the number of reference pixels (non-count value) of the histogram with a larger inclination limit value is counted, and the brightness offset value is set as an external parameter with a value of 0 to n, and (non-count value × offset value / n) is added to the actual count value of both parties and output.

[0197] Here, the offset value is determined based on how much the brightness (a+b) of the end point value reduced by the gradient limit is raised.

[0198] Output = actual count value of both sides + (non-count value × offset value / n)

[0199] (n: reference pixel number, such as 128 or 256)

[0200] (Offset value is 0 to n)

[0201] Through the above, such as Figure 25 As shown, although there is no change in the over-sparse tone mapping characteristics of the image, the entire image becomes brighter only by the amount of the shift.

[0202] Figure 26 This is an example of an installation example in which the offset amount is automatically calculated for each area. In this installation example, the read reference pixel value and the focus pixel value are compared, the histogram value of the reference pixel is increased by 1, and compared with a pre-set inclination limit value. If the histogram value of the reference pixel is larger than the inclination limit value, it is counted as a non-count value.

[0203] In parallel with the above, reference pixel values ​​below the preset tilt limit are simply added, divided by the number of reference pixels to calculate the average value, and the resulting value, divided by the maximum brightness of the pixel (256 for 8-bit), is proportionally distributed between 0 and n and used as the offset value. In other words, by directly using the average brightness of the reference pixels as the offset value, the brightness of the pixel of interest P0 can be aligned with the brightness of the surrounding area.

[0204] Then, as shown below, (non-count value × offset value / n) is added to both true count values ​​and output.

[0205] Output = actual count value of both sides + (non-count value × offset value / n)

[0206] Figure 27 This is a hypothetical tone map based on the above-mentioned processing. In this implementation example, by directly setting the average brightness of the reference pixel as the offset value, a tone map that matches the surrounding brightness can be automatically generated.

[0207] Through the above-mentioned image processing, the extremely bright part that stands out from the surrounding area is eliminated, and the overall image does not become darker, but sometimes the overall image contrast is insufficient. Figure 28 An example of installation in which this is eliminated is shown.

[0208] In this installation example, the number of reference pixels that satisfy both the conditions of being below the pixel value of the focus pixel P0 and below the inclination limit value is set as the actual count value of both parties, and then the number of reference pixels (non-count value) of the histogram with a larger inclination limit value is counted.

[0209] Then, the intensity value of contrast is set as an external parameter having a value of 0 to n, and the brightness of the pixel of interest P0 is output by multiplying the two true count values ​​by {n / (n-non-count value×intensity value / n)}.

[0210] Output = actual count value of both parties × n / (n - non-count value × intensity value / n)

[0211] (n: reference pixel number, such as 128 or 256)

[0212] (Intensity value is 0~n)

[0213] Figure 29 This is a hypothetical tone mapping diagram of the image obtained by the above-mentioned processing. In the above-mentioned installation example, the end point value (maximum brightness) reduced by the slope limit is uniformly raised by offsetting, but in this installation example, the slope also changes due to the multiplication process.

[0214] Figure 30This example shows an implementation that simultaneously utilizes both the offset and contrast functions. In this implementation, the number of reference pixels that satisfy both the pixel value of the pixel of interest P0 and the tilt limit value is used as the true count value for both. In parallel with the above, the reference pixel values ​​below the preset tilt limit value are simply added, and the average value is calculated by dividing by the number of reference pixels, which is then used as the offset value.

[0215] Then, the actual count values ​​of both are multiplied by {n / (n-uncounted value × intensity value / n)}, and {uncounted value × (n-intensity value) / n × offset value / n} is added and output. In other words, output = {n / (n-uncounted value × intensity value / n)} + {uncounted value × (n-intensity value) / n × offset value / n}.

[0216] (n: reference pixel number, such as 128 or 256)

[0217] (Offset value and intensity value are 0 to n)

[0218] The above processing eliminates areas with extreme brightness that differ from surrounding areas (such as noise) and corrects overall image contrast. However, this processing can bias the image toward a specific brightness in areas where a majority of pixels have the same value. For example, if pixel values ​​are biased in a dark image, the image may appear washed out.

[0219] Figure 31 Is further improvement Figure 22 In the figure of the portion shown, in this installation example, two comparators are prepared for comparing reference pixel values ​​and focus pixel values, and the number of reference pixels having a brightness smaller than the focus pixel value and being below the tilt limit value and the number of reference pixels having a brightness equal to the focus pixel value and being below the tilt limit value are counted separately.

[0220] To the count value of luminance smaller than the pixel value of interest obtained in the previous item, the count value of luminance equal to the pixel value of interest is proportionally added, corresponding to the luminance of interest.

[0221] As a result, if Figure 33 As shown, the end point (maximum brightness) is fixed, and a tone map is obtained in which the starting point is moved to the origin (0, 0).

[0222] As described above, once the dark underexposure-corrected image (FC output image) and the sharpened output image are obtained, these images are synthesized using the principle of flat-field frames. In the present invention, during synthesis, the dark underexposure-corrected image and the sharpened output image are not simply mixed. Instead, the dark underexposure-corrected image is allocated more heavily to dark areas, while the sharpened output image is allocated more heavily to bright areas.

[0223] When the above is implemented using a CPU, calculations are performed between frames after the flat field frame is actually generated.

[0224] On the other hand, when performing real-time processing in a circuit such as an FPGA, a flat-field frame corresponding to a single tensor is not actually created in the frame buffer. Instead, a line buffer is used, and a ring buffer corresponding to the blur diameter is used (a structure that reuses as many as necessary). In other words, rather than creating a flat-field frame corresponding to a single screen, a horizontally long, thin flat-field frame is generated for each scanned line, in synchronization with the screen scan.

[0225] The use of the line buffer for real-time processing of the blur diameter of the flat-field frame is similar to the flat-field frame used for dark underexposure correction. In the case of dark underexposure correction, the flat-field frame is created by Gaussian blurring the luminance information and then performing level correction. However, the level correction values ​​used for image composition and dark underexposure correction may differ. Therefore, only the luminance is blurred using Gaussian in the line buffer, and different level corrections are performed for dark underexposure correction and image composition. Furthermore, since level correction is performed using addition and multiplication, it can be calculated in real time just before output.

[0226] Figure 34 (a) is the original image, showing underexposure in dark areas and unclear areas. (b) is the image after underexposure processing. While the underexposure has been eliminated, the contrast in bright areas is insufficient, and the bright areas of the wires are overexposed. (c) is the image after sharpening processing, showing excessive contrast in dark areas and a chaotic grain in the backlit areas. (d) is the image after image synthesis (the present invention), eliminating the issues with both underexposure and sharpening processing. The wires are clearly visible, and the grain in the backlit areas is uniform.

Claims

1. An image processing method, characterized in that: When processing the input image data in units of pixels, n reference pixels are set around a pixel of interest that serves as a scanning position for image scanning, and the pixel value, i.e., brightness, of the pixel of interest is sequentially compared with the pixel value, i.e., brightness, of each reference pixel. The number of reference pixels having a pixel value below the pixel of interest is counted. In parallel, the histogram value of each reference pixel is increased by 1 and compared with a preset inclination limit value. The true count values ​​of both sides are obtained by counting the number of reference pixels that satisfy both the condition that the reference pixel value is below the pixel of interest and the condition that the histogram value of the pixel is below the inclination limit value. The true count values ​​of both sides are proportionally distributed and output, wherein n is an integer. In the processing step, the brightness of the pixel of interest is compared with the brightness of the reference pixels around it, the number of reference pixels having a brightness smaller than the brightness of the pixel of interest is counted, and the brightness of the pixel of interest is corrected according to a given algorithm based on the count value; When the number of reference pixels with brightness smaller than that of the focus pixel is 1 and the number of reference pixels with brightness larger than that of the focus pixel is 7, the maximum brightness that can be output is set to 1 and the value of the reference pixel is corrected to 1 / 8 of the brightness.

2. The image processing method according to claim 1, wherein: The number of reference pixels not counted in the comparison with the tilt limit value is counted as a non-count value. In addition, the brightness offset value is set as an external parameter and substituted into the following formula (1) and output to adjust the brightness of the entire screen. Formula (1) Output = actual count value of both sides + (non-count value × offset value / n) n is the number of reference pixels.

3. The image processing method according to claim 1, wherein: The average value of the reference pixel values ​​is calculated and set as the brightness offset value. When outputting, the adaptive offset value corresponding to the focus pixel is added to automatically and adaptively adjust the brightness of the picture.

4. The image processing method according to claim 2, wherein: The number of reference pixels not counted in the comparison with the slope limit value is counted as a non-count value. In addition, the contrast intensity value is set as an external parameter, substituted into the following formula (2) and output, thereby having an adaptive contrast intensity value. Formula (2) Output = {actual count value of both sides + (non-count value × offset value / n)} × n / (n-non-count value × contrast intensity value / n) n is the number of reference pixels, and the contrast intensity value is 0 to n.

5. The image processing method according to claim 2 or 3, characterized in that: The contrast intensity value is set as an external parameter, and the average value of the reference pixel values ​​is calculated. In addition, the number of reference pixels that are not counted in the comparison with the slope limit value is set as a non-count value and counted. The average value, non-count value, and contrast intensity value are substituted into the following formula (3) to output: Formula (3) Output = Output of claim 2 or 3 × {n / (n-non-count value × contrast intensity value / n)} + {non-count value × (n-contrast intensity value) / n × offset value / n} n is the number of reference pixels, and the contrast intensity value is 0 to n.

6. The image processing method according to claim 1, wherein: The focus pixel value is compared with each reference pixel value, and the number of reference pixels with pixel values ​​equal to the focus pixel value and the number of reference pixels with pixel values ​​smaller than the focus pixel value are counted separately. The true count value of each is obtained by adding the number of pixels of the former to the number of pixels of the latter in proportion to the value of the focus pixel.

7. An image processing method, characterized in that: The Blur value of the brightness blur is obtained by performing Gaussian Blur processing on the brightness of each pixel in the Y plane of the input image, that is, the brightness plane memory, and normalizing the Blur value to set it as distribution information with a value of 0 to 1.

0. Then, a threshold is set between the normalized values ​​0 to 1.0, and all pixels with values ​​greater than the threshold are set to 1.

0. For dark pixels with values ​​smaller than the threshold, the brightness multiplier n of the darkest pixel is determined, and the distribution information is corrected so that its inverse 1 / n is set to the lowest value of the distribution information. The brightness of the input image is divided by the corrected dark brightness distribution information 1 / n to 1.0 as the denominator.

8. The image processing method according to claim 7, wherein: The calculation result of the Blur value is cached only in the line buffer and processed in parallel with the correction of the distribution information.

9. An image processing method, characterized in that: Image processing combines the image that has been sharpened and the image that has been underexposed in the dark areas. The dark underexposure processing is performed by performing Gaussian Blur processing on the brightness of each pixel in the Y plane of the input image, that is, the brightness plane memory, to obtain the Blur value of the brightness blur, and normalizing the Blur value to set it as distribution information with a value of 0 to 1.

0. A threshold is further set between the normalized values ​​0 to 1.0, and all pixels with values ​​greater than the threshold are set to 1.

0. For dark pixels with values ​​smaller than the threshold, the brightness multiplier n of the darkest pixel is determined, and the distribution information is corrected so that its inverse 1 / n is set to the lowest value of the distribution information. The brightness of the input image is divided by the corrected dark brightness distribution information 1 / n to 1.0 as the denominator.

10. The image processing method according to claim 9, wherein: When the sharpening process processes the input image data in units of pixels, n reference pixels are set around the pixel of interest that becomes the scanning position of the image scan, the pixel value of the pixel of interest, i.e., the brightness, and the pixel value of each reference pixel, i.e., the brightness, are compared in sequence, the number of reference pixels having a pixel value below the pixel of interest is counted, and in parallel, the histogram value of each reference pixel is increased by 1 and compared with a preset inclination limit value, and the true count values ​​of both sides are obtained by counting the number of reference pixels that satisfy both the condition that the reference pixel value is below the pixel of interest and the condition that the histogram value of the pixel is below the inclination limit value, and the true count values ​​of both sides are distributed and output in proportion, wherein n is an integer. In the processing step, the brightness of the pixel of interest is compared with the brightness of the reference pixels around it, the number of reference pixels having a brightness smaller than the brightness of the pixel of interest is counted, and the brightness of the pixel of interest is corrected according to a given algorithm based on the count value; When the number of reference pixels with brightness smaller than that of the focus pixel is 1 and the number of reference pixels with brightness larger than that of the focus pixel is 7, the maximum brightness that can be output is set to 1 and the value of the reference pixel is corrected to 1 / 8 of the brightness.

Images