Method, data processing system, computer program product, and computer readable medium for determining image sharpness - Patents.com

Abstract

The present invention relates to a method for determining whether an input image is sufficiently sharp, the method comprising the steps of providing a blur score threshold, inputting the input image into an image processing system, generating a two-dimensional frequency spectrum of the input image by the image processing system, generating a one-dimensional frequency spectrum (14) from the two-dimensional frequency spectrum, fitting a straight line (16) to the one-dimensional frequency spectrum (14), determining a blur score value based on a residual of the fitting, and considering the input image to be sufficiently sharp based on a comparison of the blur score value to the blur score threshold. The present invention further relates to a data processing system, a computer program product, and a computer readable medium for implementing the method.

Description

[Technical field]

[0001] The present invention relates to a method for determining whether an input image is sufficiently sharp, for example to ensure that the input image contains sufficient detail or information for a planned use or further image processing, and further relates to a data processing system, a computer program product, and a computer readable medium for implementing the method. [Background technology]

[0002] Measuring or determining the clarity of an image is used in various applications. Applications such as medical applications or those related to autonomous driving usually require a higher level of clarity, since an insufficiently clear image can lead to erroneous decisions and cause health risks or accidents. A blurred image, i.e. an image lacking clarity, usually has, for example, poorly defined edges and poor distinction between light and shadow areas. Possible causes of blurred images are summarized in US Pat. No. 5,399,433, including camera out-of-focus, incorrect focus on the foreground or background, best focus set at a certain distance and therefore out-of-focus on objects at other distances, camera or object movement during image capture (motion blur), or digitization or post-processing of the image. Furthermore, blurred images can make edge detection or segmentation tasks more difficult to perform, since blur increases the uncertainty of the results.

[0003] Patent Document 2 discloses a blur detection system for estimating and reducing image blur of a digital image. A digital image is usually encoded and stored by forward discrete cosine transform (DCT). A single DCT coefficient, a set of DCT coefficients, or a comparison of one or more DCT coefficients with other DCT coefficients is used as a blur index. Each DCT coefficient is associated with its corresponding frequency component, thus generating a two-dimensional histogram. The two-dimensional (2D) histogram is then reduced to a one-dimensional (1D) histogram, similar to a radial frequency spectrum. The blur index is determined from the 1D histogram, for example, by using the maximum DCT coefficient in one region of the image, or by using the ratio of the standard deviation of the first region to the standard deviation of another region, a large ratio representing a blurred image and a small ratio representing a focused image.

[0004] US Patent No. 5,399,633 discloses a method and apparatus for measuring the quality of a compressed video sequence without the use of a reference, whereby quality information is derived directly from the individual image frames. The image frames are transformed by a Fast Fourier Transform (FFT) and the ratio between the accumulated mid- to high-frequency amplitudes and the accumulated low-frequency amplitudes is determined. This ratio is used to judge the sharpness of the image, since a smaller ratio indicates that there is more low-frequency content in the image and therefore the image may appear blurred.

[0005] US Patent No. 5,999,333 discloses a system for estimating blur values ​​based on edges detected in an input image and spectral energy information of the input image in the frequency domain. The image is transformed from the spatial domain to the frequency domain, and the frequency components are analyzed based on a model for classifying blurred and unblurred (sharp) images. The 2D power spectrum of the input image is calculated, from which a direction-independent 1D spectrum is calculated. The 2D spectrum is divided into circular regions corresponding to low, mid, and high frequency regions, and the blur values ​​are determined based on the counts in each region. Alternatively, edge detection is used to determine the blur values.

[0006] US Patent No. 5,399,633 discloses a method for measuring the sharpness of an image, in which the judgment of whether an image is sharp or not is based on a 2D spectrum generated by a discrete Fourier transform. Directional information is also used to judge the sharpness of the image.

[0007] Non-Patent Document 1 discloses a method for estimating image sharpness of a face image. Non-Patent Document 2 discloses a method for evaluating image sharpness suitable for correcting image blur caused by shaking of a handheld camera.

[0008] In view of known techniques, a method is needed that helps enable a decision on the sharpness of an image, regardless of the content or objects of the image. Known techniques that use edge detection to complement the decision of whether an image is sharp enough depend heavily on the content of the image, which may affect the decision of sharpness. If equally sharp images capture different scenes, for example, one scene contains only a few objects and the other scene contains more objects, the result will be a different amount of edges detected, and therefore a different decision based on the detected edges. For this reason, methods based on edge detection cannot be directly used for unknown or dynamically changing scenes. Furthermore, a reliable method is needed that enables a decision of sharpness in real time for dynamically changing scenes.

[0009] Some other known techniques use machine learning tools to determine whether an image is sufficiently clear. The use of machine learning tools requires a significant amount of training data and also requires individual classification of the training data, which requires significant time and effort. For the above reasons, there is a strong need for a solution that provides an accurate and objective determination more easily and quickly. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] Canadian Patent Application Publication No. 2 562 480 [Patent Document 2] US Patent Application Publication No. 2004 / 0120598 [Patent Document 3] US Patent Application Publication No. 2004 / 0156559 [Patent Document 4] US Patent Application Publication No. 2014 / 0003734 [Patent Document 5] Korean Patent No. 10-1570602 [Non-patent literature]

[0011] [Non-Patent Document 1] P.Minin et al. “Sharpness estimation in facial images by spectrum approximation”, Signal, Image and Video Processing, vol. 11, no. 1, pages 163-170 (5 July 2016) [Non-Patent Document 2] Qingbo Lu et al. “A No-Reference Image Sharpness Metric Based on Structural Information Using Sparse Representation”, Information Sciences, Elsevier, vol. 369, pages 334-346 (27 June 2016) Summary of the Invention

[0012] The main object of the present invention is to provide a method for determining whether an image is sharp or not, which method avoids as far as possible the drawbacks of the prior art approaches.

[0013] It is an object of the present invention to provide a method by which a decision regarding the sharpness of an image can be made in a more efficient manner than prior art approaches, regardless of the content of the image. It is therefore an object of the present invention to provide a reliable method by which it can be determined whether an image is sufficiently sharp or not.

[0014] It is a further object of the invention to provide a data processing system comprising means for carrying out the steps of the method according to the invention.

[0015] It is further an object of the present invention to provide a non-transitory computer program product for carrying out the steps of the method according to the present invention on one or more computers, and a non-transitory computer-readable medium comprising instructions for executing the steps of the method on one or more computers.

[0016] The object of the invention can be achieved by a method according to claim 1. Furthermore, the object of the invention can be achieved by a data processing system according to claim 14, a non-transitory computer program product according to claim 15 and a non-transitory computer readable medium according to claim 16. Preferred embodiments of the invention are defined in the dependent claims.

[0017] The main advantage of the method according to the invention compared to the prior art approaches comes from the fact that it is possible to determine in a fast and easy way whether an image is sufficiently clear, thus also allowing a real-time determination, which on the one hand allows a continuous analysis of the images coming from the camera stream, and on the other hand also allows a readjustment of the camera or its settings in order to capture a sufficiently clear image.

[0018] It has been recognized that the one-dimensional frequency spectrum generated from the two-dimensional frequency spectrum of an image, and in particular the shape or envelope of the one-dimensional frequency spectrum, corresponds to the sharpness of the image. It has been found that a sufficiently sharp image results in a one-dimensional frequency spectrum with a smooth envelope, while a blurred image tends to result in a one-dimensional frequency spectrum with a less smooth envelope, i.e. the envelope contains some structure. For this reason, when fitting a line to the one-dimensional frequency spectrum, a sharp image has a relatively good fit, in contrast to a blurred image, which has a less good fit. The residual of the fit can be used as a parameter that can characterize the quality of the fit and thus can be used as a basis for judging the sharpness of the image. This parameter can be compared, for example, to a threshold or a set of thresholds that correspond to a measure of sharpness.

[0019] Certain embodiments of the method according to the invention are able to generate such a threshold based on the image itself.

[0020] A further advantage of the method according to the present invention is that it does not require much calculation, and therefore allows real-time judgment of image sharpness.Due to the limited calculation requirement, the method according to the present invention can be executed twice for each input image, once for the original input image and once for the blurred version (blurred image) of the input image, and still be able to make a judgment in real time.Advantageously, the method according to the present invention is image content independent, i.e. scene independent, and therefore the method can be used for images capturing any kind of scene.

[0021] The method according to the invention can therefore be used in any vision-based scene recognition system, including medical applications (medical imaging) or improving vision in self-driving or autonomous vehicles. [Brief description of the drawings]

[0022] Preferred embodiments of the invention are described below, by way of example, with reference to the following drawings:

[0023] [Figure 1] 1 is an example of an input image. [Diagram 2] 2 is an example of a blurred image generated from the input image of FIG. 1. [Figure 3A] FIG. 2 is an enlarged detail of FIG. 1. [Figure 3B] FIG. 3 is an enlarged detail of FIG. [Figure 4] 2 is an exemplary two-dimensional spectrum of the input image of FIG. 1. [Diagram 5] 3 is an exemplary two-dimensional spectrum of the blurred image of FIG. 2. [Figure 6] 5 is an exemplary one-dimensional spectrum (histogram) generated from FIG. 4 and a line fitted to the histogram. [Figure 7] 6 is an exemplary one-dimensional spectrum (histogram) generated from FIG. 5 and a line fitted to the histogram. [Figure 8] 2 shows the progression of the blur score value as a function of a blur parameter representing the degree of blur in an input image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] The present invention relates to a method for determining whether an input image 10 is sufficiently clear. The level of sufficient clarity may depend on the specific application of the method, for example, medical applications or applications related to autonomous driving may require a higher level of clarity, since the clarity of the image may directly affect the decision based on the input image 10, and therefore a health risk or accident may occur if the image is not clear enough. Based on the decision, an input image 10 with insufficient clarity can be labeled (flagged) as such, or even ignored for the specific application, or a warning signal can be generated. Furthermore, the level of sufficient clarity can be set independently for each application.

[0025] According to the present invention, a method for determining whether an input image 10 is sufficiently sharp comprises the step of providing a blur score threshold for the input image 10. The blur score threshold is preferably a value that distinguishes between an input image 10 being sufficiently sharp or being insufficiently sharp for the intended use. The step of providing a blur score threshold preferably comprises a decision made based on empirical data between a first scenario and a second scenario. In the first scenario, the blur score threshold is a pre-determined value, and in the second scenario, the blur score threshold is determined by a further step based on the input image 10 itself.

[0026] In the first scenario, the blur score threshold is a predetermined value, preferably determined based on empirical data such as the camera type or settings of the camera recording the input image 10, the lighting conditions when the input image 10 was recorded, the content or expected content of the input image 10, etc. In the second scenario, the blur score threshold may be determined directly based on the input image 10 itself.

[0027] The first scenario preferably corresponds to applications where only a limited amount of blur is expected for the input image 10 (i.e. the blur of the input image 10 corresponds to a blur parameter range up to 0.95 in FIG. 8). Thus, in the first scenario, an empirically determined (predetermined) blur score threshold can be used. The first scenario relates, for example, to the case where the input image 10 is recorded by a camera of an autonomous vehicle during the day in good weather conditions, and thus the input images 10 recorded by the camera are expected to have similar sharpness. In such a scenario, an empirically predetermined blur score threshold can be provided for the input image 10, preferably based on the camera type and camera settings. In a preferred embodiment, in the first scenario, the same blur score threshold can be provided for all input images 10.

[0028] Preferably, the pre-determined blur score threshold is determined based on the gain of the camera relative to the noise of the input images 10. In a preferred embodiment, a set of pre-determined blur score thresholds for different camera settings or imaging conditions is provided, and an appropriate blur score threshold is selected for each input image 10 or for each set of input images 10. By using the pre-determined blur score threshold, input images 10 having blur corresponding to a blur parameter in the range of 0.3 or 0.4 (i.e., very slight blur that cannot be seen by the human eye) are preferably classified as sufficiently sharp. However, if even such input images 10 should be distinguished by the method according to the invention, the second scenario can be selected.

[0029] The second scenario preferably corresponds to applications where the blur of the input image 10 may vary in a wider range, and therefore the blur score threshold is preferably determined based on the input image 10 itself. The second scenario relates to cases where the input image 10 may be recorded by a camera of an autonomous vehicle, for example, at night or in poor weather conditions (e.g., rainy weather), or where there is no a priori knowledge of the quality of the input image 10, and therefore the sharpness or blur of the recorded input image 10 may vary in a wider range, and therefore a more robust classification method is required. In such scenarios, the blur score threshold is preferably provided for each input image 10 individually, and based on the input image 10 itself. Thus, the second scenario, in which the blur score threshold is calculated based on the input image 10 itself, can also be used for any kind of input image 10, i.e. for input images 10 recorded in any weather or image conditions, including good and bad conditions.

[0030] Furthermore, the method according to the invention comprises the step of inputting an input image 10 into an image processing system. An exemplary input image 10 is shown in Fig. 1. An enlarged detail 10' of the input image 10 of Fig. 1 is shown in Fig. 3A. The input image 10 may be an image taken by a digital camera or may be a digitized (e.g. scanned) photograph, or even the input image 10 may be a frame of a camera stream, and the input image 10 may be a raw image or a pre-processed image.

[0031] The method according to the invention further comprises the step of generating a two-dimensional frequency spectrum 12 (see FIG. 4 for an example) of the input image 10, the two-dimensional frequency spectrum 12 being generated by an image processing system. The two-dimensional frequency spectrum 12 of the input image 10 is preferably generated by applying a two-dimensional Fourier transform, such as a Fast Fourier Transform, or alternatively by a two-dimensional wavelet transform, such as a Discrete Cosine Transform, on the input image 10.

[0032] Due to the applied two-dimensional Fourier transform, the method according to the invention can also process input images 10 that are at least partially blurred (e.g., when a nearby object that is not the intended content of the input image 10 is captured on the input image 10). In such cases, the remaining parts of the input image 10 can still contain high-, mid-, and low-frequency components, and thus the method according to the invention can determine whether most of the input image 10 is sufficiently sharp. If the largest part of the input image 10 is covered by blurred objects, the method can classify the input image as not being sufficiently sharp and therefore can suggest that no meaningful conclusions should be drawn from this input image 10. Furthermore, this feature fits well with the intended use and application of the method according to the invention, which can avoid unclear and blurred images. If further use of an input image 10 having clear and unclear (blurred) parts should be avoided regardless of the overall sharpness of the input image 10, in such a case the input image 10 can be filtered by other methods, for example by applying edge detection to the input image 10.

[0033] From the two-dimensional frequency spectrum 12, a one-dimensional frequency spectrum 14 is generated as described in detail below, and a straight line 16 is fitted to the one-dimensional frequency spectrum 14 (see FIG. 6 for an example). The straight line 16 is preferably fitted by a linear regression method, and a blur score value is determined based on the residual of the fitting. The residual is preferably the distance of the data points of the one-dimensional frequency spectrum 14 and the fitted straight line 16, the distance being preferably determined by any known metric, for example the distance being the Euclidean distance or the squared Euclidean distance. It has been found that the blur score value is larger in blurred images than in sharp images, and therefore the input image 10 is considered to be sufficiently sharp if the blur score value is below the blur score threshold. The blur score value is independent of the actual scene of the input image 10, and therefore the method according to the invention can be used for continuous sharpness monitoring, i.e. for input images 10 coming from a camera stream, even if the camera stream captures dynamically changing scenes.

[0034] The two-dimensional frequency spectrum 12 typically includes a two-dimensional amplitude spectrum and a two-dimensional phase spectrum, the two of which together uniquely describe the input image 10. Preferably, the one-dimensional frequency spectrum 14 is generated from the two-dimensional amplitude spectrum of the two-dimensional frequency spectrum 12, the two-dimensional amplitude spectrum including the amplitudes of the spatial frequencies. The two-dimensional phase spectrum may preferably be omitted with respect to the determination of the one-dimensional frequency spectrum 14.

[0035] A one-dimensional frequency spectrum 14 can be generated from the two-dimensional frequency spectrum 12 or the two-dimensional amplitude spectrum by applying any two-to-one binary relationship, and the resulting one-dimensional frequency spectrum 14 is ordered by frequency. As an example, the one-dimensional frequency spectrum 14 is generated by frequency mapping or by taking one or more slices of the two-dimensional frequency spectrum 12 or the two-dimensional amplitude spectrum.

[0036] Preferably, the logarithm of the two-dimensional amplitude spectrum is calculated and the logarithm of the two-dimensional amplitude spectrum is used to generate the one-dimensional frequency spectrum 14. By calculating the logarithm of the two-dimensional amplitude spectrum, frequencies with smaller and larger amplitudes are more equally represented in the one-dimensional frequency spectrum 14.

[0037] Alternatively, the one-dimensional frequency spectrum 14 is a histogram having bins corresponding to each frequency range of the two-dimensional frequency spectrum 12 and histogram values ​​for each bin, each histogram value preferably being the average amplitude or integrated amplitude within the corresponding frequency range.

[0038] Preferably, the two-dimensional frequency spectrum 12 is central, i.e. the frequency corresponding to zero frequency is located at the center of the spectrum and the frequencies increase radially outward from the center (similar to Figures 4 and 5). The one-dimensional frequency spectrum 14 in such a case is preferably generated by taking radial averaging or radial slicing of the two-dimensional frequency spectrum 12 or two-dimensional amplitude spectrum.

[0039] Besides determining the blur score threshold based on empirical data (in the first scenario), it is also possible to determine the blur score threshold based on the input image 10 itself (in the second scenario) by the following steps: Preferably, a blurred image 20 (see FIG. 2 for an example) is generated from the input image 10, preferably by Gaussian blurring, where the Gaussian function has a standard deviation in the range of 0.01 to 2. By using an image processing system, a further two-dimensional frequency spectrum 22 (see FIG. 5 for an example) of the blurred image 20 is generated, preferably the further two-dimensional frequency spectrum 22 is generated in the same way as the two-dimensional frequency spectrum 12, i.e. by using the same transformation, i.e. the same two-dimensional Fourier transform or the same two-dimensional wavelet transform. From the further two-dimensional frequency spectrum 22, a further one-dimensional frequency spectrum 24 (see FIG. 7 for an example) is generated, and a further straight line 26 (see FIG. 7 for an example) is fitted to the further one-dimensional frequency spectrum 24. Preferably, the further one-dimensional frequency spectrum 24 is generated in the same way as the one-dimensional frequency spectrum 14, and the further straight line 26 is fitted in the same way as the straight line 16, i.e. by applying the same fitting method, e.g. the same linear regression method. The residual of the fitting of the further straight line 26 is calculated, the residual being the blur score threshold. Again, the residual is preferably calculated with the same metric as the fitting of the straight line 16. By using the same method and parameters to generate the blur score value of the input image 10 and the blur score threshold from the blurred image 20, the blur score values ​​and the blur score threshold can be easily compared.

[0040] The comparison between the blur score value and the blur score threshold may preferably be performed based on a difference score value generated by subtracting the blur score threshold from the blur score value, where a difference score value having a positive or non-negative value indicates that the input image 10 is sufficiently sharp and a difference score value having a negative value indicates that the input image 10 is not sufficiently sharp, i.e., is blurred.

[0041] Another way of comparing the blur score value with the blur score threshold is to generate a score ratio, where the score ratio is generated by dividing the blur score value by the blur score threshold, with a score ratio having a value greater than or equal to 1 indicating that the input image 10 is sufficiently sharp and a score ratio having a value less than 1 indicating that the input image 10 is blurred.

[0042] A detailed example of a method according to the invention is described below in relation to FIGS.

[0043] Fig. 1 shows an input image 10 taken by a camera of a vehicle, preferably an autonomous vehicle. A two-dimensional Fourier transform, in this example a two-dimensional Fast Fourier Transform (FFT), is used to generate a two-dimensional frequency spectrum 12 from the input image 10. The two-dimensional frequency spectrum 12 is shown in Fig. 3, where the two-dimensional frequency spectrum 12 is centered, i.e. the center of the two-dimensional frequency spectrum 12 corresponds to zero frequency, from which the frequency increases radially outward. Sharp details in the input image 10 correspond to higher spectral values ​​at higher frequencies, and blurring of the input image 10 suppresses spectral values ​​at higher frequencies.

[0044] Figure 2 shows a blurred image 20 generated from the input image 10 of Figure 1 by Gaussian blurring, where the standard deviation of the Gaussian blurring is selected to be 0.7. Figures 3A and 3B show enlarged details 10', 20' of Figures 1 and 2, respectively, from which it can be clearly seen that the input image 10 shown in Figures 1 and 3A is sharper than the blurred image 20, i.e. the edges of objects in Figures 2 and 3B are less distinct.

[0045] FIG. 4 is a two-dimensional frequency spectrum 12 generated by an image processing system based on the input image 10 of FIG. 1, and FIG. 5 is a further two-dimensional frequency spectrum 22 generated by the image processing system based on the blurred image 20 of FIG.

[0046] The two-dimensional frequency spectrum 12 and the further two-dimensional frequency spectrum 22 are both centered spectra generated by a two-dimensional FFT transform applied to the input image 10 and the blurred image 20, respectively. From a comparison of Figures 4 and 5 it can be seen that the further two-dimensional frequency spectrum 22 generated from the blurred image 20 contains less high frequency components due to the fact that the high frequency components resulting from sharp details are reduced by the blurring, and therefore the further two-dimensional frequency spectrum 22 decays faster towards higher frequencies (i.e. towards the periphery of the further two-dimensional frequency spectrum 22) compared to the two-dimensional frequency spectrum 12.

[0047] 6 and 7 show examples of a one-dimensional frequency spectrum 14 and a further one-dimensional frequency spectrum 24, respectively. The one-dimensional frequency spectrum 14 according to FIG. 6 and the further one-dimensional frequency spectrum 24 according to FIG. 7 are both histograms generated from the two-dimensional frequency spectrum 12 according to FIG. 4 and the further two-dimensional frequency spectrum 22 according to FIG. 5, respectively. Each histogram has bins, in this case frequency bins, corresponding to a frequency range, preferably a uniform, single-sized frequency range. The frequency bins can be interpreted as concentric rings around the center of the two-dimensional frequency spectrum 12 and the further two-dimensional frequency spectrum 22, and the histograms according to FIG. 6 and FIG. 7 have frequency bins corresponding to concentric rings with a width of one pixel, therefore each histogram according to FIG. 6 and FIG. 7 has 256 frequency bins. The number of frequency bins therefore corresponds to the size of the two-dimensional frequency spectrum 12 and the further two-dimensional frequency spectrum 22, respectively.

[0048] The histogram values ​​of the histograms of Figures 6 and 7 are determined in the following manner: First, a respective two-dimensional amplitude spectrum is generated based on the two-dimensional frequency spectrum 12 and the further two-dimensional frequency spectrum 22, respectively. Second, the logarithm of each two-dimensional amplitude spectrum is calculated, and then the histogram values ​​are generated from the logarithm of each two-dimensional amplitude spectrum by radial averaging, i.e., for each frequency bin, the corresponding histogram value is the average amplitude (intensity) within that frequency range (concentric rings), and thus the histogram values ​​are frequency intensities expressed in logarithms (log frequency intensity).

[0049] In Figures 6 and 7 the frequency bins are indicated by their distance (preferably in pixels) from the centre of the two-dimensional frequency spectrum 12 and the further two-dimensional frequency spectrum 22.

[0050] In Fig. 6, a straight line 16 is fitted to the one-dimensional frequency spectrum 14 (histogram) by way of linear regression, for example by any linear regression method known in the art. A residual characterizing the quality of the fitting is calculated for the fitting, with a smaller residual value indicating a good fitting of the straight line 16 to the one-dimensional frequency spectrum 14 and a larger residual value indicating a less good fitting of the straight line 16 to the one-dimensional frequency spectrum 14. The residual is preferably divided by the size of the input image 10, for example by the number of frequency bins, i.e. generating a blur score value that is independent of the size of the input image 10. If the input image 10 has sharp details, the histogram (one-dimensional frequency spectrum 14) is fairly balanced, i.e. its envelope is flat, thus resulting in a good straight line fitting and the residual and blur score value have small values. For blurred images, the histogram envelope has features that typically reduce the quality of the linear fitting, i.e., resulting in a larger residual and a larger blur score value. A clear correlation has been found between the flatness of the envelope and the sharpness of the input image 10. With respect to FIG. 6, the blur score value calculated based on the residual of the fitting is 105. If the residual is divided by the number of bins (which depends on the size of the input image 10), a normalized blur score value can be generated that is independent of the size of the input image 10 and the number of bins of the histogram. With respect to FIG. 6, the normalized blur score value is 0.41, which is the blur score value divided by the number of frequency bins, and the histogram of FIG. 6 contains 256 frequency bins. Both the blur score value and the normalized blur score value are independent of the actual scene of the input image 10, i.e., the content of the input image 10.

[0051] In Fig. 7, a further straight line 26 is fitted to the further one-dimensional frequency spectrum 24 (histogram) by means of linear regression, and the fitting residual is calculated. In the example shown in Fig. 7, the linear regression method used in fitting the further straight line 26 to the further one-dimensional frequency spectrum 24 is the same as that used in fitting the straight line 16 to the one-dimensional frequency spectrum 14 in Fig. 6. A blur score value can also be calculated for the blurred image 20 of Fig. 2. For Fig. 7, the blur score value is 116, and therefore the normalized blur score value is 0.45, and both the blur score value and the normalized blur score value are calculated in the same way as in Fig. 6. Since the input image 10 and the blurred image 20 have the same size, the histogram of Fig. 7 also has 256 bins.

[0052] Based on a comparison of Figures 6 and 7, it can be seen that the blur score value calculated for the blurrier image (e.g., blurred image 20) is greater than the sharper image (e.g., input image 10), and therefore the blur score value can be used to indicate whether the input image 10 is sufficiently sharp.

[0053] 8 shows the evolution of the blur score value as a function of a blur parameter that represents the degree of blurring of the input image 10. For example, by applying a Gaussian blur with increasing blur parameters to the input image 10, a series of blurred images 20 are generated from the input image 10 by blurring. The blur parameter is preferably the standard deviation of the Gaussian function used in the Gaussian blur.

[0054] Surprisingly, the blur score value of the blurred image 20 decreases modestly at first, then starts to decrease more sharply around the blur parameter (standard deviation) of 0.4, and a steep increase is observed after the blur parameter (standard deviation) is 0.75. It should be noted that a sharp image blurred by a blur parameter less than 0.4 can still be considered sharp, since such a low level of blur is barely visible to the human eye. The different effects observed for blur, i.e., sharper decreases and increases in the blur score value, can also be used to indicate the sharpness or blur of the input image 10. For example, if the input image 10 is sufficiently sharp, i.e., has a blur score value of about 0.4, the blurred image 20 generated from the input image 10 by blurring (e.g., by Gaussian blur with a standard deviation of 0.7) has a blur score value of 0.32. The difference between the two blur score values ​​produced by subtracting the blur score value of blurred image 20 from the blur score value of input image 10 is a positive (non-negative) number (0.8).

[0055] However, if the input image 10 is blurred such that, for example, the blur is equivalent to the effect of a Gaussian blur with a standard deviation of 0.4 or more, the blur score value is less than about 0.39. When the input image 10 is further blurred with a Gaussian blur with a standard deviation of 0.7, a blurred image 20 is obtained that has a blur equivalent to the effect of a Gaussian blur with a standard deviation of 1.1 or more. Such a blurred image 20 may be associated with a blur score value of 0.44 (or more). In this case, the difference between the two blur score values ​​produced by subtracting the blur score value of the blurred image 20 from the blur score value of the input image 10 is a negative number (-0.05).

[0056] Based on the above observation, the blurred image 20 can be used to determine the blur score threshold, i.e., the blur score value of the blurred image 20 generated by blurring the input image 10 can be used as the blur score threshold. In this way, comparing the blur score value of the input image 10 with the blur score threshold determined from the blurred image 20 clearly indicates whether the input image 10 is sharp enough.

[0057] Preferably, appropriate blur parameters should be selected, ie blur parameters that result in significant visible blurring of the input image 10 by generating the blurred image 20 .

[0058] In a preferred embodiment, a clear image is taken and a series of blurred images 20 is generated by blurring, each blurred image of the series of blurred images 20 being generated by applying a series of subsequent blurs, preferably of the same type. Preferably, the series of subsequent blurs is performed by applying different blur parameters uniformly distributed over a predetermined range. For example, a Gaussian blur is used to generate the series of blurred images 20, each blurred image of the series of blurred images 20 being generated by a Gaussian blur with a different standard deviation, preferably the standard deviation is uniformly distributed in the range of 0.1 to 1.2. Preferably, a blur score value is calculated for each of the series of blurred images 20 according to the present invention, i.e. by generating a further two-dimensional frequency spectrum 22 of each blurred image 20, generating a further one-dimensional frequency spectrum 24 of the further two-dimensional frequency spectrum 22, fitting a further straight line 26 to the further one-dimensional frequency spectrum 24, and the blur score value is determined based on the residual of the fitting. After calculating the blur score value of each blurred image 20, a blur parameter (standard deviation of Gaussian blur) is selected that corresponds to the minimum blur score value of the set of blurred images 20. Thus, Fig. 8 can also be interpreted as a tool for determining appropriate blur parameters in the second scenario.

[0059] According to FIG. 8, in the case of Gaussian blur, the minimum (smallest) blur score value belongs to the standard deviation of 0.75, so it can be selected as a suitable blur parameter for generating the blurred image 20 from the input image 10. Obviously, different blur parameters can be selected according to the number of different blur parameters used to generate a series of blurred images 20. For example, if FIG. 8 is constructed by blur parameters of 0.1 or 0.2 increments instead of blur parameters of 0.05 increments, the smallest blur score value would belong to the standard deviation of 0.7 or 0.8, respectively, and this blur parameter would be used to generate the blurred image 20 for determining the blur score threshold. However, this does not affect the final conclusion of the method for determining whether the input image 10 is sufficiently sharp.

[0060] Furthermore, the invention relates to a data processing system comprising means for carrying out the steps of the method according to the invention. The data processing system preferably comprises an input for receiving an input image 10. Furthermore, the data processing system preferably comprises an image processing system for generating a two-dimensional frequency spectrum 12 from the input image 10. The data processing system preferably comprises means for generating a one-dimensional frequency spectrum 14 from the two-dimensional frequency spectrum 12, which means are preferably part of the image processing system.

[0061] The data processing system preferably further comprises a fitting unit for fitting a straight line 16 to the one-dimensional frequency spectrum 14. The fitting unit is preferably configured for determining a residual of the fitting. The data processing system preferably further comprises means for determining a blur score value based on the residual of the fitting.

[0062] Furthermore, the data processing system preferably further comprises a comparison unit for comparing the blur score value with a blur score threshold.

[0063] The data processing system, preferably as part of the image processing system, preferably further comprises a blurring unit, which preferably applies Gaussian blurring or any other blurring method known in the art to generate a blurred image 20 from the input image 10. Preferably, the blurring unit has an input for blur parameters characterizing the degree of blurring.

[0064] Furthermore, the present invention is a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out an embodiment of the method according to the present invention.

[0065] The computer program product may be executable by one or more computers.

[0066] Furthermore, the present invention relates to a computer readable medium comprising instructions which, when executed by a computer, cause the computer to carry out an embodiment of a method according to the present invention.

[0067] The computer readable medium may be a unitary medium, or may comprise more distinct pieces. [Explanation of symbols]

[0068] 10 Input images 10' Expanded Detail 12 Two-dimensional frequency spectrum 14 One-dimensional frequency spectrum 16 straight line 20 Blurred Images 20' Expanded Detail 22 Further two-dimensional frequency spectrum 24 Further One-Dimensional Frequency Spectra 26 More Straight Lines

Claims

1. A method for determining whether an input image (10) is sufficiently sharp, comprising the steps of: - providing a blur score threshold; - inputting said input image (10) into an image processing system; - generating, by said image processing system, a two-dimensional frequency spectrum (12) of said input image (10); - generating a one-dimensional frequency spectrum (14) from said two-dimensional frequency spectrum (12); - fitting a straight line (16) to said one-dimensional frequency spectrum (14); - determining a blur score value based on the residuals of said fitting and considering said input image (10) to be sufficiently sharp based on a comparison of said blur score value with said blur score threshold; The method includes:

2. 2. The method of claim 1, wherein the one-dimensional frequency spectrum (14) is generated from a two-dimensional amplitude spectrum of the two-dimensional frequency spectrum (12).

3. 3. The method of claim 2, wherein a logarithm of the two-dimensional amplitude spectrum is calculated, and the one-dimensional frequency spectrum (14) is generated from the logarithm of the two-dimensional amplitude spectrum.

4. 4. The method of claim 2 or 3, wherein the one-dimensional frequency spectrum (14) is a histogram having bins corresponding to frequency ranges of the two-dimensional frequency spectrum (12) and histogram values ​​for each bin, each histogram value being the average amplitude or integrated amplitude in a corresponding frequency range.

5. 4. The method according to claim 1, wherein the two-dimensional frequency spectrum (12) is central, with a frequency corresponding to zero frequency being located at the center of the spectrum and increasing in frequency radially outward from the center, and the one-dimensional frequency spectrum (14) is generated by radial averaging.

6. The method according to any one of claims 1 to 3, characterized in that the two-dimensional frequency spectrum (12) of the input image (10) is generated by a two-dimensional Fourier transform or a two-dimensional wavelet transform.

7. The method according to any one of claims 1 to 3, characterized in that the straight line (16) is fitted by a linear regression method.

8. providing the blur score threshold includes determining between a first scenario and a second scenario based on empirical data; In the first scenario, the blur score threshold is a predetermined value; In the second scenario, the blur score threshold is calculated by the following steps: - generating a blurred image (20) from said input image (10), - generating, by said image processing system, a further two-dimensional frequency spectrum (22) of said blurred image (20), - generating a further one-dimensional frequency spectrum (24) from said further two-dimensional frequency spectrum (22), and - fitting a further straight line (26) to said further one-dimensional frequency spectrum (24); The method of any one of claims 1 to 3, characterized in that the blur score threshold is determined by: and the blur score threshold is based on the residuals of the fitting.

9. In the second scenario, the blurred image (20) is generated by blurring with blur parameters, The blur parameters are calculated by the following steps: - acquiring a clear image, - generating a sequence of blurred images (20) based on said sharp image by applying a sequence of subsequent blurs, - generating a further two-dimensional frequency spectrum (22) for each blurred image (20), generating a further one-dimensional frequency spectrum (24) of said further two-dimensional frequency spectrum (22) and calculating a blur score value for each blurred image of the series of blurred images (20) by fitting a further straight line (26) to said further one-dimensional frequency spectrum (24), said blur score value being determined based on a residual of said fitting; and The method according to claim 8, characterized in that the blur parameter is determined by the step of: selecting the blur parameter that corresponds to the minimum of the blur score values ​​of the series of blurred images (20).

10. 10. The method of claim 9, wherein the blurred image (20) is generated by Gaussian blurring.

11. 10. The method of claim 9, wherein the further two-dimensional frequency spectrum (22) is generated in the same way as the two-dimensional frequency spectrum (12), the further one-dimensional frequency spectrum (24) is generated in the same way as the one-dimensional frequency spectrum (14), and the further straight line (26) is fitted in the same way as the straight line (16).

12. 4. The method according to claim 1, wherein the comparison of the blur score value with the blur score threshold is performed by subtracting the blur score threshold from the blur score value to generate a difference score value, and a difference score value having a positive or non-negative value indicates that the input image (10) is sufficiently sharp.

13. 4. The method according to claim 1, wherein the comparison of the blur score value with the blur score threshold is performed by dividing the blur score value by the blur score threshold to generate a score ratio, and a score ratio having a value greater than 1 or equal to or greater than 1 indicates that the input image (10) is sufficiently sharp.

14. A data processing system comprising means for performing the steps of the method of claim 1.

15. A non-transitory computer program product comprising instructions that cause a computer to perform the method of claim 1 when the program is executed by the computer.

16. A non-transitory computer readable medium comprising instructions that, when executed by a computer, cause the computer to perform the method of claim 1.

Images