Method and electronic device for detecting blur in image

By detecting the entropy and probability fusion of image regions, differentiating different types of blurs is solved, and the problem of difficult to distinguish and deal with blur types in the prior art is solved, and accurate correction and quality improvement of images are achieved.

CN119072715APending Publication Date: 2024-12-03SAMSUNG ELECTRONICS CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380032850.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-09
Filing Date
2023-04-10
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to distinguish and deal with blurring intentionally introduced by users, blurring caused by subjects in motion, blurring caused by cameras in motion, and blurring caused by automatic focus failure, resulting in the debuffering operation that may degrade image quality.

Method used

By detecting the entropy of multiple regions in the image, classifying it into clear, blurred and candidate blur areas, and fusing the global blur probability, local blur probability and intentional blur probability, a determination about correcting global and local blur is generated to avoid intentional blur images Perform defuzzing operation.

Benefits of technology

Accurate detection and distinction of different types of blur is achieved, deterioration of image quality is avoided, and the accuracy of defuzzing operations and image quality is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119072715A_ABST
    Figure CN119072715A_ABST
Patent Text Reader

Abstract

Embodiments herein are used to provide a method and an electronic device for detecting blur in an input image. The method includes detecting, by an electronic device, one or more candidate blur regions of a plurality of regions in the input image. Further, the method includes determining a confidence score for the one or more candidate blur regions in the input image. Further, the method includes determining a confidence score for global blur in the input image. Further, the method includes determining a confidence score for intentional blur in the input image. Further, the method includes detecting, by the electronic device, at least a blur type and an intensity of the blur type in the input image based on a confidence score for the one or more candidate blur regions, a confidence score for the global blur, and a confidence score for the intentional blur.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to image processing, and more particularly to a method and electronic device for detecting blur in an image. Background Art

[0002] Image enhancement has gained a lot of attention recently, especially in the consumer market including but not limited to smartphones. Leading smartphone vendors have made remarkable progress in image enhancement in areas including but not limited to high dynamic range (HDR). However, one of the most common artifacts seen in images is blur, making deblurring a very critical enhancement method.

[0003] like Figures 1A to 1D As shown, blur in an image is caused by a moving subject, a moving camera, autofocus failure, and intentional lens or bokeh blur. Conventional methods and systems can deblur blur in an image, however, conventional methods cannot distinguish blur intentionally introduced by a user ( Figure 1D ), blur caused by moving subjects ( Figure 1A ), blur caused by a moving camera ( Figure 1B ) and blur caused by autofocus failure ( Figure 1C ). Therefore, in the blurred image ( Figure 1D ) will degrade the quality of the image. Summary of the invention

[0004] Solution to the problem

[0005] The main purpose of the embodiments of this document is to provide a method and an electronic device for detecting blur in an input image. The method includes detecting at least a blur type and an intensity of the blur type in the input image. When the blur type is detected as intentional blur, the electronic device does not perform a deblurring operation.

[0006] Another object of an embodiment of the present invention is to measure multiple entropies of multiple regions in an input image, and classify a first group of regions in the multiple regions having an entropy lower than a first threshold as clear, classify a second group of regions in the multiple regions having an entropy higher than a second threshold as blurred, and classify a third group of regions in the multiple regions having an entropy higher than the first threshold and lower than the second threshold as candidate blurred regions.

[0007] It is yet another object of embodiments herein to fuse global blur probabilities, local blur probabilities, and intentional blur probabilities in an input image to generate a determination regarding correcting global blur and / or local blur.

[0008] Therefore, an embodiment of the present invention will provide a method for detecting blur in an input image. The method includes detecting, by the electronic device, one or more candidate blur regions among a plurality of regions in the input image. In addition, the method includes determining, by the electronic device, a confidence score of the one or more candidate blur regions in the input image. In addition, the method includes determining, by the electronic device, a confidence score of a global blur in the input image. In addition, the method includes determining, by the electronic device, a confidence score of an intentional blur in the input image. In addition, the method includes: detecting, by the electronic device, a blur type and an intensity of the blur type in at least the input image based on the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

[0009] In an embodiment, the method includes measuring, by the electronic device, a plurality of entropies of the plurality of regions in the input image. In addition, the method includes classifying, by the electronic device, a first group of regions in the plurality of regions having an entropy below a first threshold as clear, a second group of regions in the plurality of regions having an entropy above a second threshold as blurry, and a third group of regions in the plurality of regions having an entropy above the first threshold and below the second threshold as candidate blurry regions.

[0010] In an embodiment, the method includes measuring, by the electronic device, a plurality of entropies of the plurality of regions in the input image. In addition, the method includes determining, by the electronic device, a value of the entropy near the center of the input image as low, and determining a value of the entropy near the edge of the input image as high. In addition, the method includes determining, by the electronic device, a confidence score of the intentional blur as high.

[0011] In an embodiment, the method includes generating, by the electronic device, a segmented image indicating the one or more candidate blur regions by fusing a candidate blur region mask with the input image, wherein the candidate blur region mask indicates entropy of the multiple regions in the input image. In addition, the method includes determining, by the electronic device, a confidence score for each of the one or more candidate blur regions in the segmented image.

[0012] In an embodiment, the method comprises analyzing, by the electronic device, an input image as a whole. Furthermore, the method comprises determining, by the electronic device, a confidence score for the global blur in the input image based on a level of presence of the global blur in the input image.

[0013] In an embodiment, the method includes calculating, by the electronic device, a first weight for a confidence score for the one or more candidate blur regions based on a percentage of pixels associated with the one or more candidate blur regions, calculating a second weight for a confidence score for the global blur based on a percentage of pixels associated with the global blur region, and calculating a third weight for a confidence score for the intentional blur based on a percentage of pixels associated with the intentional blur region. In addition, the method includes detecting the blur type and the intensity of the blur type in the input image using the first weight, the second weight, the third weight, the confidence score of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

[0014] In an embodiment, the method includes determining, by the electronic device, that at least the blur type and the intensity of the blur type satisfy a blur threshold. In addition, the method includes displaying a recommendation on the electronic device, wherein the recommendation is related to at least one of deleting the input image, enhancing the input image, deblurring the input image, and recapturing the input image. In addition, the method includes generating a tag including an image quality parameter and storing the tag associated with the input image in a media database, wherein the image quality parameter includes at least one of the blur type and the intensity of the blur type.

[0015] Therefore, embodiments of the present invention provide a method for detecting global blur in an input image requiring blur correction by an electronic device. In addition, the method includes estimating, by the electronic device, a global blur probability as a measure of a confidence level that there is global blur. In addition, the method includes detecting, by the electronic device, one or more local regions with candidate blurs in an image. In addition, the method includes measuring, by the electronic device, entropy in the detected one or more local regions. In addition, the method includes selecting, by the electronic device, one or more local regions with a predefined entropy range for local blur correction. In addition, the method includes estimating, by the electronic device, a local blur probability as a measure of a confidence level that there is local blur. In addition, the method includes detecting, by the electronic device, one or more clear regions with a predefined entropy range. In addition, the method includes estimating, by the electronic device, an intentional blur probability as a measure of a confidence level that there is blur intentionally introduced by a user. In addition, the method includes fusing, by the electronic device, the global blur probability, the local blur probability, and the intentional blur probability to generate a determination about correcting global blur and / or local blur.

[0016] Therefore, an embodiment of the present invention is to provide an electronic device for detecting blur in an input image, comprising: a memory; a processor coupled to the memory; and a blur detector coupled to the memory and the processor. The blur detector is configured to detect one or more candidate blur regions among a plurality of regions in the input image. The blur detector is configured to: determine a confidence score of the one or more candidate blur regions in the input image. The blur detector is configured to determine a confidence score of a global blur in the input image. The blur detector is configured to determine a confidence score of an intentional blur in the input image. The blur detector is configured to detect at least a blur type and an intensity of the blur type in the input image based on the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

[0017] Therefore, an embodiment of the present invention provides an electronic device for blur correction management of an input image, comprising: a memory; a processor coupled to the memory; and a blur detector coupled to the memory and the processor. The blur detector is configured to detect global blur in an input image that requires blur correction. The blur detector is also configured to estimate a global blur probability as a measure of the confidence level of the presence of global blur. The blur detector is also configured to detect one or more local regions with candidate blurs in the image. The blur detector is also configured to measure entropy in the detected one or more local regions. The blur detector is also configured to select one or more local regions with a predefined entropy range for local blur correction. The blur detector is also configured to estimate a local blur probability as a measure of the confidence level of the presence of the local blur. The blur detector is also configured to detect one or more clear regions with a predefined entropy range. The blur detector is also configured to estimate an intentional blur probability as a measure of the confidence level of the presence of blur intentionally introduced by the user. The blur detector is further configured to fuse the global blur probability, the local blur probability, and the intentional blur probability to generate a determination regarding correcting the global blur and / or the local blur.

[0018] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and accompanying drawings. However, it should be understood that the following description, although indicating preferred embodiments and many of their specific details, is given by way of illustration and not limitation. Many changes and modifications may be made within the scope of the embodiments herein, and the embodiments herein include all such modifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present disclosure is illustrated in the accompanying drawings, and the same reference numerals indicate corresponding parts throughout the drawings. The embodiments of the present invention will be better understood from the following description with reference to the accompanying drawings, in which:

[0020] Figures 1A to 1D are captured images showing blur caused by a moving subject, blur caused by a moving camera, blur caused by autofocus failure, and intentionally induced blur, respectively, according to the prior art;

[0021] Figure 1E is a sequence diagram showing a comparison between a conventional local region detection for deblurring according to the prior art and the proposed local region detection;

[0022] Figure 2 is a block diagram of an electronic device for detecting blur in an input image according to an embodiment as disclosed herein;

[0023] Figure 3 is a flow chart illustrating a method of detecting blur in an input image according to an embodiment as disclosed herein;

[0024] Figure 4A is a sequence diagram illustrating operations performed for determining the type and intensity of blur in an input image according to an embodiment as disclosed herein;

[0025] Figure 4B is a schematic diagram illustrating operations performed for determining the type and intensity of blur using blur probabilities in an input image according to embodiments as disclosed herein;

[0026] Figure 4C is a sequence diagram illustrating operations performed for detecting intentional blur in an input image according to an embodiment as disclosed herein;

[0027] Figure 4D is a sequence diagram illustrating operations performed for detecting local blur in an input image according to an embodiment as disclosed herein;

[0028] Figure 4E is a sequence diagram illustrating operations performed for detecting local blur using blur in the foreground in an input image according to an embodiment as disclosed herein;

[0029] Figure 4F is a sequence diagram illustrating operations performed for detecting global blur in an input image according to an embodiment as disclosed herein;

[0030] Figure 4G is a sequence diagram illustrating operations performed for detecting sharp in an input image according to an embodiment as disclosed herein;

[0031] Figure 4H is a sequence diagram illustrating another example of detecting local blur in an input image according to an embodiment as disclosed herein;

[0032] Figure 5 is a schematic diagram showing a blurred scene detected in an image available in a gallery of an electronic device according to an embodiment as disclosed herein;

[0033] Fig. 6A is a graph illustrating classification of multiple regions based on entropy according to an embodiment as disclosed herein;

[0034] Figure 6B is a sequence diagram illustrating entropy-based fuzzy candidate positioning and sharpness positioning according to an embodiment as disclosed herein;

[0035] Fig. 7A is a sequence diagram illustrating detection of blur types in an input image according to an embodiment as disclosed herein;

[0036] Figure 7B is a sequence diagram illustrating local fuzzy candidate refinement according to an embodiment as disclosed herein;

[0037] Figure 8 is a flow chart illustrating a content management hub for image quality assessment and image enhancement services according to embodiments as disclosed herein;

[0038] 9A to 9C is a captured image showing a re-master feature in a smartphone according to an embodiment as disclosed herein;

[0039] Fig.10 is a sequence diagram illustrating blur positioning for controlled artifact-free deblurring according to an embodiment as disclosed herein;

[0040] Fig.11A is a schematic diagram showing identification of highly blurred images and low blurred images in a gallery and making recommendations accordingly according to an embodiment as disclosed herein;

[0041] Fig. 11B is a schematic diagram illustrating the operation of a deblurring engine according to an embodiment as disclosed herein;

[0042] Fig.12 is a schematic diagram showing identification of subtle motion in IOT applications or monitoring according to an embodiment as disclosed herein;

[0043] Fig.13is a sequence diagram illustrating identification of subtle motion in IOT applications or monitoring according to an embodiment as disclosed herein;

[0044] Fig.14 is a schematic diagram illustrating assistance for automatic capture of factory camera calibration according to an embodiment as disclosed herein; and

[0045] Fig.15 is a sequence diagram illustrating assistance for automatic capture of factory camera calibration according to embodiments as disclosed herein. DETAILED DESCRIPTION

[0046] The embodiments of this article and its various features and advantageous details are explained more fully with reference to the non-limiting embodiments shown in the drawings and described in detail in the following description. The description of known components and processing techniques is omitted so as not to unnecessarily obscure the embodiments of this article. In addition, because some embodiments can be combined with one or more other embodiments to form new embodiments, the various embodiments described herein do not have to be mutually exclusive. Unless otherwise stated, the term "or" used herein refers to a non-exclusive or. The examples used herein are intended only to facilitate understanding of the manner in which the embodiments of this article can be practiced, and further enable those skilled in the art to practice the embodiments of this article. Therefore, the examples should not be interpreted as limiting the scope of the embodiments of this article.

[0047] As conventional in the art, embodiments may be described and illustrated according to blocks that perform one or more functions described. These blocks, which may be referred to herein as managers, units, modules, hardware components, etc., are physically implemented by analog and / or digital circuits (such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hard-wired circuits, etc.), and may optionally be driven by firmware and software. The circuit may, for example, be embodied in one or more semiconductor chips, or on a substrate support (such as a printed circuit board, etc.). The circuits constituting the blocks may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuits), or by a combination of dedicated hardware that performs some functions of the blocks and a processor that performs other functions of the blocks. Without departing from the scope of the present disclosure, each block of the embodiment may be physically divided into two or more interacting and discrete blocks. Similarly, without departing from the scope of the present disclosure, the blocks of the embodiment may be physically combined into more complex blocks.

[0048] The accompanying drawings are used to help easily understand various technical features, and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. Therefore, except for those specifically set forth in the accompanying drawings, the present disclosure should be interpreted as extending to any changes, equivalents and substitutes. Although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are usually only used to distinguish one element from another element.

[0049] Therefore, an embodiment of the present invention will provide a method for detecting blur in an input image. The method includes detecting, by an electronic device, one or more candidate blur regions among a plurality of regions in an input image. The method includes determining, by an electronic device, a confidence score for one or more candidate blur regions in the input image. In addition, the method includes determining, by an electronic device, a confidence score for a global blur in the input image. In addition, the method includes determining, by an electronic device, a confidence score for an intentional blur in the input image. In addition, the method includes detecting, by an electronic device, a blur type and an intensity of a blur type in at least the input image based on the confidence scores of one or more candidate blur regions, the confidence score for the global blur, and the confidence score for the intentional blur.

[0050] Therefore, embodiments of the present invention provide a method for detecting global blur in an input image that requires blur correction by an electronic device. In addition, the method includes estimating, by the electronic device, a global blur probability as a measure of the confidence level that there is global blur. In addition, the method includes detecting, by the electronic device, one or more local regions with candidate blurs in the image. In addition, the method includes measuring, by the electronic device, the entropy in the detected one or more local regions. In addition, the method includes selecting, by the electronic device, one or more local regions with a predefined entropy range for local blur correction. In addition, the method includes estimating, by the electronic device, a local blur probability as a measure of the confidence level that there is local blur. In addition, the method includes detecting, by the electronic device, one or more clear regions with a predefined entropy range. In addition, the method includes estimating, by the electronic device, an intentional blur probability as a measure of the confidence level that there is blur intentionally introduced by the user. In addition, the method includes fusing, by the electronic device, the global blur probability, the local blur probability, and the intentional blur probability to generate a determination about correcting global blur and / or local blur.

[0051] Therefore, an embodiment of the present invention is to provide an electronic device for detecting blur in an input image, comprising: a memory; a processor, coupled to the memory; and a blur detector, coupled to the memory and the processor. The blur detector is configured to detect one or more candidate blur regions among multiple regions in the input image. The blur detector is configured to determine a confidence score for one or more candidate blur regions in the input image. The blur detector is configured to determine a confidence score for global blur in the input image. The blur detector is configured to determine a confidence score for intentional blur in the input image. The blur detector is configured to detect at least the blur type and the intensity of the blur type in the input image based on the confidence scores of one or more candidate blur regions, the confidence score of global blur, and the confidence score of intentional blur.

[0052] Therefore, an embodiment of the present invention is to provide an electronic device for blur correction management of an input image, comprising: a memory; a processor, coupled to the memory; and a blur detector, coupled to the memory and the processor. The blur detector is configured to detect global blur in an input image that requires blur correction. The blur detector is also configured to estimate a global blur probability as a measure of the confidence level of the presence of global blur. The blur detector is also configured to detect one or more local regions with candidate blurs in the image. The blur detector is also configured to measure the entropy in the detected one or more local regions. The blur detector is also configured to select one or more local regions with a predefined entropy range for local blur correction. The blur detector is also configured to estimate a local blur probability as a measure of the confidence level of the presence of local blur. The blur detector is also configured to detect one or more clear regions with a predefined entropy range. The blur detector is also configured to estimate an intentional blur probability as a measure of the confidence level of the presence of blur intentionally introduced by the user. The blur detector is also configured to fuse the global blur probability, the local blur probability, and the intentional blur probability to generate a determination about correcting global blur and / or local blur.

[0053] In general, advances in mobile camera sensors have significantly facilitated image enhancement applications, including but not limited to deblurring, denoising, and sharpening. Despite significant technological advances, blur and noise remain the most significant factors that degrade the perceived quality of input images. While global motion blur is typically caused by camera shake during capture and relative motion between the camera and the object, defocus or local blur occurs due to large apertures and incorrect focus settings. Blurs in images significantly degrade the quality of the input image and result in loss of detailed information. Therefore, blur detection is critical to identifying blurry images and triggering the deblurring engine in order to enrich the user experience by seamlessly providing high-quality sharp images.

[0054] In most cases, blur and noise are inherently related, with denoising removing fine structures as well as unwanted details, while blur removal restores structures and fine details. This interconnectedness makes the development of image enhancement algorithms extremely challenging. Therefore, the quality of the enhanced image critically depends on the order in which the enhancement engines are applied. In this regard, the blur detection probability plays a key role in determining the order in which the enhancement engines need to be triggered to achieve the best quality restoration. Therefore, the proposed method and electronic device use an automatic blur detection methodology and the associated probabilities. Unlike conventional methods and systems, the proposed method and electronic device perform blur detection in an input image through an effective fusion of local blur properties and global blur properties. Unlike conventional methods and systems, the proposed method and electronic device segment or locate motion candidates in the input image that may contribute to the blur in the input image.

[0055] Currently, problems faced by users include but are not limited to blur in an image significantly deteriorating the perceived quality of the image, difficulty in reading scanned blurry documents, challenges in developing deblurring engines that are agnostic to the blur level, and difficulty in determining the order in which enhancement engines will be applied to obtain the best quality enhanced image.

[0056] Conventional methods and systems include, but are not limited to, multi-frame noise reduction (MFNR) and high dynamic range (HDR) imaging techniques to perform image enhancement. Conventional methods and systems use photometric differences to generate a motion map showing the degree of motion caused for each pixel to determine blur in an image, however, conventional methods require multiple frames to generate a motion map and are not applicable to a single image.

[0057] In other conventional methods and systems, the image is first converted into an edge map by convolving the image with an edge filter (e.g., Sobel, Laplace, etc.). The variance of the edge map is used to determine the degree of blur in the image, where a high variance indicates a sharp image and a low variance indicates a blurry image. However, conventional methods cannot distinguish intentional blur from artifacts, and further, because conventional methods only consider the overall view of the image, conventional methods cannot detect local blur.

[0058] In other conventional methods and systems, the image is first converted to some frequency domain, such as wavelet, Fourier, etc. In addition, the low-frequency areas are classified as blurry. However, conventional methods cannot identify intentional blur in images.

[0059] Unlike conventional methods and systems, the proposed method and electronic device automatically detects images with unintentionally created blur artifacts from a gallery and passes them to a deblur enhancement engine to remove blur.

[0060] Unlike conventional methods and systems, the proposed method and electronic device automatically detects bokeh blur in images and images clicked in portrait mode.

[0061] Different from conventional methods and systems, the proposed method and electronic device are able to recognize blurry images and identify intentional blur introduced by the user by locating candidate blur regions.

[0062] Other conventional methods and systems use frequency transform or wavelet transform to perform blur detection in the frequency domain. However, it is difficult to implement frequency transform in mobile devices. Unlike conventional methods and systems, the proposed method and electronic device perform blur detection in red, green, and blue (RGB) / brightness (YUV) / hue saturation value (HSV) / other color domains, and thus the proposed method and electronic device eliminate the need for frequency transform. In addition, the proposed method and electronic device do not locate the "area of ​​interest" to be checked for blur, resulting in that images or videos with a bokeh effect in the background are classified as blurred.

[0063] Other conventional methods and systems only perform detection of defocus blur. Unlike conventional methods and systems, the proposed method and electronic device are also capable of detecting motion blur.

[0064] Other conventional methods and systems involve generating a local sharpness map from an input image, wherein the local sharpness map uses an edge filter to mark sharp areas in the input image. However, conventional methods and systems do not refine the sharpness map or use any global information in the case of bokeh / defocus blur, resulting in misclassification.

[0065] Other conventional methods and systems rely on temporal information to determine global / local motion and cannot be directly extended to single-frame blur detection. Unlike conventional methods and systems, the proposed method and electronic device focuses on local blur candidates and only requires single-frame information to determine blurry or non-blurred areas.

[0066] Other conventional methods and systems use regression to estimate blur or unblur masks from input images, where blur or unblur masks require pixel-wise labeled ground truth images to train the network. Conventional methods are extended to detect motion and defocus blur, however, conventional methods do not include global blur detection and local blur detection.

[0067] Other conventional methods and systems do not propose a methodology for detecting blurry frames, and therefore are even applied to frames that do not need deblurring, resulting in artifacts. Unlike conventional methods and systems, the proposed method and electronic device are able to detect blurry images or videos, and accurately detect global blue due to camera panning, defocusing, and local due to motion of local components. The proposed method and electronic device detect areas in the input image that need to be deblurred, so the proposed method and electronic device do not mistakenly deblur the user's intentional blur. Examples of intentional blur include, but are not limited to, artistic bokeh blur, lens blur.

[0068] Other conventional methods and systems determine blur scores based on edges in the image. The width of the edge in the image is determined and a threshold is used to classify the image as sharp or blurry. This threshold is difficult to determine and varies for different capture devices. Unlike conventional methods and systems, the proposed method and electronic device uses a neural network to refine motion candidates and does not require tuning for different capture devices.

[0069] Other conventional methods and systems use frequency conversion which is slow on mobile devices. Unlike conventional methods and systems, the proposed method and electronic device do not require frequency conversion.

[0070] Other conventional methods and systems detect motion blur at the time of capture and are unable to detect blur after an image has been captured. Unlike conventional methods and systems, the proposed method and electronic device detect blur at any point in the journey of an image, including but not limited to during capture, post-capture after saving to a gallery, or after uploading to / downloading from a social networking service.

[0071] Figures 1A to 1D 1 and 2 are captured images respectively showing blur caused by a subject in motion, blur caused by a camera in motion, blur caused by autofocus failure, and intentionally induced blur according to the related art.

[0072] Generally, blur in an image is caused by four reasons: 1) Figure 1A The blur caused by the moving subject, as shown in Figure 1B The blur caused by the moving camera, 3) Figure 1C The blur caused by autofocus failure is shown in the figure. Figure 1D The blur shown is due to intentional lens induced blur or bokeh.

[0073] Figure 1E is a sequence diagram showing a comparison of conventional local region detection and the proposed local region detection for deblurring according to the prior art.

[0074] The conventional method and system detects one salient object (101) in an input image (421) for deblurring. Unlike the conventional method and system, the proposed method and electronic device detects a plurality of local regions (102) in an input image (421) for deblurring.

[0075] Conventional methods and systems cannot be used for global deblurring when the entire image is of interest. Conventional methods and systems forcibly select a sub-region of interest and deblur only that region. Unlike conventional methods and systems, the proposed method and electronic device focus on sharpness defined by pixel contributions and therefore evaluate the input image (421) in terms of the perceived impact of deblurring on the user.

[0076] Referring now to the drawings, and more particularly to Figures 2 to 15 , wherein like reference numerals consistently denote corresponding features throughout the accompanying drawings, a preferred embodiment is shown.

[0077] Figure 2 is a block diagram of an electronic device (100) for detecting blur in an input image according to an embodiment as disclosed herein.

[0078] The electronic device (100) includes a memory (201), a communicator (202), a processor (203), and a blur detector (204). The blur detector (204) includes a global blur confidence score determiner (205), a local blur confidence score determiner (206), an intentional blur confidence score determiner (207), and a blur type and intensity detector (208). The blur detector (204) can be implemented as part of the processor (203) (such as an image processor).

[0079] An electronic device (100) according to an embodiment as disclosed herein is used to detect blur in an input image. Examples of the electronic device (100) include, but are not limited to, smartphones, tablet computers, personal digital assistants (PDAs), Internet of Things (IoT) devices, AR devices, VR devices, and wearable devices.

[0080] In an embodiment, the memory (201) stores instructions to be executed by the processor (203). The memory (201) may include a non-volatile storage element. Examples of such non-volatile storage elements may include a magnetic hard disk, an optical disk, a floppy disk, a flash memory, or a form of an electrically programmable memory (EPROM) or an electrically erasable programmable (EEPROM) memory. In addition, in some examples, the memory (201) may be considered a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or a propagating signal. However, the term "non-transitory" should not be interpreted as the memory (201) being non-removable. In some examples, the memory (201) may be configured to store a larger amount of information than the memory. In some examples, the non-transitory storage medium may store data that may change over time (e.g., in a random access memory (RAM) or a cache memory). The memory (201) may be an internal storage unit, or it may be an external storage unit of the electronic device (100), cloud storage, or any other type of external storage.

[0081] The processor (203) communicates with the memory (201), and the processor (203) is configured to execute instructions stored in the memory (201) and perform various processes. The processor (203) may include one or more processors, which may be a general-purpose processor (such as a central processing unit (CPU), an application processor (AP), etc.), a graphics processing unit (such as a graphics processing unit (GPU)), a visual processing unit (VPU) and / or an artificial intelligence (AI) dedicated processor (such as a neural processing unit (NPU)).

[0082] The communicator (202) is configured to communicate internally between internal hardware components and with external devices (e.g., eNodeB, gNodeB, server, etc.) via one or more networks (e.g., radio technologies). The communicator (202) includes electronic circuitry dedicated to implementing standards for wired or wireless communication.

[0083] The blur detector is implemented by processing circuits (such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hard-wired circuits, etc.), and may optionally be driven by firmware. The circuits may be implemented, for example, in one or more semiconductor chips, or on a substrate support (such as a printed circuit board, etc.).

[0084] A global blur confidence score determiner (205) determines a confidence score of a global blur in an input image. A local blur confidence score determiner (206) detects one or more candidate blur regions among a plurality of regions in the input image and determines confidence scores of the one or more candidate blur regions in the input image. An intentional blur confidence score determiner (207) determines a confidence score and a blur type of intentional blur in the input image. In addition, a blur type and intensity detector (208) detects at least a blur type and an intensity of the blur type in the input image based on the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

[0085] In an embodiment, the blur detector (204) is configured to measure a plurality of entropies of a plurality of regions in the input image. The blur detector (204) is further configured to classify a first group of regions in the plurality of regions having an entropy below a first threshold as sharp, a second group of regions in the plurality of regions having an entropy above a second threshold as blurry, and a third group of regions in the plurality of regions having an entropy above the first threshold and below the second threshold as candidate blurry regions.

[0086] In an embodiment, the blur detector (204) is configured to measure a plurality of entropies for a plurality of regions in the input image. The blur detector (204) is further configured to determine a value of entropy towards the center of the input image as low, and to determine a value of entropy towards the edge of the input image as high. The blur detector (204) is further configured to determine a confidence score of intentional blur as high.

[0087] In an embodiment, the blur detector (204) is configured to generate a segmented image indicating one or more candidate blur regions by fusing a candidate blur region mask with an input image, wherein the candidate blur region mask indicates entropy of a plurality of regions in the input image. The blur detector (204) is further configured to determine a confidence score for each of the one or more candidate blur regions in the segmented image.

[0088] In an embodiment, the blur detector (204) is configured to analyze the input image as a whole.The blur detector (204) is further configured to determine a confidence score for global blur in the input image based on a level of presence of global blur in the input image.

[0089] In an embodiment, the blur detector (204) is configured to calculate a first weight for a confidence score for one or more candidate blur regions based on a percentage of pixels associated with the one or more candidate blur regions, a second weight for a confidence score for global blur based on a percentage of pixels associated with the global blur region, and a third weight for a confidence score for intentional blur based on a percentage of pixels associated with the intentional blur region. The blur detector (204) is further configured to detect blur types and intensities of blur types in an input image using the first weight, the second weight, the third weight, the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

[0090] In an embodiment, the blur detector (204) is configured to determine that at least a blur type and an intensity of the blur type satisfy a blur threshold. The blur detector (204) is further configured to display a recommendation on the electronic device (100), wherein the recommendation is related to at least one of deleting the input image, enhancing the input image, deblurring the input image, and recapturing the input image. The blur detector (204) is further configured to generate a tag including an image quality parameter and store the tag associated with the input image in a media database, wherein the image quality parameter includes at least one of the blur type and the intensity of the blur type.

[0091] In an embodiment, a blur detector (204) is configured to detect global blur in an input image that requires blur correction. The blur detector (204) is further configured to estimate a global blur probability as a measure of a confidence level that global blur exists. The blur detector (204) is further configured to detect one or more local regions in the image having candidate blur. The blur detector (204) is further configured to measure entropy in the detected one or more local regions. The blur detector (204) is further configured to select one or more local regions having a predetermined entropy range for local blur correction. The blur detector (204) is further configured to estimate a local blur probability as a measure of a confidence level that local blur exists. The blur detector (204) is further configured to detect one or more clear regions having a predefined entropy range and estimate an intentional blur probability as a measure of a confidence level that blur intentionally introduced by a user exists. The blur detector (204) is further configured to fuse the global blur probability, the local blur probability, and the intentional blur probability to generate a determination about correcting global blur and / or local blur.

[0092] Figure 3 is a flow chart illustrating a method of detecting blur in an input image according to an embodiment as disclosed herein.

[0093] In step 301, the electronic device (100) detects one or more candidate blur regions among a plurality of regions in an input image.

[0094] In step 302, the electronic device (100) determines confidence scores for one or more candidate blur regions in an input image.

[0095] In step 303, the electronic device (100) determines a confidence score for the global blur in the input image.

[0096] At step 304, the electronic device (100) determines a confidence score for intentional blur in the input image.

[0097] In step 305, the electronic device (100) detects at least a blur type and an intensity of the blur type in the input image based on the confidence scores of one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

[0098] The various actions, actions, blocks, steps, etc. in the flowchart can be performed in the order presented, in a different order, or simultaneously. In addition, in some embodiments, some of the actions, actions, blocks, steps, etc. can be omitted, added, modified, skipped, etc. without departing from the scope of the present disclosure.

[0099] Figure 4A is a sequence diagram illustrating operations performed to determine the type and intensity of blur in an input image according to embodiments as disclosed herein.

[0100] At 422, the electronic device (100) detects one or more local regions in the input image that have candidate blurs. The electronic device (100) identifies regions in the input image that are candidates for the presence of blurs. The entropy in the neighborhood of each pixel is calculated, and if h>entropy>l, the pixel is marked as a blur candidate.

[0101] Input: Image X W×H×C

[0102] Output: Candidate mask M W×H

[0103] At 423, the electronic device (100) measures entropy in the detected one or more local regions. In addition, the electronic device (100) fuses the input image with the mask.

[0104] At 424, the electronic device (100) selects one or more local regions having a predetermined entropy range for local blur correction. The electronic device (100) predicts whether blur exists in the identified candidate regions. This is a deep learning model that takes a combination of an input image and a blur candidate mask and outputs blur probabilities for the identified candidates. Global blur detection differs from local blur detection in that local blur detection looks at candidate regions rather than the entire image.

[0105] Input: Image X W×H×C and the candidate mask M W×H

[0106] Output: Probability P of local blur in the image local (x)

[0107] At 425, the electronic device (100) estimates a local blur probability as a measure of the confidence level that a local blur exists.

[0108] At 426, the electronic device (100) detects global blur in the input image that requires blur correction. The electronic device (100) predicts whether global blur is present in the input image. This is a deep learning model that looks at the entire image and predicts the probability of global blur in the input image.

[0109] Input: Image X W×H×C

[0110] Output: Probability P of global blur in the image global (x)

[0111] At 427, the electronic device (100) estimates a global ambiguity probability as a measure of the confidence level that global ambiguity exists.

[0112] At 428, the electronic device (100) detects one or more clear regions with a predefined entropy range. The electronic device (100) locates the clear regions in the input image. The entropy in the neighborhood of each pixel is calculated, and if l> entropy, the pixel is marked as clear.

[0113] Input: Image X W×H×C

[0114] Output: Clarity mask S W×H

[0115] At 429 , the electronic device ( 100 ) fuses the input image with the sharpness mask.

[0116] At 430, the electronic device (100) detects the presence of intentional blur in the input image. The electronic device (100) predicts whether the blur in the input image is intentionally introduced by the user. This is a deep learning model that takes the input image and the sharpness mask as input and outputs the probability that the blur is intentional. Intentional blur detection detects a lens blur or bokeh blur image as sharp or not degraded by blur.

[0117] Input: Image X W×H×C and clarity mask S W×H

[0118] Output: Probability P of intentional blur in the image int (x)

[0119] At 431 and 432, the electronic device (100) estimates an intentional blur probability as a measure of the confidence level that there is blur intentionally introduced by the user.

[0120] At 433, the electronic device (100) fuses the determinations from global blur detection, local blur detection, and intentional blur detection to make a final determination as to whether the image is degraded by blur.

[0121] Input: Global fuzzy probability P global (x), probability of local blur P local (x) and the probability of intentional ambiguity P int (x)

[0122] Output: Probability P of blur in the image blur (x)

[0123] Figure 4B is a schematic diagram illustrating operations performed to determine the type and intensity of blur using blur probabilities in an input image in accordance with embodiments as disclosed herein.

[0124] The probability that the input image is blurry is determined as a convex combination of the global blur probability, the local blur probability, and the intentional blur probability:

[0125] [Mathematical formula 1]

[0126] P blur (x) = W global P globl (x)+W local P local (x)+W int (1-P int (x))

[0127] Among them, W global (444), W local (445) and W int(426) represents the weights of global blur, local blur and intentional blur corresponding to the blur type finally determined. The weights are determined based on the percentage of pixels involved in the determination for each branch:

[0128] [Mathematical formula 2]

[0129]

[0130] [Mathematical formula 3]

[0131]

[0132]

[0133] Where e represents an exponential function. The weight depends on how many pixels contribute to the decision for that particular branch. That is, if the number of pixels contributing to the local blur decision in the image is high, the weight for the local branch is high.

[0134] In an embodiment, C_global is the fraction of pixels that contribute to the determination of global motion blur, C_local is the fraction of pixels that contribute to the determination of local motion blur, and C_int is the fraction of pixels that contribute to the determination of intentional blur. In addition, W_global, W_local, W_int are the contributions of global motion, local motion, and intentional blur to the final determination, respectively, and e is an exponential function.

[0135] In an embodiment, the contribution of each branch depends on the fraction of pixels that contribute to the determination of that branch.Thus, the fusion based on the parts of the image that contribute to the blur determination allows the input image to be evaluated in terms of the perceptual impact of deblurring on the user.

[0136] Figure 4C is a sequence diagram illustrating operations performed for detecting intentional blur in an input image according to an example.

[0137] exist Figure 4C In , based on the probability of global blur, the probability of local blur and the probability of intentional blur, the blur type is detected as intentional blur.

[0138] [Formula 4]

[0139]

[0140] also, Figure 4C There is less local blur in the thumb region (P_local(x)=0.6), and the effect of deblurring this region has minimal impact on user perception. Therefore, no deblurring is required. This determination is taken when the amount of blur is minimal and the region is also small.

[0141] Figure 4D is a sequence diagram illustrating operations performed for detecting local blur in an input image according to an example.

[0142] exist Figure 4D In , the blur type is detected as local blur based on the probability of global blur, the probability of local blur, and the probability of intentional blur.

[0143] [Formula 5]

[0144]

[0145] In addition, Figure 4D In the image, there is strong local blur in the leg area (P_(x)=0.99), so the effect of deblurring this area has a significant impact on the user perception. Therefore, deblurring is needed. This determination is made when the amount of blur is very high and deblurring helps significantly improve the user experience.

[0146] Figure 4E is a sequence diagram illustrating operations performed for detecting local blur using blur of a foreground in an input image according to an example.

[0147] exist Figure 4E In , based on the probability of global blur, the probability of local blur, and the probability of intentional blur, the blur type is detected as local blur due to the blur in the foreground.

[0148] [Mathematical formula 6]

[0149]

[0150] Figure 4F is a sequence diagram illustrating operations performed for detecting global blur in an input image according to an example.

[0151] exist Figure 4F In , based on the global fuzzy probability, local fuzzy probability and intentional fuzzy probability, the fuzzy type is detected as global fuzzy.

[0152] [Mathematical formula 7]

[0153]

[0154] Figure 4G is a sequence diagram illustrating operations performed for detecting sharpness in an input image according to an example.

[0155] exist Figure 4G In , the blur type is detected as no blur based on the probability of global blur, the probability of local blur, and the probability of intentional blur.

[0156] [Mathematical formula 8]

[0157]

[0158] Figure 4H is a sequence diagram illustrating another example of detecting local blur in an input image according to an example.

[0159] exist Figure 4H In , based on the probability of global blur, the probability of local blur, and the probability of intentional blur, the blur type is detected as local blur due to the blur in the foreground.

[0160] [Mathematical formula 9]

[0161]

[0162] Figure 5 is a schematic diagram illustrating detection of blurred images in a gallery according to an embodiment as disclosed herein.

[0163] The blur detection module (502) has a blur detector (204), wherein the blur detector (204) analyzes the gallery (501) with blurry images. At (503), the blur detection module (502) determines whether blur is detected in the image of the gallery. When blur is detected in the image, the blur detection module (502) requests the deblur engine (504). At (505), the gallery is updated with the deblurred image.

[0164] Fig. 6A is a graph (600) illustrating classification of multiple regions based on entropy according to embodiments as disclosed herein.

[0165] The electronic device (100) measures multiple entropies of multiple regions in an input image, and classifies a first group of regions among the multiple regions having an entropy lower than a first threshold (e.g., "l") as clear regions (603), classifies a second group of regions among the multiple regions having an entropy higher than a second threshold (e.g., "h") as blurred regions (602), and classifies a third group of regions among the multiple regions having an entropy higher than the first threshold and lower than the second threshold as candidate blurred regions (601), wherein the blurred regions (602) are also globally blurred.

[0166] Figure 6B is a sequence diagram illustrating entropy-based fuzzy candidate locations and sharp locations according to an embodiment as disclosed herein.

[0167] The input image (421) is analyzed by pixel-by-pixel entropy calculation (604). The input image is analyzed to generate a blur candidate positioning mask and a sharp positioning mask.

[0168] P(x) — Probability distribution of the neighborhood of x

[0169] H(x) — Entropy at pixel x

[0170] z — the z pixel position in the neighborhood of x (n(x))

[0171] [Formula 10]

[0172]

[0173] In Equation 9, v(z) is the value of pixel z, and v(w) is the value of pixel w in the neighborhood of x.

[0174] The input image (421) describes the process of creating candidate masks and clear masks. As a first step, an entropy mask is calculated by pixel-by-pixel entropy calculation. After blurring, the candidate mask is generated by thresholding as shown in Equation 9. If h> entropy>1, the pixel is marked as a blur candidate. Finally, a clear mask is generated from the entropy mask by thresholding. If l> entropy, the pixel is marked as clear.

[0175] Fig. 7A is a sequence diagram illustrating detection of blur types in an input image according to an embodiment as disclosed herein.

[0176] The input image (421) is processed by blur candidate generation (702). A mask is generated for the input image (421), and the mask includes but is not limited to blur candidate regions (703, 707), blur regions (704, 710), local blur detectors (705, 709), and clear regions (708).

[0177] Figure 7B is a sequence diagram illustrating local fuzzy candidate refinement according to an embodiment as disclosed herein.

[0178] Reference Figure 7B , showing blur detection of a conventional method (701) and a proposed method. In the proposed method and electronic device (100), blur candidate localization (422) identifies blurry "candidate" regions. In addition, a local blur candidate refinement (424) block is allowed to focus on specific regions of the image that may have potential blur and estimate a blur score for each of the candidate regions. The regions affected by the application of deblurring are then selected. Local blur candidate refinement (424) may estimate blur scores for candidate region 1 (702), candidate region 2 (703), candidate region 3 (704), and candidate region 4 (705) as 0.6, 0.85, 0.58, and 0.45, respectively.

[0179] The conventional method (701) only marks the area as blurry / non-blurry, while the proposed method and electronic device (100) first identify the candidates and then select the area with the greatest impact on deblurring. In addition, the conventional method (701) for blur localization misses partially blurred areas, such as candidate area 1 (702) and area 3 (704). In addition, since candidate area 4 (705) is relatively clear, the proposed method and electronic device (100) reject candidate area 4 (705).

[0180] Figure 8 is a flow chart illustrating a content management hub for image quality assessment and image enhancement services according to embodiments as disclosed herein.

[0181] Typically, a gallery (801) of an electronic device (100) includes a plurality of images. These images are analyzed by a content management hub (CMH) (802) that controls "image quality assessment" (803) and "image enhancement services". Image quality assessment (803) includes quality score prediction of media, detection and estimation of degradation in media to improve enhancement. Images are analyzed using intrinsic parameter analysis (804) to perform blur candidate generation, blur classification, and blur estimation.

[0182] At 805 and 806, a tag is generated and contains information describing image quality parameters of the gallery image, such as "blur type" and / or "intensity".

[0183] At 807, the image enhancement service is called to apply the deblur enhancement to produce an enhanced image (808). At 809, an image quality assessment is performed to verify the quality and aesthetic score for the enhanced image (808). When the quality improves, the gallery and media DBs are updated and the tags are updated. The deblurred image (810) can be labeled as "none" blur type. At 811, the gallery can be updated with the enhanced media.

[0184] 9A to 9C is a captured image showing a reset feature in a smartphone according to an embodiment as disclosed herein.

[0185] Fig.9A shows an input image with intentional blur due to being captured in portrait mode. When the user selects the reset picture option, in conventional methods, such as Fig. 9B As shown, since the input image is detected as blurry and thus deblurring is performed, artifacts are generated (901). However, in the proposed method, Fig. 9C As shown, the input image is classified as a sharp image and the background blur is intentional, so no deblurring is performed, thereby avoiding artifacts (902).

[0186] Fig.10is a sequence diagram illustrating blur positioning for controlled artifact-free deblurring according to embodiments as disclosed herein.

[0187] In the proposed method and electronic device (100), an input image (421) is analyzed by blur localization (1002) to generate an entropy mask. At (1003), the blur area in the generated entropy mask is shown. At (1004), the proposed method and electronic device (100) recommend deblurring only the blur area (1003). Image (1005) discloses the input image (421) after deblurring. As shown in 1006, due to the proposed method and electronic device (100), since the deblurring is only operated in the blur area, the clear area remains unchanged. However, in the conventional method, as shown in 1007, the deblurring occurs without blur localization. Therefore, at 1008, the input image (421) is deblurred without localizing the blur. Therefore, at 1009, artifacts are generated due to the deblurring in the clear area.

[0188] Fig.11A is a schematic diagram illustrating the identification of highly blurred images and low blurred images in a gallery and the corresponding recommendation according to an embodiment as disclosed herein.

[0189] At 1102, the electronic device (100) detects a blurred image from a gallery. In addition, the electronic device (100) classifies the detected blurred image into a slightly blurred image (1103) and a highly blurred image (1104). In addition, at 1106, the electronic device (100) suggests a slightly blurred image for reset to deblur the blur in the slightly blurred image. At (1105), the electronic device (100) also suggests a highly blurred image (1104) for cleanup.

[0190] Fig. 11B is a schematic diagram illustrating the operation of a deblurring engine according to an embodiment as disclosed herein.

[0191] At 1107, the input image is subjected to blur candidate localization using the entropy mask to generate blur candidates (e.g. Fig. 11B 1, fuzzy candidate 2, and fuzzy candidate 3 are shown. At 1108, the fuzzy candidates are analyzed to provide estimated fuzziness scores. At 1109, local fuzziness scores are generated based on the estimated fuzziness scores of the fused fuzzy candidates. The estimated fuzziness scores (0.65, 0.9, and 0.7) of the fuzzy candidates are given to the deblurring engine (1111). Because fuzzy candidate 2 has the highest fuzziness compared to the other fuzzy candidates, the deblurring engine (1111) can apply stronger deblurring to fuzzy candidate 2.

[0192] Fig.12is a schematic diagram illustrating identification of subtle motions in IOT applications or monitoring according to an embodiment as disclosed herein.

[0193] At 1202, the preview stream (1201) of the IOT application or monitoring is monitored by the proposed method and the electronic device (100) to determine blur. At 1203, the electronic device (100) detects whether motion is detected in the candidate. At 1205, when motion is detected in the candidate, the electronic device (100) sends a trigger event / notification to the IoT hub. At 1204, when motion is not detected in the candidate, the electronic device (100) continues to monitor the preview stream.

[0194] Furthermore, the proposed method and electronic device (100) identify potential movement candidates for motions of interest to a particular IoT device. Potential movement identification is useful for applications including but not limited to baby monitors, smart home controllers, surveillance feeds, security. Thus, when motions of interest (including small motions) are detected, appropriate notifications and alerts can be triggered.

[0195] Fig.13 is a sequence diagram illustrating identifying subtle motions in IOT applications or monitoring according to an embodiment as disclosed herein.

[0196] At 1301, the electronic device (100) detects a region of interest (ROI) of a preview stream. At 1303, the electronic device (100) detects blur by comparing a blur probability of an input image from the preview stream with a threshold.

[0197] When the blur probability is less than a threshold value (e.g., 0.5), the electronic device continues (1302) to capture the next image. At 1304, when the blur probability is greater than a threshold value (e.g., 0.5), blur and motion are detected in the input image. At 1305, the electronic device (100) sends a notification of potential suspicious motion. In addition, at 1306, the electronic device (100) determines whether the detected motion requires action. At 1307, when the detected motion requires action, the electronic device (100) takes the required action. At 1308, when the detected motion does not require action, the electronic device (100) does not take the required action.

[0198] Fig.14 is a schematic diagram illustrating assistance for automatic capture of factory camera calibration according to embodiments as disclosed herein.

[0199] At 1401, the camera captures a fixed pattern at different angles. At 1402, the electronic device (100) detects blur in the captured input image. At 1403, the electronic device (100) determines whether local motion is detected. At 1404, when no local motion is detected, the electronic device (100) continues to capture the next angle or ends capturing. At 1405, the electronic device (100) discards the captured input image and retakes when local motion is detected.

[0200] The proposed method and electronic device (100) identify blurry images in automatic capture scenarios (such as factory camera calibration), wherein accurate and fast local / global blur detection can be used to discard and recapture blurry images. Blurred images may cause inaccurate calibration parameters and cannot be used for processes such as factory camera calibration.

[0201] Fig.15 is a sequence diagram illustrating assistance for automatic capture of factory camera calibration according to embodiments as disclosed herein.

[0202] At 1501, the electronic device (100) performs an automatic capture scene, such as factory camera calibration. At 1502, the electronic device (100) determines whether blur is detected in the input image during camera calibration. The electronic device (100) determines whether the blur probability is less than a threshold. At 1503, when the blur probability is less than the threshold, the electronic device (100) continues to capture images. At 1505, the electronic device (100) determines whether a retake is required when the blur probability is not less than the threshold, and performs a retake if a retake is required.

[0203] The foregoing description of specific embodiments will so fully reveal the general nature of the embodiments herein that others can easily modify and / or adapt such specific embodiments for various applications without departing from the general concept by applying current knowledge, and therefore, such adaptations and modifications should and are intended to be understood as being within the meaning and scope of equivalents of the disclosed embodiments. It should be understood that the wording or terminology employed herein is for descriptive and not limiting purposes. Therefore, although the embodiments herein have been described according to preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced by modification within the scope of the embodiments as described herein.

Claims

1. A method for detecting blur in an input image, include: Detecting, by an electronic device, one or more candidate blur regions among a plurality of regions in the input image; determining, by the electronic device, confidence scores for the one or more candidate blur regions in the input image; determining, by the electronic device, a confidence score for global blur in the input image; determining, by the electronic device, a confidence score for intentional blur in the input image; as well as At least a blur type and an intensity of the blur type in the input image are detected by the electronic device based on the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

2. The method according to claim 1, in, The step of detecting, by the electronic device, the one or more candidate blur regions in the input image comprises: measuring, by the electronic device, a plurality of entropies of the plurality of regions in the input image; and The electronic device classifies a first group of regions among the multiple regions having an entropy lower than a first threshold as clear, a second group of regions among the multiple regions having an entropy higher than a second threshold as blurred, and a third group of regions among the multiple regions having an entropy higher than the first threshold and lower than the second threshold as candidate blurred regions.

3. The method according to claim 1, in, The step of determining, by the electronic device, a confidence score of the intentional blur in the input image comprises: measuring, by the electronic device, a plurality of entropies of the plurality of regions in the input image; The electronic device determines, as a low value, each of the plurality of entropies near the center of the input image, and determines, as a high value, each of the plurality of entropies near the edge of the input image; and A confidence score of the intentional blur is determined by the electronic device to be high.

4. The method according to claim 1, in, The step of determining, by the electronic device, confidence scores of the one or more candidate blur regions in the input image comprises: generating, by the electronic device, a segmented image indicating the one or more candidate blurry regions by fusing a candidate blurry region mask with the input image, wherein the candidate blurry region mask indicates entropies of the plurality of regions in the input image; and A confidence score for each of the one or more candidate blur regions in the segmented image is determined by the electronic device.

5. The method according to claim 1, in, The step of determining, by the electronic device, a confidence score of the global blur in the input image comprises: analyzing the input image as a whole by the electronic device; and A confidence score for the global blur in the input image is determined, by the electronic device, based on a level of presence of the global blur in the input image.

6. The method according to claim 1, in, The step of detecting, by the electronic device, the blur type and the intensity of the blur type in at least the input image using the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur comprises: Calculating, by the electronic device, a first weight for confidence scores for the one or more candidate blur regions based on a percentage of pixels associated with the one or more candidate blur regions, a second weight for the confidence score for the global blur based on a percentage of pixels associated with a global blur region, and a third weight for the confidence score for the intentional blur based on a percentage of pixels associated with an intentional blur region; and The blur type and the intensity of the blur type in the input image are detected using the first weight, the second weight, the third weight, the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

7. The method according to claim 1, include: Determining, by the electronic device, that at least the blur type and the intensity of the blur type satisfy a blur threshold; as well as The electronic device performs at least one of the following operations: displaying a recommendation on the electronic device, wherein the recommendation is related to at least one of deleting the input image, enhancing the input image, deblurring the input image, and recapturing the input image, and A tag including an image quality parameter including at least one of the blur type and an intensity of the blur type is generated and stored in association with the input image in a media database.

8. A blur correction management method for an input image in an electronic device, include: detecting, by the electronic device, global blur in the input image requiring blur correction; estimating, by the electronic device, a global ambiguity probability as a measure of a confidence level that the global ambiguity exists; detecting, by the electronic device, one or more local regions having candidate blurs in the input image; measuring, by the electronic device, entropy in the detected one or more local regions; selecting, by the electronic device, one or more local regions having a predefined entropy range for local blur correction; estimating, by the electronic device, a local blur probability as a measure of a confidence level that the local blur exists; detecting, by the electronic device, one or more clear regions comprising a predefined entropy range; estimating, by the electronic device, an intentional blur probability as a measure of a confidence level that there is intentional user-introduced blur; as well as The global blur probability, the local blur probability, and the intentional blur probability are fused, by the electronic device, to generate a determination regarding correcting the global blur and / or the local blur.

9. An electronic device for detecting blur in an input image, include: Memory; a processor coupled to the memory; as well as A blur detector is coupled to the memory and the processor (203) and is configured to: detecting one or more candidate blurry regions among a plurality of regions in the input image; determining confidence scores for the one or more candidate blurry regions in the input image; determining a confidence score for a global blur in the input image; determining a confidence score for intentional blur in the input image; as well as At least a blur type and an intensity of the blur type in the input image are detected based on the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

10. The electronic device according to claim 9, in, The step of detecting the one or more candidate blurry regions in the input image comprises: measuring a plurality of entropies of the plurality of regions in the input image; and A first group of regions among the multiple regions having an entropy lower than a first threshold is classified as clear, a second group of regions among the multiple regions having an entropy higher than a second threshold is classified as blurred, and a third group of regions among the multiple regions having an entropy higher than the first threshold and lower than the second threshold is classified as candidate blurred regions.

11. The electronic device according to claim 9, in, The step of determining a confidence score for the intentional blur in the input image comprises: measuring a plurality of entropies of the plurality of regions in the input image; Determine a value of each of the plurality of entropies close to a center of the input image as low, and determine a value of each of the plurality of entropies close to an edge of the input image as high; and The confidence score of the intentional blur is determined to be high.

12. The electronic device according to claim 9, in, The step of determining confidence scores of the one or more candidate blurry regions in the input image comprises: generating a segmented image indicating the one or more candidate blurry regions by fusing a candidate blurry region mask with the input image, wherein the candidate blurry region mask indicates entropies of the plurality of regions in the input image; and A confidence score is determined for each of the one or more candidate blurry regions in the segmented image.

13. The electronic device according to claim 9, in, The step of determining a confidence score for the global blur in the input image comprises: analyzing the input image as a whole; and Based on a level of presence of the global blur in the input image, a confidence score for the global blur in the input image is determined.

14. The electronic device according to claim 9, in, Using the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur to detect the blur type and the intensity of the blur type in at least the input image includes: Calculating a first weight for a confidence score for the one or more candidate blur regions based on a percentage of pixels associated with the one or more candidate blur regions, calculating a second weight for the confidence score for the global blur based on a percentage of pixels associated with a global blur region, and calculating a third weight for the confidence score for the intentional blur based on a percentage of pixels associated with an intentional blur region; and The blur type and the intensity of the blur type in the input image are detected using the first weight, the second weight, the third weight, the confidence scores of the one or more candidate blur regions, the confidence score of the global blur, and the confidence score of the intentional blur.

15. The electronic device according to claim 9, in, The blur detector (204) is configured to: determining that at least the blur type and the intensity of the blur type satisfy a blur threshold; as well as Do at least one of the following: displaying a recommendation on the electronic device, wherein the recommendation is related to at least one of deleting the input image, enhancing the input image, deblurring the input image, and recapturing the input image, and A tag including an image quality parameter including at least one of the blur type and an intensity of the blur type is generated and stored in association with the input image in a media database.

Citation Information

Cited By

  • A fast method and system for simultaneously predicting video blur and glint pixel by pixel

    CN122714947A