Method for improving glare and applications thereof

Abstract

The application discloses a method for improving glare, comprising: collecting an initial image including glare, pre-processing the initial image to obtain a pre-processed image; performing global binarization processing on the pre-processed image to generate a binary image; constructing a full-black mask with the same size as the binary image, indicating a region to be repaired on the full-black mask to generate a final mask; and outputting a repaired image based on the final mask. The application also provides a device for improving glare, a storage medium, a computer program product and an electronic device. The application can remove different types of glare as a whole without increasing any hardware cost and is suitable for any type of image sensor.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method, apparatus, storage medium, computer program product, and electronic device for improving glare. Background Technology

[0002] Glare refers to the phenomenon in backlit photos where strong light directly enters the lens, causing the area around the light source to appear overly bright, or creating a halo or fogging effect. In mobile phone cameras, due to differences in image sensors, some lenses have a lower degree of glare reduction, resulting in more severe glare.

[0003] Currently, hardware coating technology is generally used to reduce glare. Specifically, after light enters the camera module, it is reflected between the cover glass and the lens, between lenses, between the lens and the infrared filter, and between the infrared filter and the image sensor. A coating is then applied to the surface of the device that generates the reflected light to improve glare.

[0004] However, improving glare through coating is too costly, and different coating processes are required to better reduce glare when different types of glare are present, making it impossible to completely remove glare. Summary of the Invention

[0005] This specification provides an embodiment of a method for improving glare, aiming to save costs and eliminate glare overall. The method for improving glare includes:

[0006] An initial image including glare is acquired; the initial image is preprocessed to obtain a preprocessed image; the preprocessed image is globally binarized to generate a binary image; a full-black mask with the same size as the binary image is constructed, and the area to be repaired is indicated on the full-black mask to generate a final mask; based on the final mask, the repaired image is output.

[0007] Further, preprocessing the initial image includes: converting the initial image into a grayscale image, and blurring the grayscale image using a Gaussian matrix to obtain a blurred image.

[0008] Further, the global binarization processing of the preprocessed image includes: setting a first pixel threshold, resetting the regions in the preprocessed image larger than the first pixel threshold to pure white, and resetting the regions in the preprocessed image smaller than or equal to the first pixel threshold to pure black.

[0009] Preferably, the first pixel threshold is set to 180.

[0010] Furthermore, after constructing the all-black mask, the connected component information of the binary image is obtained, and the labels of all regions in the binary image are obtained according to the connected component information. The non-background labels are traversed, and if the number of pixels in the region corresponding to the label is greater than the set second pixel threshold, the corresponding region in the all-black mask is marked as white.

[0011] Furthermore, the output repaired image includes filling the glare area using an edge-aware gradient method.

[0012] This specification also provides an embodiment of a device for improving glare, comprising:

[0013] The preprocessing module is used to acquire an initial image including glare, and to preprocess the initial image to obtain a preprocessed image;

[0014] The binarization processing module is used to perform global binarization processing on the preprocessed image to generate a binary image;

[0015] The mask generation module is used to construct a full-black mask with the same size as the binary image, indicate the area to be repaired on the full-black mask, and generate the final mask.

[0016] The image restoration module is used to output a restored image based on the final mask.

[0017] This specification also provides a storage medium storing a computer program, characterized in that the computer program, when executed by a processor, performs the steps of the above-described method.

[0018] This specification also provides a computer program product that stores at least one instruction, which, when executed by a processor, implements the steps of the above-described method.

[0019] This specification also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, the computer program being loaded by the processor and executing the steps of the above-described method.

[0020] This invention, through global binarization and independent masking, can remove different types of glare without increasing hardware costs and is applicable to any type of image sensor. Attached Figure Description

[0021] Figure 1 A schematic flowchart illustrating a method for improving glare provided in an embodiment of this specification;

[0022] Figure 2A schematic diagram of a device for improving glare provided in the embodiments of this specification;

[0023] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0025] Please see Figure 1 This diagram illustrates a method for improving glare, as provided in an embodiment of this specification. Specifically, the method for improving glare in this embodiment may include the following steps:

[0026] S1: Acquire an initial image including glare, preprocess the initial image to obtain a preprocessed image.

[0027] In one or more embodiments of this specification, preprocessing the initial image may specifically include: converting the initial image into a grayscale image, and blurring the grayscale image using a Gaussian matrix to obtain a blurred image, which is the preprocessed image. Using a Gaussian matrix to blur the image can reduce noise.

[0028] S2 performs global binarization on the preprocessed image to generate a binary image.

[0029] In one or more embodiments of this specification, global binarization of a preprocessed image may specifically include: setting a first pixel threshold, resetting regions in the preprocessed image larger than the first pixel threshold to pure white, and resetting regions in the preprocessed image smaller than or equal to the first pixel threshold to pure black, thereby obtaining a binary image.

[0030] In one or more embodiments of this specification, the first pixel threshold is set to 180. That is, when the pixel value of a certain area is greater than 180, that area is a glare area, and the glare area is set to full white, while other areas are set to full black.

[0031] Since the resulting binary image may contain small patches of noise, in one or more embodiments of this specification, the binary image may be subjected to erosion and dilation. More specifically, a 3×3 rectangular kernel can be used to perform two erosion operations on the binary image to eliminate noise points with an area less than 10 pixels, and then a 5×5 circular kernel can be used to perform three dilation operations on the binary image to connect broken areas and smooth edges.

[0032] S3. Construct a full-black mask with the same size as the binary image, indicate the area to be repaired on the full-black mask, and generate the final mask.

[0033] In one or more embodiments of this specification, after constructing a full black mask, the connected component information of the binary image is obtained, and the labels of all regions in the binary image are obtained based on the connected component information. The non-background labels are traversed. If the number of pixels in the region corresponding to the label is greater than the set second pixel threshold, the corresponding region in the full black mask is marked as white. The white region is the region to be repaired, and the mask after marking is the final mask.

[0034] In one or more embodiments of this specification, the second pixel threshold may be set to 100 or other suitable values.

[0035] S4, based on the final mask, outputs the repaired image.

[0036] In one or more embodiments of this specification, an edge-aware gradient method can be used to fill glare areas to achieve natural restoration. It should be understood that other methods can also be used to restore images, selected according to actual needs.

[0037] Please see Figure 2 This is a schematic diagram of a glare-reducing device provided in an embodiment of this specification. This glare-reducing device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. For example... Figure 2 As shown, in one or more embodiments of this specification, the glare-improving device 1 includes a preprocessing module 11, a binarization processing module 12, a mask generation module 13, and an image restoration module 14, specifically including:

[0038] The preprocessing module 11 is used to acquire an initial image including glare, preprocess the initial image, and obtain a preprocessed image.

[0039] Binarization module 12 is used to perform global binarization on the preprocessed image to generate a binary image;

[0040] The mask generation module 13 is used to construct a full-black mask with the same size as the binary image, indicate the area to be repaired on the full-black mask, and generate the final mask;

[0041] Image restoration module 14 is used to output a restored image based on the final mask.

[0042] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0043] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figure 1 The method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figure 1 The specific details of the illustrated embodiments will not be elaborated here.

[0044] This specification also provides a computer program product that stores at least one instruction, which is loaded and executed by the processor as described above. Figure 1 The method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figure 1 The specific details of the illustrated embodiments will not be elaborated here.

[0045] The embodiments in this specification also provide Figure 3 The diagram shows the structure of the electronic device. Figure 3 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile storage, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to implement the aforementioned network video skinning method.

[0046] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0047] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0048] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0049] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0050] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0051] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0052] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0053] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0054] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0055] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0056] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0057] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0058] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0059] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0061] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0062] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for improving glare, characterized in that, include: An initial image including glare is acquired, and the initial image is preprocessed to obtain a preprocessed image; The preprocessed image is subjected to global binarization to generate a binary image; Construct a full-black mask with the same size as the binary image, indicate the area to be repaired on the full-black mask, and generate the final mask; Based on the final mask, the repaired image is output.

2. The method for improving glare according to claim 1, characterized in that, Preprocessing the initial image includes: converting the initial image into a grayscale image, and blurring the grayscale image using a Gaussian matrix to obtain a blurred image.

3. The method for improving glare according to claim 1, characterized in that, The global binarization process for the preprocessed image includes: setting a first pixel threshold, resetting regions in the preprocessed image larger than the first pixel threshold to pure white, and resetting regions in the preprocessed image smaller than or equal to the first pixel threshold to pure black.

4. The method for improving glare according to claim 3, characterized in that, The first pixel threshold is set to 180.

5. The method for improving glare according to claim 1, characterized in that, After constructing the all-black mask, the connected component information of the binary image is obtained. Based on the connected component information, the labels of all regions in the binary image are obtained. Non-background labels are traversed. If the number of pixels in the region corresponding to the label is greater than the set second pixel threshold, the corresponding region in the all-black mask is marked as white.

6. The method for improving glare according to claim 1, characterized in that, The output restored image includes filling the glare area using an edge-aware gradient method.

7. A device for improving glare, characterized in that, include: The preprocessing module is used to acquire an initial image including glare, and to preprocess the initial image to obtain a preprocessed image; The binarization processing module is used to perform global binarization processing on the preprocessed image to generate a binary image; The mask generation module is used to construct a full-black mask with the same size as the binary image, indicate the area to be repaired on the full-black mask, and generate the final mask. The image restoration module is used to output a restored image based on the final mask.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer program product having at least one instruction stored thereon, characterized in that, When the at least one instruction is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

10. An electronic device, comprising: A processor and a memory, characterized in that the memory stores a computer program, which is loaded by the processor and executes the steps of the method as claimed in any one of claims 1 to 6.

Images