Definition evaluation method and device for microlens array light field image

Abstract

The invention relates to a definition evaluation method and device for a microlens array light field image. The method comprises the following steps: determining a micro-lens array light field image to be processed and a micro-lens array white image; according to the micro lens array light field image and the micro lens array white image, single micro lens mask data, single micro lens gray scale image data and micro lens center gray scale image data are obtained; obtaining standard deviation data of a micro-lens array light field image according to the single micro-lens mask data and the single micro-lens gray scale image data; obtaining object edge area mask data according to the standard deviation data of the micro lens array light field image; obtaining the number of microlenses in the object edge area according to the mask data in the object edge area; obtaining edge response data of the object edge region according to the micro-lens center grayscale image data and the object edge region mask data; and dividing the sum of the edge response data of the object edge area by the number of the microlenses in the object edge area to obtain the definition of the light field image of the microlens array.

Description

Technical Field

[0001] This application relates to the field of light field data, specifically to a method and apparatus for evaluating the sharpness of light field images of microlens arrays. Background Technology

[0002] Light field technology has developed rapidly and received widespread attention both domestically and internationally in recent years. Light field data acquisition equipment is mainly divided into camera arrays and microlens array single cameras. Currently, single cameras based on microlens arrays are more commonly used and have greater application value. With the explosive growth of camera applications, higher demands are being placed on the automation level of cameras. Image sharpness evaluation methods are a key technology in the camera autofocus process.

[0003] Currently, common techniques for evaluating the image sharpness of microlenses include the following: MTF (Modulation Transfer Function) analysis: MTF is a quantitative method for evaluating image sharpness, measuring the characteristics of details at different scales in an image. PSF (Point Spread Function) analysis: PSF is a function used to describe the imaging quality of an optical system, representing the blurring caused by a point light source passing through the optical system. By analyzing the difference in PSF curves before and after microlensing, the degree of improvement in image sharpness by the microlens can be evaluated. Resolution testing: Resolution is one of the important indicators for evaluating image sharpness. Resolution test images (such as checkerboard patterns, lines, etc.) can be used to compare the image resolution before and after microlensing to assess the impact of microlensing on image sharpness.

[0004] However, existing technologies directly analyze and evaluate the image as a whole, and do not make good use of the detailed information of the microlens image when evaluating the image, resulting in inaccurate evaluation results of sharpness. Summary of the Invention

[0005] To address the aforementioned problems, the purpose of this application is to provide a method and apparatus for evaluating the sharpness of light field images of microlens arrays.

[0006] According to a first aspect of the present application, a method for evaluating the sharpness of a microlens array light field image is provided, comprising: determining a microlens array light field image to be processed and a microlens array white image; obtaining individual microlens mask data, individual microlens grayscale image data, and microlens center grayscale image data based on the microlens array light field image and the microlens array white image; calculating, one by one, the standard deviation data of the microlens array light field image based on the individual microlens mask data and the individual microlens grayscale image data; obtaining object edge region mask data based on the standard deviation data of the microlens array light field image using binarization calculation; obtaining the number of microlenses in the object edge region based on the object edge region mask data; obtaining edge response data of the object edge region based on the microlens center grayscale image data and the object edge region mask data using convolution calculation and mask processing; and dividing the sum of the edge response data of the object edge region by the number of microlenses in the object edge region to obtain the sharpness of the microlens array light field image.

[0007] Furthermore, the step of dividing the sum of the edge response data of the object's edge region by the number of microlenses in the object's edge region to obtain the sharpness of the microlens array light field image includes: the edge response data of the object's edge region is matrix data based on convolution calculation and masking processing; the sharpness is calculated based on the edge response data of the object's edge region and the number of microlenses in the object's edge region, using the following formula:

[0008]

[0009] In the formula, I gl,margin (m,n) represents the edge response data matrix of the object's edge region, ||M margin (m,n)||0 represents the number of microlenses in the edge region of the object, and ‖‖0 represents the zero norm of the vector, used to calculate M. margin The number of non-zero elements in (m,n).

[0010] Furthermore, the step of obtaining edge response data of the object's edge region based on the grayscale image data at the center of the microlens and the mask data of the object's edge region, using convolution calculation and mask processing, includes: calculating an extended template based on a preset Gaussian filter convolution template and a preset Laplacian operator; performing convolution calculation on the grayscale image data at the center of the microlens based on the extended template to obtain convolution result data; and performing mask processing on the convolution result data based on the mask data of the object's edge region to obtain edge response data of the object's edge region.

[0011] Furthermore, the step of calculating the standard deviation data of the light field image of the microlens array based on the individual microlens mask data and the individual microlens grayscale image data includes: calculating the grayscale mean based on the individual microlens mask data and the individual microlens grayscale image data, using the following formula:

[0012]

[0013] In the formula, I microlens(m,n) (x,y) represents the grayscale image data of a single microlens, M binary(m,n) (x,y) represents the data for a single microlens mask, μ (m,n) Let m and n represent the mean grayscale value, m be the m rows and n columns of the microlens, and x and y be the x rows and y columns of the pixels. The standard deviation is calculated based on the individual microlens mask data, the individual microlens grayscale image data, and the mean grayscale value, using the following formula:

[0014]

[0015] In the formula, σ (m,n) The standard deviation is represented; based on the standard deviation, the standard deviation data of the microlens array light field image is obtained.

[0016] Furthermore, the step of obtaining the object edge region mask data based on the standard deviation data of the microlens array light field image and using binarization calculation includes: performing binarization calculation on the standard deviation map to obtain the object edge region mask data, as shown in the following formula:

[0017]

[0018] In the formula, I σ (m,n) represents the standard deviation data of the light field image of the microlens array, λ σ The threshold value represents the standard deviation of grayscale, and m,n represents the number of rows and columns of the microlens.

[0019] According to a second aspect of the embodiments of this application, a sharpness evaluation device for a microlens array light field image is provided, comprising: an image determination module, configured to determine a microlens array light field image to be processed and a microlens array white image; a data acquisition module, configured to obtain individual microlens mask data, individual microlens grayscale image data, and microlens center grayscale image data based on the microlens array light field image and the microlens array white image; a standard deviation calculation module, configured to calculate, one by one, the standard deviation data of the microlens array light field image based on the individual microlens mask data and the individual microlens grayscale image data; and a binarization calculation module, configured to calculate, based on the... The standard deviation data of the light field image of the microlens array is used to calculate the object edge region mask data based on binarization. A microlens number calculation module is used to calculate the number of microlenses in the object edge region based on the object edge region mask data. An edge response data calculation module is used to calculate the edge response data of the object edge region based on the microlens center grayscale image data and the object edge region mask data, using convolution calculation and mask processing. A sharpness calculation module is used to divide the sum of the edge response data of the object edge region by the number of microlenses in the object edge region to obtain the sharpness of the microlens array light field image.

[0020] Furthermore, the sharpness calculation module is specifically used for: the edge response data of the object's edge region being matrix data based on convolution calculation and masking; and calculating the sharpness based on the edge response data of the object's edge region and the number of microlenses in the object's edge region, using the following formula:

[0021]

[0022] In the formula, I gl,margin (m,n) represents the edge response data matrix of the object's edge region, ||M margin (m,n)||0 represents the number of microlenses in the edge region of the object, and ‖‖0 represents the zero norm of the vector, used to calculate M. margin The number of non-zero elements in (m,n).

[0023] Furthermore, the edge response data calculation module for the object edge region is specifically used for: calculating an extended template based on a preset Gaussian filter convolution template and a preset Laplacian operator; performing convolution calculation on the grayscale image data at the center of the microlens based on the extended template to obtain convolution result data; and performing masking processing on the convolution result data based on the object edge region mask data to obtain edge response data for the object edge region.

[0024] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; and a processor configured to read executable instructions from the memory and execute the instructions to implement the sharpness evaluation method for a microlens array light field image provided in the first aspect of this application.

[0025] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a processor, implement the steps of the sharpness evaluation method for a microlens array light field image provided in the first aspect of this application.

[0026] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0027] This application calculates the standard deviation of the light field image of a microlens array based on individual microlens mask data and individual microlens grayscale image data. This standard deviation is used to determine the presence of edges in a single image using binarization, forming object edge region mask data. The number of microlenses in the object edge region is then calculated. The number of microlenses in the object edge region characterizes the focusing degree of the light field image of the microlens array; fewer microlenses in the object edge region result in better focusing and higher sharpness. By performing convolution calculations on the grayscale image data of the microlens center, areas with rich image details are highlighted. Through masking, region of interest data is extracted to obtain edge response data of the object edge region. This edge response data characterizes the richness of edge details; richer details and higher values ​​indicate sharper edges. The sharpness of the light field image of the microlens array is evaluated by considering the number of edges and the richness of edge details, thus accurately assessing the sharpness of the light field image of the microlens array. This application designs a method for calculating the number of edges for a single microlens feature and combines it with the richness of edge details to improve the rationality of sharpness evaluation and help improve the quality of camera autofocus. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating the use of a sharpness evaluation method for a microlens array light field image according to an exemplary embodiment;

[0030] Figure 2 The image is a microlens image of a sharpness evaluation method for a microlens array light field image according to an exemplary embodiment;

[0031] Figure 3 This is an apparatus diagram illustrating a method for evaluating the sharpness of a light field image of a microlens array, according to an exemplary embodiment. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0033] Exemplary Method 1

[0034] like Figure 1 As shown in the embodiments of this application, a method for evaluating the sharpness of a light field image of a microlens array is provided, including the following steps:

[0035] In step S101, the light field image of the microlens array to be processed and the white image of the microlens array are determined.

[0036] Acquire the light field image of the microlens array to be processed.

[0037] Prepare a white background. In a well-lit indoor environment, manually focus using a microlens array single camera to obtain a white image (grayscale image) of the microlens array captured by the single camera.

[0038] In step S102, based on the light field image of the microlens array and the white image of the microlens array, individual microlens mask data, individual microlens grayscale image data, and microlens center grayscale image data are obtained.

[0039] Based on the white image of the microlens array, the mask data and center pixel coordinates of all microlenses are obtained. The white image (grayscale image) captured by the single camera of the microlens array is binarized to obtain the mask image. Region growing is performed on the mask image to obtain a series of individual microlens masks. Combining the white image of the microlens array and the individual microlens mask images yields a single microlens white image. Calculating the maximum grayscale value of each individual microlens white image reveals the center positions of a series of microlenses.

[0040] Based on the light field image of the microlens array, combined with the microlens mask data and the center pixel coordinates, grayscale image data of individual microlenses and grayscale image data of the microlens center are obtained. The microlens array used in this application is arranged in a horizontally and vertically uniform manner; therefore, the center position of the microlens array can be stored in a matrix. Based on the focusing area, the center position of the microlens array, and the light field image of the microlens array, grayscale image data of the microlens center and the corresponding individual microlens mask data and individual microlens grayscale image data are filtered and generated.

[0041] In step S103, the standard deviation data of the light field image of the microlens array is calculated one by one based on the individual microlens mask data and the individual microlens grayscale image data.

[0042] The mean grayscale value is calculated based on the individual microlens mask data and the individual microlens grayscale image data, using the following formula:

[0043]

[0044] In the formula, I microlens(m,n) (x,y) represents the grayscale image data of a single microlens, M binary(m,n) (x,y) represents the data for a single microlens mask, μ (m,n) The grayscale value is represented by m and n, where m and n represent the number of rows and columns of the microlens, and x and y represent the number of rows and columns of the pixels.

[0045] The standard deviation is calculated based on the individual microlens mask data, the individual microlens grayscale image data, and the grayscale mean, using the following formula:

[0046]

[0047] In the formula, σ (m,n) It represents the standard deviation.

[0048] The standard deviation is used to characterize the dispersion of gray values, so as to determine whether there are edges in a single microlens image.

[0049] Based on the aforementioned standard deviation, the standard deviation data of the microlens array light field image is obtained. The standard deviation data is matrix data, and the formula is as follows:

[0050] I σ (m,n)=σ (m,n)

[0051] In step S104, based on the standard deviation data of the light field image of the microlens array, the mask data of the object edge region is obtained by binarization calculation.

[0052] The standard deviation map is binarized to obtain the mask data of the object edge region, as shown in the following formula:

[0053]

[0054] In the formula, I σ (m,n) represents the standard deviation data of the light field image of the microlens array, λ σ The threshold value represents the standard deviation of grayscale, and m,n represents the number of rows and columns of the microlens.

[0055] A grayscale standard deviation threshold is set. Values ​​greater than or equal to this threshold are considered to indicate the presence of edges in a single microlens image and are assigned a value of 1. Values ​​less than this threshold are considered to indicate the absence of edges in a single microlens image and are assigned a value of 0. This yields the object edge region mask data indicating the presence of edges.

[0056] In step S105, the number of microlenses in the object edge region is obtained based on the mask data of the object edge region.

[0057] Based on the object edge region mask data M margin (m,n), calculate the number of values ​​assigned as 1 in the data to obtain the number of microlenses in the object's edge region ||M margin (m,n)||0, ‖‖0 denotes the zero norm of the vector, used to calculate M. margin The number of non-zero elements in (m,n).

[0058] In step S106, based on the grayscale image data of the microlens center and the mask data of the object edge region, edge response data of the object edge region is obtained by convolution calculation and mask processing.

[0059] The extended template is calculated based on the preset Gaussian filter convolution template and the preset Laplacian operator.

[0060] Since microlens images may contain some noise, and the subtle differences between each microlens image can also cause noise, it is necessary to use Gaussian filtering to suppress noise in the grayscale image data of the microlens center.

[0061] The Laplacian operator is the simplest isotropic differential operator and can highlight regions with rapid changes in grayscale. High-resolution images contain rich edge details, so the Laplacian operator can respond to regions with rich details.

[0062] The extended template is obtained by multiplying the Gaussian filter convolution matrix template with the Laplacian operator 8-neighborhood matrix template, as shown in Table 1:

[0063] Table 1 Extended Template

[0064] 1 / 16 3 / 16 4 / 16 3 / 16 1 / 16 3 / 16 0 -6 / 16 0 3 / 16 4 / 16 -6 / 16 -20 / 16 -6 / 16 4 / 16 3 / 16 0 -6 / 16 0 3 / 16 1 / 16 3 / 16 4 / 16 3 / 16 1 / 16

[0065] Based on the extended template, the grayscale image data at the center of the microlens is convolved and its absolute value is taken to obtain the convolution result data I. gl,abs (m,n).

[0066] Based on the object edge region mask data, the convolution result data is masked to obtain the edge response data map of the object edge region.

[0067] Based on the object edge region mask data M margin (m,n), for the convolution result data I gl,abs The region of interest is extracted by masking (m,n), and the edge response data of the generated object edge region is denoted as I. gl,margin (m,n), the formula is as follows:

[0068]

[0069] The edge response of an object characterizes the richness of edge details after noise removal; the richer the details and the larger the value, the clearer the edge.

[0070] In step S107, the sum of the edge response data of the object edge region is divided by the number of microlenses in the object edge region to obtain the clarity of the microlens array light field image.

[0071] In the embodiments of this application, sharpness means the average edge response of the object's edge region.

[0072] The edge response data of the object's edge region is matrix data based on convolution calculation and masking processing;

[0073] Sharpness, or the average edge response value of the edge region in the light field image, is calculated based on the edge response data of the object's edge region and the number of microlenses in the object's edge region. The formula is as follows:

[0074]

[0075] In the formula, I gl,margin (m,n) represents the edge response data matrix of the object's edge region, ||M margin (m,n)||0 represents the number of microlenses in the edge region of the object, and ‖‖0 represents the zero norm of the vector, used to calculate M. margin The number of non-zero elements in (m,n). The numerator is the sum of the object edge responses in the area to be focused, which characterizes the richness of edge details. The sharper the edge, the larger the value, and it is directly proportional to the image sharpness. The denominator is the number of individual microlens grayscale images occupied by the object edge area. The sharper the edge, the smaller the number, and it is inversely proportional to the image sharpness.

[0076] Specifically, a microlens image is composed of individual microlens grayscale images. When a single microlens grayscale image is pure black or pure white, it represents the image background or object. The presence of both black and white indicates the existence of edges. After edge localization, the edge regions of the microlens image are... Figure 2 As shown, by comparing the focused and unfocused areas of the object edge in the microlens image, the object edge appears only in a small number of single microlens images in the focused image, while it appears repeatedly in the unfocused image. This indicates that the degree of focus of the microlens image is inversely proportional to the number of single microlens images in which the object edge appears. In other words, the fewer the number of single microlens images in which the object edge appears, the clearer the image.

[0077] In summary, by combining the two values ​​mentioned above, image sharpness can be evaluated based on the number of individual microlens images, thereby making full use of the detailed information in the microlens images and increasing the rationality of the sharpness evaluation.

[0078] Exemplary device

[0079] In the embodiments of this application, such as Figure 3 It includes an image determination module 301, a data acquisition module 302, a standard deviation calculation module 303, a binarization calculation module 304, an edge microlens quantity calculation module 305, an edge response data calculation module 306 for the object edge region, and a sharpness calculation module 307.

[0080] Image determination module 301 is used to determine the light field image of the microlens array to be processed and the white image of the microlens array;

[0081] The data acquisition module 302 obtains individual microlens mask data, individual microlens grayscale image data, and microlens center grayscale image data based on the light field image of the microlens array and the white image of the microlens array.

[0082] The standard deviation calculation module 303 is used to calculate the standard deviation data of the light field image of the microlens array one by one based on the individual microlens mask data and the individual microlens grayscale image data, and to calculate the grayscale mean based on the individual microlens mask data and the individual microlens grayscale image data, using the following formula:

[0083]

[0084] In the formula, I microlens(m,n) (x,y) represents the grayscale image data of a single microlens, M binary(m,n) (x,y) represents the data for a single microlens mask, μ (m,n)Let m and n represent the mean grayscale value, m be the m rows and n columns of the microlens, and x and y be the x rows and y columns of the pixels. The standard deviation is calculated based on the individual microlens mask data, the individual microlens grayscale image data, and the mean grayscale value, using the following formula:

[0085]

[0086] In the formula, σ (m,n) The standard deviation is represented; based on the standard deviation, the standard deviation data of the microlens array light field image is obtained.

[0087] The binarization calculation module 304 is used to obtain the object edge region mask data based on the standard deviation data of the light field image of the microlens array and binarization calculation. The object edge region mask data is obtained by binarizing the standard deviation map, as shown in the following formula:

[0088]

[0089] In the formula, I σ (m,n) represents the standard deviation data of the light field image of the microlens array, λ σ The threshold value represents the standard deviation of grayscale, and m,n represents the number of rows and columns of the microlens.

[0090] The edge microlens number calculation module 305 is used to obtain the number of microlenses in the edge region of the object based on the mask data of the object edge region;

[0091] The edge response data calculation module 306 for the object edge region is used to obtain edge response data of the object edge region based on the grayscale image data of the microlens center and the mask data of the object edge region, through convolution calculation and mask processing; calculate an extended template based on a preset Gaussian filter convolution template and a preset Laplacian operator; perform convolution calculation on the grayscale image data of the microlens center based on the extended template to obtain convolution result data; and perform mask processing on the convolution result data based on the mask data of the object edge region to obtain an edge response data map of the object edge region.

[0092] The sharpness calculation module 307 is used to divide the sum of the edge response data of the object's edge region by the number of microlenses in the object's edge region to obtain the sharpness of the microlens array light field image. The edge response data of the object's edge region is matrix data based on convolution calculation and mask processing. The sharpness is calculated based on the edge response data of the object's edge region and the number of microlenses in the object's edge region, using the following formula:

[0093]

[0094] In the formula, I gl,margin(m,n) represents the edge response data matrix of the object's edge region, ||M margin (m,n)||0 represents the number of microlenses in the edge region of the object, and ‖‖0 represents the zero norm of the vector, used to calculate M. margin The number of non-zero elements in (m,n).

[0095] Exemplary electronic devices

[0096] This embodiment proposes an electronic device, including: one or more processors, an internal memory, and an external memory. The internal memory stores instructions, which, when executed by the one or more processors, cause the one or more processors to perform a sharpness evaluation method for a microlens array light field image as described in any of the preceding embodiments.

[0097] The processor is used to execute all or part of the steps in the sharpness evaluation method for a microlens array light field image as described in the embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0098] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the sharpness evaluation method for microlens array light field images described in the embodiments.

[0099] Computer storage media

[0100] A computer program is stored on the computer-readable storage medium, which, when executed by one or more processors, implements a sharpness evaluation method for a microlens array light field image as described in any of the foregoing embodiments.

[0101] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0102] In the 1930s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0103] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0104] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0105] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0106] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0110] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0111] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0112] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0113] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0114] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0115] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0116] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A method for evaluating the sharpness of light field images of microlens arrays, characterized in that, include: Determine the light field image and white image of the microlens array to be processed; Based on the light field image of the microlens array and the white image of the microlens array, individual microlens mask data, individual microlens grayscale image data, and microlens center grayscale image data are obtained. Based on the individual microlens mask data and the individual microlens grayscale image data, the standard deviation data of the light field image of the microlens array is calculated one by one to obtain the standard deviation data of the light field image of the microlens array. Based on the standard deviation data of the light field image of the microlens array, the mask data of the object edge region is obtained by binarization calculation; The number of microlenses in the object's edge region is obtained based on the mask data of the object's edge region; Based on the grayscale image data of the microlens center and the mask data of the object edge region, the edge response data of the object edge region is obtained by convolution calculation and mask processing. The sharpness of the microlens array light field image is obtained by dividing the sum of the edge response data of the object's edge region by the number of microlenses in the object's edge region.

2. The method according to claim 1, characterized in that, The step of dividing the sum of edge response data of the object's edge region by the number of microlenses in the object's edge region to obtain the sharpness of the microlens array light field image includes: The edge response data of the object's edge region is matrix data based on convolution calculation and masking processing; Sharpness is calculated based on the edge response data of the object's edge region and the number of microlenses in the object's edge region, using the following formula: In the formula, I gl,margin (m,n) represents the edge response data matrix of the object's edge region, ||M margin (m,n)||0 represents the number of microlenses in the edge region of the object, and ‖‖0 represents the zero norm of the vector, used to calculate M. margin The number of non-zero elements in (m,n).

3. The method according to claim 1, characterized in that, The step of obtaining edge response data of the object's edge region based on the grayscale image data of the microlens center and the mask data of the object's edge region, using convolution calculation and mask processing, includes: The extended template is calculated based on the preset Gaussian filter convolution template and the preset Laplacian operator; Based on the extended template, the grayscale image data at the center of the microlens is convolved to obtain the convolution result data; Based on the object edge region mask data, the convolution result data is masked to obtain the edge response data of the object edge region.

4. The method according to claim 1, characterized in that, The standard deviation data of the light field image of the microlens array is calculated one by one based on the individual microlens mask data and the individual microlens grayscale image data, including: The mean grayscale value is calculated based on the individual microlens mask data and the individual microlens grayscale image data, using the following formula: In the formula, I microlens(m,n) (x,y) represents the grayscale image data of a single microlens, M binary(m,n) (x,y) represents the data for a single microlens mask, μ (m,n) This represents the grayscale mean, m,n represents the m rows and n columns of the microlens, and x,y represents the x rows and y columns of the pixel; The standard deviation is calculated based on the individual microlens mask data, the individual microlens grayscale image data, and the grayscale mean, using the following formula: In the formula, σ (m,n) Indicates standard deviation; Based on the standard deviation, the standard deviation data of the light field image of the microlens array is obtained.

5. The method according to claim 1, characterized in that, The step of obtaining the object edge region mask data based on the standard deviation data of the light field image of the microlens array, using binarization calculation, includes: The standard deviation map is binarized to obtain the mask data of the object edge region, as shown in the following formula: In the formula, I σ (m,n) represents the standard deviation data of the light field image of the microlens array, λ σ The threshold value represents the standard deviation of grayscale, and m,n represents the number of rows and columns of the microlens.

6. A sharpness evaluation device for light field images of microlens arrays, characterized in that, include: The image determination module is used to determine the light field image of the microlens array to be processed and the white image of the microlens array. The data acquisition module is used to obtain individual microlens mask data, individual microlens grayscale image data, and microlens center grayscale image data based on the light field image of the microlens array and the white image of the microlens array. The standard deviation calculation module is used to calculate the standard deviation data of the light field image of the microlens array one by one based on the individual microlens mask data and the individual microlens grayscale image data; The binarization calculation module is used to obtain the mask data of the object edge region based on the standard deviation data of the light field image of the microlens array and the binarization calculation. The edge microlens number calculation module is used to obtain the number of microlenses in the edge region of the object based on the mask data of the object's edge region; The edge response data calculation module for the object edge region is used to obtain the edge response data of the object edge region based on the grayscale image data of the microlens center and the mask data of the object edge region, using convolution calculation and mask processing. The sharpness calculation module is used to divide the sum of the edge response data of the object's edge region by the number of microlenses in the object's edge region to obtain the sharpness of the microlens array light field image.

7. The apparatus according to claim 6, characterized in that, The resolution calculation module is specifically used for: The edge response data of the object's edge region is matrix data based on convolution calculation and masking processing; Sharpness is calculated based on the edge response data of the object's edge region and the number of microlenses in the object's edge region, using the following formula: In the formula, I gl,margin (m,n) represents the edge response data matrix of the object's edge region, ||M margin (m,n)||0 represents the number of microlenses in the edge region of the object, and ‖‖0 represents the zero norm of the vector, used to calculate M. margin The number of non-zero elements in (m,n).

8. The apparatus according to claim 6, characterized in that, The edge response data calculation module for the object's edge region is specifically used for: The extended template is calculated based on the preset Gaussian filter convolution template and the preset Laplacian operator; Based on the extended template, the grayscale image data at the center of the microlens is convolved to obtain the convolution result data; Based on the object edge region mask data, the convolution result data is masked to obtain the edge response data of the object edge region.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the sharpness evaluation method for a microlens array light field image as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the sharpness evaluation method for a microlens array light field image as described in any one of claims 1 to 5.

Images