Color image restoration method of single-plane diffraction lens

Abstract

A color image restoration method of a single-plane diffraction lens relates to the technical field of image correction, and aims to solve the problem of image color edge lines caused by chromatic aberration in the prior art, and the method comprises the following steps: designing a single-plane diffraction lens system; constructing an energy distribution matrix model of the diffraction efficiency in the visible light band; judging and optimizing a single-plane diffraction lens; constructing a point spread function (PSF) model; solving a similarity coefficient matrix model of the color channel; the diffraction efficiency energy distribution matrix model and the point spread function model are combined to restore the image definition; and restoring the color difference influence of the image according to the similarity coefficient matrix model. The problem of image blurring caused by diffraction efficiency and the problem of color edge lines of the image caused by chromatic aberration are synchronously solved, and the blurred color image restoration problem under the diffraction efficiency problem of a single-plane diffraction lens system is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image correction, and in particular to a color image restoration method of a single-plane diffraction lens. Background Art

[0002] Currently, traditional lenses are complex in structure and mostly use refractive and catadioptric methods, resulting in a large number of lenses. Currently, single-element imaging components, such as metalenses, have low diffraction efficiency, are difficult to manufacture, require the use of special materials and manufacturing processes, and have performance limitations. Compared to traditional single-element imaging, computational imaging technology that combines optics and imaging has been widely used in the field of image correction technology. It not only alleviates the difficulties in lens processing and manufacturing, but also improves the final image quality, allowing for the correction of image details and colors.

[0003] The Chinese patent publication number is "CN104809706B", and the patent name is "A single-lens computational imaging method based on the prior of smooth color changes in images". This method extracts the spatially varying point spread function (PSF) by taking multiple blurred images of different scenes, and constructs an image reconstruction model based on the prior of smooth color changes in the image, and uses an optimization algorithm to reconstruct the color image to improve image quality. Its advantage is that it can effectively reduce noise and artifacts and correct the clarity and detail of the image. However, when dealing with problems such as chromatic aberration and field curvature, the boundary conditions required for digital calculation are difficult to meet, which can easily lead to the singularity of the algorithm. The noise introduced during the optimization process is serious, resulting in color fringing in the image. Summary of the Invention

[0004] In order to solve the problem of image color fringing caused by chromatic aberration in the prior art, the present invention proposes a color image restoration method using a single-plane diffraction lens to achieve higher-precision image correction.

[0005] To achieve the above purpose, the technical solution of the present invention to solve the technical problem is:

[0006] A color image restoration method for a single-plane diffractive lens, the method comprising the following steps:

[0007] Step 1: Design a single-plane diffractive lens system:

[0008] The single-plane diffraction lens system comprises a single-plane diffraction lens and a detector. The front surface of the single-plane diffraction lens is a Fresnel surface, and the rear surface is a diffraction surface. When light in the visible light band is incident on the single-plane diffraction lens, a blurred color image is formed on the detector.

[0009] Step 2: Construct an energy distribution matrix model of diffraction efficiency in the visible light band:

[0010] The light energy of a certain wavelength at a certain analysis level is evenly distributed within the diffuse spot, which is equivalent to the diffraction efficiency being evenly distributed within the pixels covered by the diffuse spot. The energy distribution matrix of the diffraction efficiency is constructed based on the energy value within the pixel.

[0011] First, the wavelength with a difference of one pixel size T in the diffuse spot radius at different wavelengths is taken as the characteristic wavelength of the order, that is, the wavelength with a difference of 1 is the characteristic wavelength; according to this method, the characteristic wavelength of all orders is obtained;

[0012]

[0013] Where N is the number of pixel sizes covered by the diffuse spot, and R(λ,m) is the radius of the diffuse spot;

[0014] Then, calculate the diffraction efficiency of all characteristic wavelengths of the above orders:

[0015]

[0016] Where m is the diffraction order, λ0 is the design wavelength, λ is the characteristic wavelength, n(λ) and n(λ0) are the refractive indices of the substrate material at the characteristic wavelength and the design wavelength;

[0017] Finally, the above diffraction efficiency is evenly distributed in a circle with N pixels covered by the diffuse spot, and the energy distribution matrix of a certain order and a certain characteristic wavelength is constructed:

[0018]

[0019] All energy distributions are superimposed and normalized to obtain the energy distribution matrix model of diffraction efficiency:

[0020]

[0021] Where P is the energy distribution matrix model of diffraction efficiency in the visible light band;

[0022] Step 3: Determine and optimize the single-plane diffractive lens:

[0023] Using the deconvolution algorithm, the blurred color image obtained in step 1 is initially restored to see if the image edge blur exceeds two pixel sizes. If so, the system returns to step 1 to re-optimize the design of the single-plane diffraction lens. If the image edge blur is within one pixel size, the system parameters of the current single-plane diffraction lens are retained.

[0024] Step 4: Construct the point spread function PSF model:

[0025] According to the single-plane diffraction lens system parameters saved in step 3, equally spaced sampling is performed in the Y field direction of the single-plane diffraction lens system to collect the Zernike coefficients;

[0026] According to the Zernike coefficients of equally spaced sampling points, the Zernike coefficients at any field of view position are calculated using the Newton interpolation method;

[0027] Wavefront aberration W calculated from Zernike coefficients Y (x,y):

[0028]

[0029] Where A J is the Zernike coefficient, Z J represents the Zernike polynomial value;

[0030] Calculate the PSF matrix PSF(x,y) of a pixel point of a single-plane diffraction lens system based on the wavefront aberration:

[0031] PSF(x,y)=|F[p(x,y)exp[ikW Y (x,y)]]| 2

[0032] F represents Fourier transform, p(x,y) is the pupil function;

[0033] By adding the PSF of each pixel, the PSF of the entire system is obtained;

[0034] Step 5, solve the similarity coefficient matrix model of the color channel;

[0035] Step 6: Restore image clarity by combining the diffraction efficiency energy distribution matrix model and the point spread function model;

[0036] Step 7: Restore the color difference effect of the image based on the similarity coefficient matrix model.

[0037] The step 5 is specifically as follows: separate the blurred color image, using the green channel as the reference, denoted as I ref , the remaining channels red and blue are denoted as I t ; The remaining color channels are represented by the weighted sum of the high-order derivatives of the green channel, and the expression formula is as follows:

[0038] I t ≈α0·1+α1·I ref +α2·▽ x I ref +α3·▽ y I ref +…

[0039] Where α0, α1, α2, ... are the similarity coefficients of the color channels, ▽ x I ref is the derivative of the image in the x direction, ▽y I ref The derivative of the image in the y direction;

[0040] Use the least squares method to solve the similarity coefficient matrix model:

[0041]

[0042] Where α is the similarity coefficient matrix model of the color channel.

[0043] The step six is ​​specifically as follows: combining the models obtained in step two and step four to obtain the image degradation model P PSF :

[0044] P PSF =P×PSF

[0045] According to the degradation model, the inverse solution is performed. The optimal criterion for the inverse solution of the optical system degradation model is expressed as:

[0046]

[0047] In the formula Item is the fidelity item; The term is a smoothness constraint term; represents the optimal solution of the image in the restoration process; g is the image received by the detector; γ is the weight of the smoothness constraint; C is the Laplace operator; p corresponds to different norm types; the image is processed in combination with the deconvolution algorithm to obtain the inverse solution of image degradation and restore the clarity of the image.

[0048] The step seven is specifically as follows: taking the green channel image as a reference, introducing similarity coefficients between different color channels into the optical system degradation model:

[0049]

[0050] In the formula Item is the fidelity item; It represents the optimal solution of the image in the restoration process; g is a color channel image of the clarity of the restored image; α is the color channel similarity coefficient matrix model; the image is processed in combination with the deconvolution algorithm to restore the chromatic aberration effect of the image.

[0051] The present invention has the following beneficial effects:

[0052] This method uses a constructed diffraction efficiency energy distribution matrix model combined with a point spread function (PSF) model as a preliminary image degradation model, combined with a deconvolution algorithm to process the image and restore image clarity. The color channel at the design wavelength of a single-plane diffractive lens is used as the reference channel, and the images of the remaining color channels are represented by a weighted sum of the high-order derivatives of that color channel's image. A similarity coefficient matrix model between the color channels is then constructed to further restore the effects of chromatic aberration.

[0053] The present invention simultaneously solves the image blur problem caused by diffraction efficiency and the image color fringing problem caused by chromatic aberration, and realizes the restoration problem of blurred color images under the diffraction efficiency problem of a single-plane diffraction lens system. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of the single-plane diffraction lens design method described in the present invention.

[0055] Figure 2 It is a schematic structural diagram of the single-plane diffraction lens system described in the present invention.

[0056] Figure 3 is the point diagram of the single plane diffraction lens imaging of the present invention, Figure 3 (a) is the point diagram of the single plane diffraction lens imaging when the field of view is 0°, Figure 3 (b) is the point diagram of the single plane diffraction lens imaging when the field of view is 1°. Figure 3 (c) is the point diagram of the single-plane diffraction lens imaging when the field of view is 1.5°.

[0057] Figure 4 is the image restoration comparison diagram of the present invention, Figure 4 (a) is the original image, Figure 4 (b) is the restored image. DETAILED DESCRIPTION

[0058] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0059] like Figure 1 As shown, a color image restoration method of a single-plane diffraction lens comprises the following steps:

[0060] Step 1: Design a single plane diffractive lens system:

[0061] like Figure 2As shown, the single-plane diffractive lens system comprises a single-plane diffractive lens 1 and a detector 2. The single-plane diffractive lens 1 has a Fresnel front surface and a diffractive rear surface, a thickness of 0.8 to 1 mm, and a normalized radius of 5 mm. The system operates in the 405-690 nm wavelength range, has a focal length of 50 mm, and an F-number of 5.

[0062] System working process: Light is incident from the object plane to the single-plane diffraction lens 1, continues to propagate inside the lens and gradually converges on the detector 2, forming a relatively blurred color image.

[0063] like Figure 3 As shown in FIG, the point array diagram of the single plane diffraction lens imaging. When the diffraction order is 1, the maximum diffuse spot radius is 23.367 μm (RMS). In practical applications, considering the influence of diffraction efficiency, the area of ​​the diffuse spot is larger than 23.367 μm. Figure 3 It can be seen that there are large differences in the diffuse spots at different wavelengths, which indicates that chromatic aberration has a greater impact on imaging quality.

[0064] Step 2: Construct an energy distribution matrix model of diffraction efficiency in the visible light band:

[0065] First, -1, 0, 1, 2, and 3 orders are taken as analysis orders, order 1 is selected as the main order, and the design wavelength of the single-plane diffraction lens is 587 nm.

[0066] The wavelength of the diffuse spot radius R(λ,m) at different wavelengths that differs by one pixel size is taken as the characteristic wavelength of the order. In this example, the size of one pixel is 5.87μm.

[0067]

[0068] Where N is the number of pixel sizes covered by the diffuse spot, and R(λ,m) is the radius of the diffuse spot.

[0069] Taking the 0th order as an example, the characteristic wavelengths of all the other analysis orders are obtained in the same way. The number of pixel sizes covered by the diffuse spot is calculated according to formula (1). The characteristic wavelength of the 0th order in the present invention is shown in Table 1.

[0070] Table 1:

[0071] Characteristic wavelength / nm 486 494 507 517 531 545 Diffuse spot radius / μm 184.8 189.4 196.4 201.5 208.1 214.1 Radius pixels 32 33 34 35 36 37 Characteristic wavelength / nm 559 575 593 612 635 656 Diffuse spot radius / μm 219.6 225.5 231.4 237.2 243.4 248.5 Radius pixels 38 39 40 41 42 43

[0072] Then, the diffraction efficiency of the characteristic wavelength is calculated according to formula (2).

[0073]

[0074] Where m is the diffraction order, λ0 is the center wavelength, λ is the incident wavelength, and n(λ) and n(λ0) are the refractive indices of the substrate material at the incident wavelength and the center wavelength.

[0075] According to the characteristic wavelengths selected in Table 1, the diffraction efficiency at the characteristic wavelength of the 0th order in the present invention is shown in Table 2.

[0076] Table 2:

[0077] Characteristic wavelength / nm 486 494 507 517 531 545 Diffraction efficiency 0.0281 0.0246 0.0189 0.0148 0.0097 0.0056 Characteristic wavelength / nm 559 575 593 612 635 656 Diffraction efficiency 0.0025 4.637E-04 1.159E-04 0.002 0.0072 0.0147

[0078] Finally, the diffraction efficiency is evenly distributed in a circle with N pixels covered by the diffuse spot, and the energy value in each pixel at the characteristic wavelength of the 0th order in the present invention is calculated according to Table 2, as shown in Table 3.

[0079] Table 3:

[0080]

[0081]

[0082] The energy distribution matrix can be expressed by formula (3):

[0083]

[0084] Taking 486nm of order 0 as an example, the other wavelengths and orders are the same.

[0085]

[0086] According to the above method, the energy distribution of all analysis levels is obtained, and then normalized after superposition. The superposition and normalization formulas are shown below.

[0087]

[0088] Where P is the energy distribution matrix model of diffraction efficiency in the visible light band.

[0089] Step 3: Determine and optimize the single-plane diffractive lens:

[0090] According to the deconvolution algorithm, the blurred color image obtained in step 1 is preliminarily restored to see if the image edge blur exceeds two pixel sizes. If so, return to step 1 to redesign and optimize the single-plane diffraction lens. If the image edge blur is within one pixel size, continue to save the current single-plane diffraction lens system parameters.

[0091] Step 4: Construct a point spread function (PSF) model:

[0092] According to the single-plane diffraction lens system parameters saved in step 3, equally spaced sampling is performed in the Y field direction of the single-plane diffraction lens system. The present invention divides the sampling points into 10 equal points. The Zernike coefficients of the Y field of view of the single-plane diffraction lens system in the present invention are shown in Table 4.

[0093] Table 4:

[0094]

[0095]

[0096] According to the Zernike coefficients of the ten field of view sampling points in the table, the Zernike coefficients of any field of view position are calculated using the Newton interpolation method.

[0097] The wavefront aberration W calculated by the field-of-view related Zernike coefficient is obtained according to formula (5): Y (x,y):

[0098]

[0099] Where A J is the Zernike coefficient, Z J represents the Zernike polynomial value.

[0100] Substituting the wavefront aberration obtained from formula (5) into formula (6) yields the PSF (x, y) of a certain pixel point of the single-plane diffraction lens system.

[0101] PSF(x,y)=|F[p(x,y)exp[ikW Y (x,y)]]| 2 (6)

[0102] F represents Fourier transform, and p(x,y) is the pupil function.

[0103] By adding the PSF of each pixel, the PSF of the entire system can be obtained.

[0104] Step 5, solving the similarity coefficient matrix model of the color channel;

[0105] Separate the color image and use the green channel as the reference, denoted as I ref , the remaining channels (red and blue channels) are recorded as I t The remaining color channels are represented by the weighted sum of the high-order derivatives of the green channel, and the expression formula is as follows:

[0106] I t ≈α0·1+α1·I ref +α2·▽ x I ref +α3·▽ y Iref +…(7)

[0107] Where α0, α1, α2, ... are the similarity coefficients of the color channels, ▽ x I ref is the derivative of the image in the x direction, ▽ y I ref The derivative of the image in the y direction.

[0108] The derivatives of the image in the horizontal direction (x direction) and the vertical direction (y direction) are calculated using an intermediate precision method. This method combines the advantages of the Sobel operator and the Prewitt operator and can better balance computational efficiency and accuracy.

[0109] The specific operators are as follows:

[0110] X direction:

[0111] Y direction:

[0112] For each pixel, use the above operator to perform convolution operation with the image to calculate the gradient value of the point in the x direction and y direction, that is, ▽ x I ref 、▽ y I ref .

[0113] The coefficients between color channels are converted into formula (8), and the similarity coefficient matrix model is solved using the least squares method:

[0114]

[0115] Where α is the similarity coefficient matrix model of the color channel.

[0116] Step 6: Restore image clarity by combining the diffraction efficiency energy distribution matrix model and the point spread function model;

[0117] The image degradation model P is obtained by combining the models obtained in step 2 and step 4. PSF :

[0118] P PSF =P×PSF (9)

[0119] In order to restore the image, it is necessary to perform an inverse solution based on the degradation model. The optimal criterion for the inverse solution of the optical system degradation model is expressed as:

[0120]

[0121] In the formula Item is the fidelity item; The term is a smoothness constraint term; represents the optimal solution for image restoration; g is the image received by the detector; γ is the weight of the smoothness constraint; C is the Laplace operator; and p corresponds to different norm types. By combining the deconvolution algorithm with the image, we can obtain an inverse solution to image degradation and restore image clarity.

[0122] Step 7: Restore the color difference of the image based on the similarity coefficient matrix model;

[0123] Taking the green channel image as the benchmark, the similarity coefficients between different color channels are introduced into the optical system degradation model:

[0124]

[0125] In the formula Item is the fidelity item; represents the optimal solution for the image restoration process; g is the color channel image restored in step 6; α is the color channel similarity coefficient matrix model. Combined with the deconvolution algorithm, the image is processed to restore the color difference effect of the image.

[0126] like Figure 4 The figure shows the image restoration comparison diagram of the present invention. Figure 4 (a) is the original image, Figure 4 (b) is the restored image, and it can be seen that the restoration effect is obvious.

[0127] The images restored by computational imaging are evaluated using four methods: contrast, grayscale gradient, standard deviation, and NIQE. The evaluation results are shown in Table 5.

[0128]

Claims

1. A color image restoration method for a single-plane diffraction lens, characterized in that: The method comprises the following steps: Step 1: Design a single-plane diffractive lens system: The single-plane diffraction lens system comprises a single-plane diffraction lens (1) and a detector (2); the front surface of the single-plane diffraction lens (1) is a Fresnel surface, and the rear surface is a diffraction surface; after light in the visible light band is incident on the single-plane diffraction lens (1), a blurred color image is formed on the detector (2); Step 2: Construct an energy distribution matrix model of diffraction efficiency in the visible light band: The light energy of a certain wavelength at a certain analysis level is evenly distributed within the diffuse spot, which is equivalent to the diffraction efficiency being evenly distributed within the pixels covered by the diffuse spot. The energy distribution matrix of the diffraction efficiency is constructed based on the energy value within the pixel. First, the wavelength with a difference of one pixel size T in the diffuse spot radius at different wavelengths is taken as the characteristic wavelength of the order, that is, the wavelength with a difference of 1 is the characteristic wavelength; according to this method, the characteristic wavelength of all orders is obtained; Where N is the number of pixel sizes covered by the diffuse spot, and R(λ,m) is the radius of the diffuse spot; Then, calculate the diffraction efficiency of all characteristic wavelengths of the above orders: Where m is the diffraction order, λ0 is the design wavelength, λ is the characteristic wavelength, n(λ) and n(λ0) are the refractive indices of the substrate material at the characteristic wavelength and the design wavelength; Finally, the above diffraction efficiency is evenly distributed in a circle with N pixels covered by the diffuse spot, and the energy distribution matrix of a certain order and a certain characteristic wavelength is constructed: All energy distributions are superimposed and normalized to obtain the energy distribution matrix model of diffraction efficiency: Where P is the energy distribution matrix model of diffraction efficiency in the visible light band; Step 3: Determine and optimize the single-plane diffractive lens: Using the deconvolution algorithm, the blurred color image obtained in step 1 is initially restored to see if the image edge blur exceeds two pixel sizes. If so, the system returns to step 1 to re-optimize the design of the single-plane diffraction lens. If the image edge blur is within one pixel size, the system parameters of the current single-plane diffraction lens are retained. Step 4: Construct the point spread function PSF model: According to the single-plane diffraction lens system parameters saved in step 3, equally spaced sampling is performed in the Y field direction of the single-plane diffraction lens system to collect the Zernike coefficients; According to the Zernike coefficients of equally spaced sampling points, the Zernike coefficients at any field of view position are calculated using the Newton interpolation method; Wavefront aberration W calculated from Zernike coefficients Y (x,y): Where A J is the Zernike coefficient, Z J represents the Zernike polynomial value; Calculate the PSF matrix PSF(x,y) of a pixel point of a single-plane diffraction lens system based on the wavefront aberration: PSF(x,y)=|F[p(x,y)exp[ikW Y (x,y)]]| 2 F represents Fourier transform, p(x,y) is the pupil function; By adding the PSF of each pixel, the PSF of the entire system is obtained; Step 5, solve the similarity coefficient matrix model of the color channel; Step 6: Restore image clarity by combining the diffraction efficiency energy distribution matrix model and the point spread function model; Step 7: Restore the color difference effect of the image based on the similarity coefficient matrix model.

2. The color image restoration method of a single-plane diffractive lens according to claim 1, characterized in that: The step 5 is specifically as follows: separate the blurred color image, using the green channel as the reference, denoted as I ref , the remaining channels red and blue are denoted as I t ; The remaining color channels are represented by the weighted sum of the high-order derivatives of the green channel, and the expression formula is as follows: Where α0, α1, α2, ... are the similarity coefficients of the color channels, is the derivative of the image in the x direction, The derivative of the image in the y direction; Use the least squares method to solve the similarity coefficient matrix model: Where α is the similarity coefficient matrix model of the color channel.

3. The color image restoration method of a single-plane diffractive lens according to claim 1, wherein: The step six is ​​specifically as follows: combining the models obtained in step two and step four to obtain the image degradation model P PSF : P PSF =P×PSF According to the degradation model, the inverse solution is performed. The optimal criterion for the inverse solution of the optical system degradation model is expressed as: In the formula Item is the fidelity item; The term is a smoothness constraint term; represents the optimal solution of the image in the restoration process; g is the image received by the detector; γ is the weight of the smoothness constraint; C is the Laplace operator; p corresponds to different norm types; the image is processed in combination with the deconvolution algorithm to obtain the inverse solution of image degradation and restore the clarity of the image.

4. The color image restoration method of a single-plane diffractive lens according to claim 2, wherein: The step seven is specifically as follows: taking the green channel image as a reference, introducing similarity coefficients between different color channels into the optical system degradation model: In the formula Item is the fidelity item; It represents the optimal solution of the image in the restoration process; g is a color channel image of the clarity of the restored image; α is the color channel similarity coefficient matrix model; the image is processed in combination with the deconvolution algorithm to restore the chromatic aberration effect of the image.

Images