A depth estimation method based on light field polarization characteristics

Abstract

The present invention provides a depth estimation method based on the polarization characteristics of a light field, comprising the following steps: obtaining an original lens white image through a light field camera and performing distortion correction on the image; extracting highlight pixels using the polarization characteristics of different highlight pixels under the Brewster angle; using Stokes parameters to characterize polarized light, establishing a response relationship between light intensity and polarizer rotation angle, solving the optimal polarization angle to obtain an image after removing high reflections; using a color migration algorithm to transfer the color information of the original input image to an image without specular highlight pixels to improve image quality; decoding the processed image into a multi-view sub-aperture image and using a stereo matching algorithm to perform depth estimation on the image. The present invention realizes depth estimation of an object in a single shot, reduces the spatial complexity of image acquisition, and at the same time, the excellent highlight suppression effect improves the accuracy of depth estimation, which has practical significance and good application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and digital image processing, and in particular to a depth estimation method based on light field polarization characteristics. Background Art

[0002] Traditional camera imaging cannot obtain three-dimensional information of the image. Traditional highlight removal algorithms are limited by the object being measured and the shooting scene. The high-reflection removal of a single image depends on the information decomposition inside the image and is very unstable. The multi-image method requires multiple shots, which increases the spatial complexity of the shooting and easily introduces noise. However, the specular highlight removal based on the polarization characteristics of the light field is not affected by the object being measured and only requires a single shot, which reduces the spatial complexity of image acquisition. The traditional depth estimation method requires a binocular camera, which also increases the complexity of image acquisition. The specular highlight removal and depth estimation based on the polarization characteristics of the light field can achieve single-shot multi-view imaging. After completing the high-reflection removal, the depth of the surface of the object being measured can be directly estimated, which reduces the complexity of image acquisition and improves the accuracy of image depth estimation. Summary of the invention

[0003] In order to solve the above-mentioned problems, the present invention designs a depth estimation method based on the polarization characteristics of the light field for images captured by a microlens array focusing light field camera. The excellent suppression effect of the polarizer on high reflection is utilized to effectively remove the high reflection of the image. The recording characteristic of the light position and direction of a single shot of the light field camera is utilized to realize depth estimation of the object in a single shot, thereby reducing the spatial complexity of image acquisition. At the same time, the excellent high light suppression effect improves the accuracy of depth estimation, which has practical significance and good application prospects.

[0004] According to a first aspect of the present invention, a depth estimation method based on light field polarization characteristics is provided, comprising the following steps:

[0005] Step 1, place the light field camera and the object to be measured according to the Brewster angle, obtain the original lens white image, and perform distortion correction and tilt error correction on the original lens white image;

[0006] Step 2: Rotate the polarizer and take multiple pictures for difference, extract high-reflection pixels, and solve the Stokes parameter vector of the incident light and the intensity value of the outgoing light after passing through the polarizer. Use the intensity value to establish the relationship between the image grayscale and the polarizer rotation angle, and inversely solve the image grayscale minimum value to obtain the optimal polarization rotation angle. According to the optimal polarization rotation angle, the image after removing the high reflection is obtained;

[0007] Step 3, remove the polarizing film, take an image, and use a color migration algorithm to transfer the color information of the image to the image after removing the high reflection, so as to obtain a restored light field image;

[0008] Step 4, extracting sub-aperture images of different viewing angles from the restored light field image;

[0009] Step 5: Compare the extracted sub-aperture image with the central sub-aperture image, solve the disparity between the sub-aperture image and the central sub-aperture image, and calculate the depth information of the original light field image.

[0010] Based on the above technical solution, the present invention can also make the following improvements.

[0011] Optionally, the image distortion correction is performed by the following equation:

[0012] x1=x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 ) (1)

[0013] y1=y(1+k1r 2 +k2r 4 +k3r 6 )+2p2xy+p2(r 2 +2y 2 ) (2)

[0014] Where x and y are the original image coordinates, x1 and y1 are the corrected image coordinates, k1, k2, and k3 are the radial distortion coefficients, p1 and p2 are the tangential distortion coefficients, and r is the distance from the original pixel to the distortion center. The least squares method is used to perform nonlinear fitting on each center point line of the microlens to correct the tilt error of the image.

[0015] Optionally, the performing tilt error correction on the original lens white image includes:

[0016] The least square method is used to perform nonlinear fitting on each center point line of the microlens to correct the tilt error of the image.

[0017] Optionally, performing differentiation on the image to extract high-reflective pixel points includes:

[0018] Differentiate the images at different polarization rotation angles to obtain the image difference result;

[0019] Based on the obtained results, the sum difference of the image features is derived;

[0020] Among them, the sum difference formula of image features is derived as follows:

[0021]

[0022] D is the sum difference of image features, V d represents the result of image difference, and N represents the number of times the image is differentiated.

[0023] Optionally, the relationship between the grayscale of the image and the rotation angle of the polarizer is expressed as:

[0024]

[0025] Among them, S out (x, y, θ) is the relationship between the image grayscale and the polarizer rotation angle, θ represents the polarizer rotation angle; S0(x, y) represents the full intensity coordinate of the incident light, S1(x, y) represents the intensity difference coordinate of the linear polarization component in the horizontal and vertical directions, and S2 represents the intensity difference coordinate of the polarizer symmetric with respect to the starting axis.

[0026] Optionally, the step of using a color migration algorithm to transfer the color information of the image to the image after high reflection is removed comprises the following steps:

[0027] The original image is transformed from RGB color space to lαβ color space for color restoration;

[0028] Use logarithmic operations to correct the offset of image data;

[0029] Perform selection and translation processing on the data in LMS space;

[0030] Convert the processed data back to RGB space for display.

[0031] Optionally, extracting a sub-aperture image from the original light field image includes:

[0032] The angular resolution of the light field camera is solved, the size of each stitching block is calculated, the center point of the original lens white image is identified, and the sub-aperture image is extracted starting from the upper left corner of the macro pixel corresponding to each microlens.

[0033] Optionally, extracting the sub-aperture image starting from the upper left corner of the macro pixel corresponding to each microlens includes:

[0034] Extract the stitching area under different microlenses;

[0035] The sub-aperture image under the full viewing angle is solved according to the stitching area.

[0036] Optionally, comparing the extracted sub-aperture images of different viewing angles with the central sub-aperture image, solving the disparity between the images of different viewing angles and the central viewing angle image, and calculating the depth information of the image includes:

[0037] The small displacement of different sub-aperture images in the spatial domain is converted to the frequency domain for amplification;

[0038] Solve the disparity between different sub-views relative to the central view;

[0039] The scene disparity is generated by matching the central perspective image with the other sub-aperture images.

[0040] Optionally, the depth information of the calculated image includes:

[0041] Depth estimation is performed on images using a stereo matching algorithm.

[0042] Technical effects and advantages of the present invention:

[0043] The present invention proposes a method for specular highlight removal and depth estimation based on light field polarization characteristics, which effectively removes image specular high reflection and performs depth estimation. The image high reflection is removed by effectively and stably suppressing specular high reflection through the polarizer, and sub-aperture images of different viewing angles are obtained by a single shot with a light field camera, and the parallax of different sub-aperture images is used to realize depth estimation of the object under test in a single shot.

[0044] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart of a depth estimation method based on light field polarization characteristics provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of Brewster's angle provided in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of image distortion provided by an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of extracting a sub-aperture image from a macro-pixel corresponding to a micro-lens provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In the prior art, for image surface high reflection removal and depth estimation, multi-image-based methods face the problems of inaccurate matching and excessive calculations. Single-image-based methods fail due to unstable processing effects and low robustness. Structured light-based methods have high complexity in image acquisition and poor timeliness due to system structure limitations.

[0051] Based on the defects in the background technology, the embodiment of the present invention proposes a depth estimation method based on the polarization characteristics of the light field, specifically as follows Figure 1 As shown, the following steps are included:

[0052] Step 1, place the light field camera and the object to be measured according to the Brewster angle, obtain the original lens white image, and perform distortion correction and tilt error correction on the original lens white image;

[0053] Step 2: Rotate the polarizer and take multiple pictures for difference, extract high-reflection pixels, and solve the Stokes parameter vector of the incident light and the intensity value of the outgoing light after passing through the polarizer. Use the intensity value to establish the relationship between the image grayscale and the polarizer rotation angle, and inversely solve the image grayscale minimum value to obtain the optimal polarization rotation angle. According to the optimal polarization rotation angle, the image after removing the high reflection is obtained;

[0054] Step 3, remove the polarizing film, take an image, and use a color migration algorithm to transfer the color information of the image to the image after removing the high reflection, so as to obtain a restored light field image;

[0055] Step 4, extracting sub-aperture images of different viewing angles from the restored light field image;

[0056] Step 5: Compare the extracted sub-aperture image with the central sub-aperture image, solve the disparity between the sub-aperture image and the central sub-aperture image, and calculate the depth information of the original light field image.

[0057] It can be understood that the method for specular highlight removal and depth estimation based on the light field polarization characteristics in the embodiment of the present invention removes image high reflection through the effective and stable suppression effect of the polarizer on specular high reflection, obtains sub-aperture images of different viewing angles through a single shot of the light field camera, and uses the parallax of different sub-aperture images to achieve depth estimation of the object under test in a single shot. The method described in the embodiment of the present invention can effectively reduce the spatial complexity of image acquisition, and at the same time, the excellent highlight suppression effect improves the accuracy of depth estimation, and has good application prospects.

[0058] Next, each step is further described in detail with reference to the accompanying drawings.

[0059] Step 1, placing the light field camera and the object to be measured according to the Brewster angle, obtaining the original lens white image, and performing distortion correction and tilt error correction on the original lens white image;

[0060] The experimental device involved in this method includes: a light source, a light field camera, a polarizer, and an object to be measured. The polarizer is installed at the front end of the light field camera to remove the high reflection of the mirror surface; the light source is the fill light for the light field camera. During the experiment, the light field camera and the object to be measured are placed according to the Brewster angle;

[0061] It should be noted that Brewster's angle, also known as the polarization angle, is a condition that natural light must satisfy when it is reflected from a dielectric interface to be linearly polarized light. When natural light is reflected and refracted on a dielectric interface, generally both the reflected light and the refracted light are partially polarized light. Only when the angle of incidence is a certain angle is the reflected light linearly polarized light, and its vibration direction is perpendicular to the incident plane. This angle is called Brewster's angle or polarization angle, and is represented by i0. This law is called Brewster's law. The specific Brewster angle is as follows: Figure 2 As shown, light E 1P When the reflected light E' is incident on the object from air at Brewster's angle, 1P With refracted light E` 2P perpendicular to each other, where light E 1P The angle i0 with the normal line is expressed as the deflection angle, and γ is expressed as the refracted light E` 2P The angle with the normal line.

[0062] It should be noted that step 1 specifically includes placing the light field camera and the object to be measured according to the Brewster angle, and then calibrating the light field camera to obtain the values ​​of parameters p1, p2 and r, wherein p1 and p2 are tangential distortion coefficients, and r is the distance from the original pixel to the distortion center. The obtained parameters are used to correct the light field image distortion to ensure image quality;

[0063] Next, the original lens white image is obtained, and the image is subjected to distortion correction and tilt error correction. The white image can also be used to extract the light field sub-aperture image in the subsequent Step 4.

[0064] The light field image before correction has obvious pincushion distortion. Figure 3 As shown in the figure, the parameters obtained by calibrating the camera are substituted into the following formulas (1) and (2) to correct the input image and obtain the distortion-corrected image. However, the image still has rotation errors at this time. It is necessary to identify the image tilt angle and perform tilt correction on the image to finally obtain the tilt-corrected image. Image distortion usually includes radial distortion, tangential distortion and rotation error. Therefore, in order to eliminate the error, it is necessary to perform distortion correction and tilt error correction on the original lens white image.

[0065] The image distortion correction can be performed by the following equation:

[0066] x1=x(1+k1r 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 ) (1)

[0067] y1=y(1+k1r 2 +k2r 4 +k3r 6 )+2p2xy+p2(r 2 +2y 2 ) (2)

[0068] Among them, x and y are the original image coordinates, x1 and y1 are the corrected image coordinates, k1, k2, k3 are the radial distortion coefficients, p1 and p2 are the tangential distortion coefficients, and r is the distance from the original pixel to the distortion center.

[0069] Furthermore, the least square method is used to perform nonlinear fitting on each center point connecting line of the microlens to correct the tilt error of the image.

[0070] Step 2, rotating the polarizer to take pictures at rotation angles of 0°, 45° and 90°, extracting high-reflection pixels, and solving the Stokes parameter vector of the incident light and the light intensity value of the outgoing light after passing through the polarizer, establishing the relationship between the image grayscale and the polarizer rotation angle, and obtaining the optimal polarization rotation angle by inversely solving the image grayscale minimum value, and obtaining the image after removing the high reflection according to the optimal polarization rotation angle;

[0071] The step of performing differentiation on the image and extracting high-reflective pixel points comprises:

[0072] Differentiate the images at different polarization rotation angles to obtain the image difference result;

[0073] Based on the obtained results, the sum difference of the image features is derived.

[0074] Specifically, the images at different polarization rotation angles are differentiated and the solution formula is as follows:

[0075] V d =I j -I i (3)

[0076] Among them, V d Represents the result of image difference; I j and I i Represents images taken from different rotation angles of the polarizer.

[0077] The formula for extracting mirror pixels is as follows:

[0078]

[0079] D is the sum difference of image features to make specular highlight pixels more prominent; N is the number of times the image is differentiated.

[0080] In solving the Stokes parameter vector S of the incident light in And the light intensity value of the outgoing light after passing through the polarizer output light S out Substitute the Stokes parameter vector S in The formula for obtaining is as follows:

[0081]

[0082] Among them, I represents the pictures of different polarization rotation angles; S0 represents the full intensity of the incident light, S1 represents the intensity difference of the linear polarization component in the horizontal and vertical directions, S2 represents the intensity difference of the polarizer symmetric with respect to the starting axis, and S3 is the intensity difference between the left polarized light and the right polarized light. Usually, S3 can be ignored due to the minimum probability of circular polarization.

[0083] The light intensity of the outgoing light is S out Substitute the formula for obtaining the Stokes parameter vector into the following:

[0084] S out =M×S in (6)

[0085] M represents the Mueller matrix of the polarizer, which is expressed as:

[0086]

[0087] Where θ represents the polarizer rotation angle.

[0088] In this embodiment, the relationship between the image grayscale and the polarizer rotation angle S is established. out (x, y, θ), expressed as follows:

[0089]

[0090] The optimal polarization rotation angle is inversely solved by the minimum grayscale value of the image. The formula for obtaining the optimal polarization selection angle is as follows:

[0091] S min =argmin|S out (x,y,θ) θ∈[0°,180°] (9)

[0092] The corresponding polarizer rotation angle θ is obtained by solving the local minimum of the image intensity.

[0093] Obtaining an image after removing high reflection includes: the specular high reflection incident at the Brewster angle under natural light is linearly polarized light, which can be removed using a polarizing film.

[0094] Step 3, remove the polarizing film, take an image, and use a color migration algorithm to transfer the color information of the image to the image after removing the high reflection, so as to obtain a restored light field image;

[0095] Specifically, using a color migration algorithm to transfer the color information of the image to the image after removing the high reflection includes the following steps:

[0096] The original image is transformed from RGB color space to lαβ color space for color restoration;

[0097] Use logarithmic operations to correct the offset of image data;

[0098] Perform selection and translation processing on the data in LMS space;

[0099] Convert the processed data back to RGB space for display.

[0100] The specific steps are as follows: the original image is transformed from the RGB color space to the lαβ color space for color restoration. First, the RGB space is converted to the LMS cone space. The formula is as follows:

[0101]

[0102] Use logarithmic operation to correct the offset of image data. The formula is as follows:

[0103]

[0104] The data in the LMS space is selected and translated, and the formula is as follows:

[0105]

[0106] In the formula, the l axis represents the achromatic channel, while the α and β channels are the color yellow-blue and red-green opposing channels. The color information of the target image is calculated as follows:

[0107]

[0108] Among them, σ l p , σ α p , and σ β p Represent the standard deviation of the original image l-axis, α and β channels, σl d , σ α d , and σ β d represents the standard deviation of the target image l-axis, α and β channels, and < > represents the average value. d , α d and β d Represents the values ​​of the target image l-axis, α and β channels, l p , α p and β p Represents the values ​​of the l-axis, α and β channels of the original image, l' d , α' d and β' d Represents the values ​​of the l-axis, α, and β channels of the final result image.

[0109] Convert the processed data back to RGB space for display using the following formula:

[0110]

[0111] Step 4, extract the sub-aperture image from the original light field image; solve the angular resolution of the light field camera, calculate the size of each splicing block, identify the center point of the original lens white image, and extract the sub-aperture image from the upper left corner of the macro pixel corresponding to each microlens;

[0112] The sub-aperture image extraction of the original light field image and the solution of the angular resolution of the light field camera include: substituting the following formula,

[0113]

[0114] Wherein, w represents the spatial resolution, a represents the distance from the main lens image plane to the microlens plane, and b represents the distance from the microlens to the sensor.

[0115] Calculate the size of each splicing block and substitute it into the following formula:

[0116]

[0117] Among them, L p represents the size of the mosaic block; d represents the diameter of the microlens.

[0118] Extracting the sub-aperture image starting from the upper left corner of the macro pixel corresponding to each microlens includes:

[0119] Extract the stitching area under different microlenses. The formula is as follows:

[0120]

[0121] Where N(u i , v j ) means (u i , v j ) is the vertex coordinate in the perspective view, Г is the traversal operator, s i ,t j Expressed as the coordinates of the center point of the microlens.

[0122] Solve the sub-aperture image under full viewing angle, the formula is as follows:

[0123]

[0124] I s (u i ,v j ) represents the sub-aperture images from different perspectives, and Φ represents the traversal of each vertex. i 、v j Represents the horizontal and vertical coordinates of each viewing vertex. I represents the light field image.

[0125] Step 5: Compare the extracted sub-aperture images of different perspectives with the central sub-aperture image, solve the disparity between the images of different perspectives and the central perspective image, and calculate the depth information of the image.

[0126] Specific as Figure 4 The figure shows the process of sub-aperture extraction of a focusing light field camera, and the extraction process corresponds to formulas (15)-(18). Specifically, the mosaic block 1 and the mosaic block 2 are extracted at the upper left corner of the sensor plane of the macro pixel. The mosaic block 1 includes the sub-aperture image 1, and the mosaic block 2 includes the sub-aperture image 2. The number of sub-aperture images 1 and sub-aperture images 2 is calculated by formula (15), and L is calculated by formula (16). p That is, the mosaic block of the image is used to form the sub-aperture image. Formula (17) describes the calculation process of stitching a single sub-aperture image, and formula (18) describes the calculation process of stitching all sub-aperture images. The figure takes the upper and lower perspectives as examples to visually describe the formation process of sub-aperture images at different perspectives. Finally, sub-aperture image 1 and sub-aperture image 2 are compared with the central aperture image at the center. Among them, (s i ,t j ) represents the coordinates of the center point of the microlens, b represents the distance between the microlens and the sensor; d represents the diameter of the microlens.

[0127] The small displacement of the spatial domain of different sub-aperture images is converted to the frequency domain and amplified. The formula is as follows:

[0128]

[0129] F represents Fourier transform, x is the pixel coordinate, Δx is the difference between other perspectives and the central perspective;

[0130] Solve the parallax between different sub-views relative to the central view. The formula is as follows:

[0131]

[0132] The scene disparity is generated by matching the central perspective image with the other sub-aperture images, as follows:

[0133]

[0134] In the formula, the depth information of different pixels is stored in C A In, C A Indicates the depth information of each pixel. c represents the image from the central perspective, the depth information is recorded by l, u' represents the image from other perspectives, τ1 represents the cutoff value, R x represents a rectangular pixel block, and l represents the depth information.

[0135] Finally, a stereo matching algorithm is used to estimate the depth of the image.

[0136] In summary, the embodiments of the present invention obtain sub-aperture images of different viewing angles through a single shot with a light field camera, and use the parallax of different sub-aperture images to achieve depth estimation of the object under test through a single shot.

[0137] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A depth estimation method based on light field polarization characteristics, characterized in that: The following steps are involved: Step 1, place the light field camera and the object to be measured according to the Brewster angle, obtain the original lens white image, and perform distortion correction and tilt error correction on the original lens white image; Step 2: Rotate the polarizer and take multiple pictures for difference, extract high-reflection pixels, and solve the Stokes parameter vector of the incident light and the intensity of the outgoing light after passing through the polarizer. Establish the relationship between the image grayscale and the polarizer rotation angle, and obtain the optimal polarization rotation angle by inversely solving the image grayscale minimum value. According to the optimal polarization rotation angle, obtain the image after removing the high reflection. Step 3, remove the polarizing film, take an image, and use a color migration algorithm to transfer the color information of the image to the image after removing the high reflection, so as to obtain a restored light field image; Step 4, extracting sub-aperture images of different viewing angles from the restored light field image; Step 5: Compare the extracted sub-aperture image with the central sub-aperture image, solve the disparity between the sub-aperture image and the central sub-aperture image, and calculate the depth information of the original light field image.

2. The depth estimation method based on light field polarization characteristics according to claim 1, characterized in that: The distortion correction of the image is performed by the following equation: （1） （2） in x , y is the original image coordinate, x 1. y 1 is the corrected image coordinate, k 1. k 2. k 3 is the radial distortion coefficient, p 1. p 2 is the tangential distortion coefficient, r is the distance from the original pixel to the distortion center. The least square method is used to perform nonlinear fitting on each center point line of the microlens to correct the tilt error of the image.

3. The depth estimation method based on light field polarization characteristics according to claim 1, characterized in that: The tilt error correction of the original lens white image comprises: The least square method is used to perform nonlinear fitting on each center point line of the microlens to correct the tilt error of the image.

4. The depth estimation method based on light field polarization characteristics according to claim 1, characterized in that: Differentiating the image and extracting highly reflective pixels include: Differentiate the images at different polarization rotation angles to obtain the image difference result; Based on the obtained results, the sum difference of the image features is derived; Among them, the sum difference formula of image features is derived as follows: （4） D is the sum difference of image features, V d represents the result of image difference, N Indicates the number of times the image is differentiated.

5. The depth estimation method based on light field polarization characteristics according to claim 1, characterized in that: The relationship between the grayscale of the image and the rotation angle of the polarizer is expressed as: （8） in, S out ( x , y , θ ) is the relationship between the image grayscale and the polarizer rotation angle, θ represents the polarizer rotation angle; S 0( x , y ) represents the full intensity coordinate of the incident light, S 1( x , y ) represents the intensity difference coordinates of the linear polarization component in the horizontal and vertical directions, and S2 represents the intensity difference coordinates of the polarizer symmetric with respect to the starting axis.

6. The depth estimation method based on light field polarization characteristics according to claim 1, characterized in that: The method of using a color migration algorithm to transfer the color information of the image to the image after removing the high reflection includes the following steps: The original image is RGB Color space conversion to lαβ Color space for color restoration; Use logarithmic operations to correct the offset of image data; right LMS The spatial data is selected and translated; Convert the processed data back RGB Space display.

7. The depth estimation method based on light field polarization characteristics according to claim 1, characterized in that: The extracting sub-aperture images of different viewing angles from the restored light field image comprises: The angular resolution of the light field camera is solved, the size of each stitching block is calculated, the center point of the original lens white image is identified, and the sub-aperture image is extracted starting from the upper left corner of the macro pixel corresponding to each microlens.

8. The depth estimation method based on light field polarization characteristics according to claim 7, characterized in that: The extracting of the sub-aperture image from the upper left corner of the macro pixel corresponding to each micro-lens includes: Extract the stitching area under different microlenses; The sub-aperture image under the full viewing angle is solved according to the stitching area.

9. The depth estimation method based on light field polarization characteristics according to claim 1, characterized in that: The step of comparing the extracted sub-aperture images of different viewing angles with the central sub-aperture image, solving the disparity between the images of different viewing angles and the central viewing angle image, and calculating the depth information of the image includes: The small displacement of different sub-aperture images in the spatial domain is converted to the frequency domain for amplification; Solve the disparity between different sub-views relative to the central view; The scene disparity is generated by matching the central perspective image with the other sub-aperture images.

10. The depth estimation method based on light field polarization characteristics according to claim 9, characterized in that: The depth information of the calculated image includes: Depth estimation is performed on images using a stereo matching algorithm.

Images