A method and device for evaluating spatial domain of light field cutout based on empty angle consistency

Abstract

The present invention discloses a spatial domain evaluation method and device for light field cutout based on spatial angle consistency, which comprises: step 1, extracting a central sub-aperture image of light field data; step 2, calculating a disparity map of light field data and an alpha map of a central sub-aperture image of light field data; step 3, propagating the central sub-aperture image alpha map to obtain a light field alpha map; step 4, evaluating the spatial angle consistency of light field data in the light field alpha map by using a spatial angle consistency evaluation index S-cons obtained in the spatial domain: the index S-cons value range is (0-1), the closer the index S-cons is to 1, the smaller the variance of the boundary area is, the better the spatial angle consistency of the light field data is, and the closer the index S-cons is to 0, the worse the spatial angle consistency of the light field data is. By adopting the method provided by the present invention, accurate and consistent light field cutout can be achieved and its spatial angle consistency can be evaluated.

Description

Technical Field

[0001] The present invention relates to the technical field of optical information processing, and in particular to a method and device for evaluating a light field cutout space domain based on empty angle consistency. Background Art

[0002] The light field describes the distribution of light in free space. Light field data captures the radiation of light from all directions and records the spatial, angular and intensity information of the light. Light field imaging can capture richer visual information, make up for the angular information lost in traditional photography, provide stronger capabilities for scene understanding, and improve the performance of traditional computer vision problems.

[0003] A 4D light field data can be represented by a 2D sub-aperture image array, so the 2D image cutout algorithm performed on sub-aperture images can be regarded as realizing the cutout of 4D light field data to a certain extent. The sub-aperture image operation method does not consider the connection between different sub-aperture images, and cannot maintain the spatial angle consistency of the light field data. The results obtained are difficult to meet the requirements of downstream technologies and applications. In addition, the light field data size is large, and there is a lot of redundant information between sub-aperture images. Simple sub-aperture image processing has too many redundant calculations and is too inefficient. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for evaluating the spatial domain of light field cutout based on spatial angle consistency, which can avoid the destruction of the spatial angle consistency of light field data during the cutout process and reduce calculation redundancy.

[0005] To achieve the above object, the present invention provides a spatial domain evaluation method for light field cutout based on empty angle consistency, which comprises:

[0006] Step 1, extracting the central sub-aperture image of the light field data;

[0007] Step 2, calculating the disparity map of the light field data and the alpha map of the central sub-aperture image of the light field data;

[0008] Step 3, propagate the central sub-aperture image alpha map to obtain a light field alpha map;

[0009] Step 4, the spatial angle consistency of the light field data in the light field alpha map is evaluated by using the spatial angle consistency evaluation index S-cons obtained in the spatial domain provided by formula (1): the index S-cons value range is (0-1), the closer the index S-cons is to 1, the smaller the variance of the boundary area is, and the better the spatial angle consistency of the light field data is; the closer the index S-cons is to 0, the worse the spatial angle consistency of the light field data is;

[0010]

[0011] Where N EPI is the number of EPIs in the light field alpha map, E j is the corresponding j-th EPI. There are l boundary areas in each EPI, denoted as Ω l , N l is the Ω in the jth EPI l The total number of pixels in the area, E ji is the i-th pixel in the j-th EPI of the light field alpha image, is the Ω in the jth EPI l The mean of the pixel values ​​at the boundary.

[0012] Furthermore, the calculation formula of the disparity map disp(x,y) in step 2 is formula (2):

[0013] disp(x,y)=F / Z(x,y)+F (2)

[0014] Where F is the focal length of the camera and Z(x,y) is the depth map of the scene.

[0015] Furthermore, the method for obtaining the central sub-aperture image alpha map in step 2 specifically includes:

[0016] Step 21: The confidence q of any pixel i in the disparity map disp(x,y) is i Set as formula (3):

[0017]

[0018] In the formula, p i is the disparity value of any pixel i in the disparity map disp(x,y), D=max(disp(x,y)) is the maximum value in the disparity map, and d=min(disp(x,y)) is the minimum value in the disparity map;

[0019] Step 22, set the confidence threshold t 1 and t 2 , when the confidence q i Greater than t 1 When , the area belongs to the foreground, and when the confidence q i Less than t 2 When , the area belongs to the background, and the other areas are uncertain areas in the trigraph, then the trigraph described by formula (4) is obtained:

[0020]

[0021] Step 23, convert the three-part image into I 1 and I 2 , I 1The pixels marked in the foreground and background are 1, and the pixel values ​​in other places are 0. 1 Transformed into a sparse diagonal matrix, D s , I 2 The pixel value of the foreground mark is 1, and the other pixel values ​​are 0. Converting it into a column vector is b s :

[0022]

[0023]

[0024] Step 24, solve the alpha map of the central sub-aperture image by equation (7):

[0025] α=λ(L+λ) -1 b s (7)

[0026] Where λ is the weight coefficient of the label, and L is the Laplacian matrix of the cutout.

[0027] Furthermore, in step 3, the central sub-aperture image alpha map is propagated through the light field alpha map propagation model provided by equation (8) to obtain the light field alpha map alpha u,v (x,y);

[0028]

[0029] In the formula, is the central viewpoint (u 0 ,v 0 ) is the alpha map corresponding to the sub-aperture image, disp(x,y) is the disparity corresponding to the image point (x,y) in the central sub-aperture image, Δu and Δv are the disparity between any viewpoint (u,v) and the central viewpoint (u 0 ,v 0 ) in the u and v directions.

[0030] The present invention also provides a spatial domain evaluation device for light field cutout based on empty angle consistency, which comprises:

[0031] A central sub-aperture image extraction unit, which is used to extract a central sub-aperture image of the light field data;

[0032] A computing unit, which is used to calculate a disparity map of the light field data and an alpha map of a sub-aperture image at a center of the light field data;

[0033] A light field cutout unit, which is used to propagate the central sub-aperture image alpha map to obtain a light field alpha map;

[0034] An evaluation unit is used to evaluate the spatial angle consistency of the light field data in the light field alpha map by using the spatial angle consistency evaluation index S-cons obtained in the spatial domain provided by formula (1): the index S-cons value range is (0-1), the closer the index S-cons is to 1, the smaller the variance of the boundary area is, the better the spatial angle consistency of the light field data is, and the closer the index S-cons is to 0, the worse the spatial angle consistency of the light field data is;

[0035]

[0036] Where N EPI is the number of EPIs in the light field alpha map, E j is the corresponding j-th EPI. There are l boundary areas in each EPI, denoted as Ω l , N l is the Ω in the jth EPI l The total number of pixels in the area, E ji is the i-th pixel in the j-th EPI of the light field alpha image, is the Ω in the jth EPI l The mean of the pixel values ​​at the boundary.

[0037] Furthermore, the computing unit comprises:

[0038] The disparity calculation subunit is used to calculate the disparity map disp(x,y) according to the depth map of the light field data by formula (2):

[0039] disp(x,y)=F / Z(x,y)+F (2)

[0040] Where F is the focal length of the camera and Z(x,y) is the depth map of the scene.

[0041] Furthermore, the computing unit comprises:

[0042] The central sub-aperture image alpha map calculation subunit is used to obtain the central sub-aperture image alpha map according to the central sub-aperture image of the light field data through the following specific alpha closed solution method:

[0043] Step 21: The confidence q of any pixel i in the disparity map disp(x,y) is i Set as formula (3):

[0044]

[0045] In the formula, p iis the disparity value of any pixel i in the disparity map disp(x,y), D=max(disp(x,y)) is the maximum value in the disparity map, and d=min(disp(x,y)) is the minimum value in the disparity map;

[0046] Step 22, set the confidence threshold t 1 and t 2 , when the confidence q i Greater than t 1 When , the area belongs to the foreground, and when the confidence q i Less than t 2 When , the area belongs to the background, and the other areas are uncertain areas in the trigraph, then the trigraph described by formula (4) is obtained:

[0047]

[0048] Step 23, convert the three-part image into I 1 and I 2 , I 1 The pixels marked in the foreground and background are 1, and the pixel values ​​in other places are 0. 1 Transformed into a sparse diagonal matrix, D s , I 2 The pixel value of the foreground mark is 1, and the other pixel values ​​are 0. Converting it into a column vector is b s :

[0049]

[0050]

[0051] Step 24, solve the alpha map of the central sub-aperture image by equation (7):

[0052] α=λ(L+λ) -1 b s (7)

[0053] Where λ is the weight coefficient of the label, and L is the Laplacian matrix of the cutout.

[0054] Furthermore, the light field cutout unit propagates the central sub-aperture image alpha map through the light field alpha map propagation model provided by equation (8) to obtain the light field alpha map alpha u,v (x,y);

[0055]

[0056] In the formula, is the central viewpoint (u 0 ,v 0) is the alpha map corresponding to the sub-aperture image, disp(x,y) is the disparity corresponding to the image point (x,y) in the central sub-aperture image, Δu and Δv are the disparity between any viewpoint (u,v) and the central viewpoint (u 0 ,v 0 ) in the u and v directions.

[0057] The present invention adopts the above technical solution, which has the following advantages:

[0058] By adopting the method of light field cutout based on empty angle consistency and its evaluation index provided by the present invention, accurate and consistent light field cutout is achieved, the empty angle consistency of light field data is ensured and calculation redundancy is reduced, and the established evaluation index can effectively evaluate the consistency of the obtained light field empty angle data. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic flow chart of a method for evaluating a light field cutout in a spatial domain based on empty angle consistency provided in an embodiment of the present invention.

[0060] Figure 2 A schematic flow chart of a method for evaluating a light field cutout in a spatial domain based on empty angle consistency provided in an embodiment of the present invention.

[0061] Figure 3 It is the alpha map disparity map of the central sub-aperture image.

[0062] Figure 4 Schematic diagram of the propagation model.

[0063] Figure 5 Schematic diagram of evaluation indicators. DETAILED DESCRIPTION

[0064] In the drawings, the same or similar reference numerals are used to represent the same or similar elements or elements with the same or similar functions. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0065] like Figure 1 and Figure 2 As shown, the spatial domain evaluation method of light field cutout based on empty angle consistency provided by the embodiment of the present invention includes:

[0066] Step 1: extract the central sub-aperture image of the light field data.

[0067] Step 2: Calculate the disparity map of the light field data and the central sub-aperture image alpha map of the light field data.

[0068] Step 3: propagate the central sub-aperture image alpha map to obtain a light field alpha map.

[0069] Step 4, such as Figure 5As shown in the figure, the object points at the boundary of the scene form a bunch of straight line bands in the EPI of the alpha map. The smaller the overall difference of the straight line bands, the better the consistency at the boundary. Therefore, the area at the boundary is recorded as Ω. The variance of the pixel values ​​in the Ω area is calculated to measure the consistency of the obtained light field data. If the variance is smaller, the consistency is better.

[0070] In view of this, step 4 evaluates the spatial angle consistency of the light field data in the light field alpha map through the spatial angle consistency evaluation index S-cons obtained in the spatial domain provided by formula (1). Among them, the index S-cons value range is (0-1). The closer the index S-cons is to 1, the smaller the variance of the boundary area is, and the better the spatial angle consistency of the light field data is. The closer the index S-cons is to 0, the worse the spatial angle consistency of the light field data is.

[0071]

[0072] Where N EPI is the number of EPI (Epipolar Plane Image) of the light field alpha image, E j is the corresponding j-th EPI. There are l boundary areas in each EPI, denoted as Ω l , N l is the Ω in the jth EPI l The total number of pixels in the area, E ji is the i-th pixel in the j-th EPI of the light field alpha image, is the Ω in the jth EPI l The mean of the pixel values ​​at the boundary.

[0073] The present invention establishes a light field cutout propagation model that satisfies the spatial angle consistency. The establishment of the model is based on the spatial angle coupling property of four-dimensional light field data, and can quickly obtain a high-quality light field alpha image with better spatial angle consistency. To achieve light field cutout with spatial angle consistency, the light field cutout method of Cho et al. can be used. By introducing the smooth constraint of the epipolar plane image (EPI), the spatial angle consistency of the estimated alpha image in the sub-aperture image is enhanced. In the present invention, a quantitative evaluation index of the light field alpha image is proposed. The establishment of the index is based on the essential properties of the light field data, and can objectively evaluate the spatial angle consistency of the light field alpha image after propagation. Cho et al. also proposed an index for evaluating the spatial angle consistency of light field data, which can also be evaluated using MSE, SAD, etc.

[0074] In one embodiment, in step 2, Figure 3As shown in a in FIG, according to the central sub-aperture image of the light field data, the disparity map disp(x, y) of the light field data is calculated using equation (2). Since the depth and disparity satisfy such an inverse relationship:

[0075] disp(x,y)=F / Z(x,y)+F (2)

[0076] Where F is the focal length of the camera and Z(x,y) is the depth map of the scene.

[0077] The depth can be estimated by using a fast light field image depth estimation method based on multi-view stereo matching proposed by Mishiba K et al. The depth can also be estimated by using an occlusion-aware depth estimation algorithm proposed by Wang TC et al. Once the depth map is obtained, the disparity map can be obtained by converting it using formula (2).

[0078] In one embodiment, Figure 3 As shown in b, the method for obtaining the central sub-aperture image alpha map in step 2 specifically includes:

[0079] Step 21, in order to automatically obtain the three-part image of the central sub-aperture image and solve the alpha image, it is assumed that the light field data disparity map is accurate, and the confidence is introduced into the disparity map and a threshold is selected. The accurate foreground, background and uncertain area are set according to the confidence threshold. The confidence of the disparity map is set as follows:

[0080]

[0081] In the formula, p i is the disparity value of any pixel i in the disparity map disp(x,y), and the confidence q at the pixel i Represents the probability that the point belongs to the foreground, D = max (disp (x, y)) is the maximum value in the disparity map, and d = min (disp (x, y)) is the minimum value in the disparity map. Set the threshold t 1 and t 2 , when the confidence level is greater than t 1 When the confidence is less than t 2 When , this part of the area is considered as the background, and the other areas are the uncertain areas in the three-part image, and the three-part image can be obtained:

[0082]

[0083] Where, t 1 and t 2 is a point on the straight line formed by the minimum and maximum confidence values, and t 1 and t 2The selection of t is related to the proportion of the depth range of the foreground and background in the overall depth of the scene. When the foreground depth range accounts for a large proportion, t 1 The larger the t 2 The smaller the background depth range, the larger the t 1 The smaller the t 2 The bigger.

[0084] After obtaining the three-part image, convert the three-part image into I 1 and I 2 :I 1 The pixels marked in the foreground and background are 1, and the pixel values ​​in other places are 0. 1 Transformed into a sparse diagonal matrix, D s ;I 2 The pixel value of the foreground mark is 1, and the other pixel values ​​are 0. Converting it into a column vector is b s Of course, you can use manual marking 1 and I 2 , and existing automatic methods can also be used for labeling.

[0085]

[0086]

[0087] Step 22, based on the obtained D s and b s The alpha map of the central sub-aperture image can be solved by formula (7):

[0088] α=λ(L+λ) -1 b s (7)

[0089] In the formula, λ is the weight coefficient of the mark, which is generally a larger number, such as 100, and L is the Laplace matrix of the cutout.

[0090] In addition, the central sub-aperture image can be matted by Random Walk matting, Poisson matting and other methods.

[0091] In one embodiment, in step 3, assuming that the scene is a Lambertian radiation unobstructed scene, the collected light field data contains the essential property of the space-angle coupling relationship, and the central sub-aperture image alpha map is propagated through the light field alpha map propagation model provided by formula (8) to obtain the light field alpha map alpha u,v (x,y);

[0092]

[0093] In the formula, is the central viewpoint (u 0 ,v 0 ) is the alpha map in the sub-aperture image, disp(x,y) is the disparity corresponding to the image point (x,y), Δu and Δv are the disparity between any viewpoint (u,v) and the central viewpoint (u 0 ,v 0 ) in the u and v directions.

[0094] According to the propagation model constructed in step 3, the alpha map of step 2 is propagated to obtain the light field alpha map under 9*9 viewpoints, as shown in Figure 4 (a) shows the central subaperture image alpha Figure 2 and parallax Figure 3 , Figure 4 (b) shows the alpha of the sub-aperture image obtained by propagation at 9*9 viewpoints. Figure 4 .

[0095] The embodiment of the present invention further provides a spatial domain evaluation device for light field cutout based on empty angle consistency, which includes a central sub-aperture image extraction unit, a calculation unit, a light field cutout unit and an evaluation unit, wherein:

[0096] The central sub-aperture image extraction unit is used to extract the central sub-aperture image of the light field data.

[0097] The calculation unit is used to calculate the disparity map of the light field data and the alpha map of the central sub-aperture image of the light field data.

[0098] The light field cutout unit is used to propagate the central sub-aperture image alpha map to obtain a light field alpha map.

[0099] The evaluation unit is used to evaluate the spatial angle consistency of the light field data in the light field alpha map by using the spatial angle consistency evaluation index S-cons obtained in the spatial domain provided by formula (1): the index S-cons value range is (0-1), the closer the index S-cons is to 1, the smaller the variance of the boundary area is, the better the spatial angle consistency of the light field data is, and the closer the index S-cons is to 0, the worse the spatial angle consistency of the light field data is;

[0100]

[0101] Where N EPI is the number of EPIs in the light field alpha map, E j is the corresponding j-th EPI. There are l boundary areas in each EPI, denoted as Ω l , N l is the Ω in the jth EPI l The total number of pixels in the area, E jiis the i-th pixel in the j-th EPI of the light field alpha image, is the Ω in the jth EPI l The mean of the pixel values ​​at the boundary.

[0102] In one embodiment, the calculation unit includes a disparity calculation subunit, which is used to calculate the disparity map disp(x, y) according to the depth map of the light field data by formula (2):

[0103] disp(x,y)=F / Z(x,y)+F (2)

[0104] Where F is the focal length of the camera and Z(x,y) is the depth map of the scene.

[0105] In one embodiment, the calculation unit includes a central sub-aperture image alpha map calculation sub-unit, which is used to obtain the central sub-aperture image alpha map according to the central sub-aperture image of the light field data by the following specific alpha closed solution method:

[0106] Step 21: The confidence q of any pixel i in the disparity map disp(x,y) is i Set as formula (3):

[0107]

[0108] In the formula, p i is the disparity value of any pixel i in the disparity map disp(x,y), D=max(disp(x,y)) is the maximum value in the disparity map, and d=min(disp(x,y)) is the minimum value in the disparity map;

[0109] Step 22, set the confidence threshold t 1 and t 2 , when the confidence q i Greater than t 1 When , the area belongs to the foreground, and when the confidence q i Less than t 2 When , the area belongs to the background, and the other areas are uncertain areas in the trigraph, then the trigraph described by formula (4) is obtained:

[0110]

[0111] Step 23, convert the three-part image into I 1 and I 2 , I 1 The pixels marked in the foreground and background are 1, and the pixel values ​​in other places are 0. 1 Transformed into a sparse diagonal matrix, D s , I2 The pixel value of the foreground mark is 1, and the other pixel values ​​are 0. Converting it into a column vector is b s :

[0112]

[0113]

[0114] Step 24, solve the alpha map of the central sub-aperture image by equation (7):

[0115] α=λ(L+λ) -1 b s (7)

[0116] Where λ is the weight coefficient of the label, and L is the Laplacian matrix of the cutout.

[0117] In one embodiment, the light field cutout unit propagates the central sub-aperture image alpha map through the light field alpha map propagation model provided by formula (8) to obtain the light field alpha map alpha u,v (x,y);

[0118]

[0119] In the formula, is the central viewpoint (u 0 ,v 0 ) is the alpha map of the sub-aperture image under the condition of , disp(x,y) is the disparity corresponding to the image point (x,y), Δu and Δv are the disparity between any viewpoint (u,v) and the central viewpoint (u 0 ,v 0 ) in the u and v directions.

[0120] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that the technical solutions described in the above embodiments may be modified, or some of the technical features thereof may be replaced by equivalents; these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spatial domain evaluation method for light field cutout based on empty angle consistency, characterized in that: include: Step 1, extracting the central sub-aperture image of the light field data; Step 2, calculating the disparity map of the light field data and the alpha map of the central sub-aperture image of the light field data; Step 3, propagate the central sub-aperture image alpha map to obtain a light field alpha map; Step 4, the spatial angle consistency of the light field data in the light field alpha map is evaluated by using the spatial angle consistency evaluation index S-cons obtained in the spatial domain provided by formula (1): the index S-cons value range is (0-1), the closer the index S-cons is to 1, the smaller the variance of the boundary area is, and the better the spatial angle consistency of the light field data is; the closer the index S-cons is to 0, the worse the spatial angle consistency of the light field data is; Where N EPI is the number of EPIs in the light field alpha map, E j is the corresponding j-th EPI. There are l boundary areas in each EPI, denoted as Ω l , N l is the Ω in the jth EPI l The total number of pixels in the area, E ji is the i-th pixel in the j-th EPI of the light field alpha image, is the Ω in the jth EPI l The mean of the pixel values ​​at the boundary; in: The method for obtaining the central sub-aperture image alpha map in step 2 specifically includes: Step 21: The confidence q of any pixel i in the disparity map disp(x,y) is i Set as formula (3): In the formula, p i is the disparity value of any pixel i in the disparity map disp(x,y), D=max(disp(x,y)) is the maximum value in the disparity map, and d=min(disp(x,y)) is the minimum value in the disparity map; Step 22, set confidence thresholds t1 and t2, when confidence q i When it is greater than t1, the area belongs to the foreground. i When it is less than t2, the area belongs to the background, and the other areas are uncertain areas in the trigraph. Then the trigraph described by formula (4) is obtained: Step 23, convert the three-part image into I1 and I2, where the pixels at the foreground and background marks in I1 are 1, and the pixel values ​​at other places are 0, and I1 is converted into a sparse diagonal matrix, which is D s , the pixel value of the foreground mark in I2 is 1, and the other pixel values ​​are 0. Converting it into a column vector is b s : Step 24, solve the alpha map of the central sub-aperture image by equation (7): α=λ(L+λ) -1 b s (7) Where λ is the weight coefficient of the mark, and L is the Laplace matrix of the cutout; Step 3: Propagate the central sub-aperture image alpha map through the light field alpha map propagation model provided by equation (8) to obtain the light field alpha map alpha u,v (x,y); In the formula, is the alpha map corresponding to the sub-aperture image under the central viewpoint (u0, v0), disp(x, y) is the disparity corresponding to the image point (x, y) in the central sub-aperture image, Δu and Δv are the distances between any viewpoint (u, v) and the central viewpoint (u0, v0) in the u and v directions, respectively.

2. The spatial domain evaluation method for light field cutout based on empty angle consistency according to claim 1, characterized in that: The calculation formula of the disparity map disp(x,y) in step 2 is formula (2): disp(x,y)=F / Z(x,y)+F (2) Where F is the focal length of the camera and Z(x,y) is the depth map of the scene.

3. A spatial domain evaluation device for light field cutout based on empty angle consistency, characterized in that: include: A central sub-aperture image extraction unit, which is used to extract a central sub-aperture image of the light field data; A computing unit, which is used to calculate a disparity map of the light field data and an alpha map of a sub-aperture image at a center of the light field data; A light field cutout unit, which is used to propagate the central sub-aperture image alpha map to obtain a light field alpha map; An evaluation unit is used to evaluate the spatial angle consistency of the light field data in the light field alpha map by using the spatial angle consistency evaluation index S-cons obtained in the spatial domain provided by formula (1): the index S-cons value range is (0-1), the closer the index S-cons is to 1, the smaller the variance of the boundary area is, the better the spatial angle consistency of the light field data is, and the closer the index S-cons is to 0, the worse the spatial angle consistency of the light field data is; Where N EPI is the number of EPIs in the light field alpha map, E j is the corresponding j-th EPI. There are l boundary areas in each EPI, denoted as Ω l , N l is the Ω in the jth EPI l The total number of pixels in the area, E ji is the i-th pixel in the j-th EPI of the light field alpha image, is the Ω in the jth EPI l The mean of the pixel values ​​at the boundary; in: The computing unit includes: The central sub-aperture image alpha map calculation subunit is used to obtain the central sub-aperture image alpha map according to the central sub-aperture image of the light field data through the following specific alpha closed solution method: Step 21: The confidence q of any pixel i in the disparity map disp(z,y) is i Set as formula (3): In the formula, p i is the disparity value of any pixel i in the disparity map disp(x,y), D=max(disp(x,y)) is the maximum value in the disparity map, and d=min(disp(x,y)) is the minimum value in the disparity map; Step 22, set confidence thresholds t1 and t2, when confidence q i When it is greater than t1, the area belongs to the foreground. i When it is less than t2, the area belongs to the background, and the other areas are uncertain areas in the trigraph. Then the trigraph described by formula (4) is obtained: Step 23, convert the three-part image into I1 and I2, where the pixels at the foreground and background marks in I1 are 1, and the pixel values ​​at other places are 0, and I1 is converted into a sparse diagonal matrix, which is D s , the pixel value of the foreground mark in I2 is 1, and the other pixel values ​​are 0. Converting it into a column vector is b s : Step 24, solve the alpha map of the central sub-aperture image by equation (7): α=λ(L+λ) -1 b s (7) Where λ is the weight coefficient of the mark, and L is the Laplace matrix of the cutout; The light field cutout unit propagates the central sub-aperture image alpha map through the light field alpha map propagation model provided by formula (8) to obtain the light field alpha map alpha u,v (x,y); In the formula, is the alpha map corresponding to the sub-aperture image under the central viewpoint (u0, v0), disp(x, y) is the disparity corresponding to the image point (x, y) in the central sub-aperture image, Δu and Δv are the distances between any viewpoint (u, v) and the central viewpoint (u0, v0) in the u and v directions, respectively.

4. The spatial domain evaluation device for light field cutout based on empty angle consistency according to claim 3, characterized in that: The computing unit includes: The disparity calculation subunit is used to calculate the disparity map disp(x,y) according to the depth map of the light field data by formula (2): disp(x,y)=F / Z(x,y)+F (2) Where F is the focal length of the camera and Z(x,y) is the depth map of the scene.

Images