A method for quality detection of a light field image

Abstract

The application discloses a quality detection method of a light field image; the method is as follows: 1, a measured light field image is decomposed to obtain a sub-aperture image sequence; 2, the sub-aperture image sequence is paired two by two to obtain multiple left-right view pairs; each left-right view pair generates a light field three-dimensional view; 3, a neural network for quality evaluation is constructed; and the neural network is used to generate a quality score S of the measured light field image. The application comprehensively considers the image perception experience and the image recognizability of the light field image, and can accurately analyze the quality of the light field image.

Description

Technical Field

[0001] This invention belongs to the field of image and 3D detection technology, and specifically designs a method for quality detection of light field images. Background Technology

[0002] Light field imaging is an emerging media format that has garnered widespread attention in recent years. Unlike traditional image processing techniques, light field imaging, based on image rendering technology, records the distribution of light rays in space, capturing richer information from real-world scenes. Because it records light rays rather than pixels, light field images can provide spatial and depth information that is difficult to obtain with traditional images, ideally offering viewers a six-degrees-of-freedom viewing experience. However, from generation to reaching the viewer, light field images are often accompanied by many image processing techniques, such as image compression and encoding. These techniques inevitably introduce varying degrees of distortion into the original image. This distortion not only affects the viewer's experience but may also influence crucial decisions, such as using radar imagery to determine terrain in military operations. Therefore, better evaluating the quality of light field images is essential for guiding the acquisition, processing, and application of light field technologies.

[0003] Currently, quality assessment of light field images can be broadly categorized into subjective and objective assessments. Subjective assessment methods typically involve human intervention and the design of a viewing mode, allowing volunteers to adapt to the viewing mode and quality assessment metrics to score the light field image. Clearly, subjective assessment is resource-intensive and time-consuming, making it unsuitable for large-scale applications. Objective assessment methods, on the other hand, consider designing an effective quality assessment model that takes the light field image or its variants as input and directly generates a quality score through model processing. This approach offers greater feasibility compared to subjective assessment.

[0004] However, considering the unique properties of light field images, especially the spatial and depth information lacking in traditional planar images, quality assessment of light field images cannot directly rely on methods used for planar image quality assessment. Existing solutions for light field image quality assessment mostly evaluate only a few attributes of the light field, thus failing to achieve a good balance between comprehensiveness and accuracy. Particularly in the currently popular field of deep learning, existing light field image quality assessment methods do not utilize techniques such as neural networks to improve assessment effectiveness. Therefore, it is necessary to propose more effective solutions based on current research advancements and the unique properties of light field images. Summary of the Invention

[0005] The purpose of this invention is to address the limitations of existing light field image evaluation methods, which have high requirements for image quality, by providing a deep learning-based method for light field image quality evaluation. This method considers the most important three-dimensional attribute of the light field image and, by combining deep learning strategies, achieves a weighted balance between the three-dimensional and planar attributes of the light field image in quality evaluation, helping users select higher-quality light field images.

[0006] In a first aspect, the present invention provides a method for quality detection of a light field image, comprising the following steps:

[0007] Step 1: Decompose the measured light field image to obtain a sub-aperture image sequence.

[0008] Step 2: Pair the sub-aperture image sequences in pairs to obtain multiple left and right view pairs; each left and right view pair generates a three-dimensional light field view.

[0009] Step 3: Construct a neural network for quality assessment; and use the neural network to generate the quality score S of the measured light field image.

[0010] 3-1. Construct a network preprocessing layer to denoise the multiple 3D light field views CI generated in step 2, generating denoised images CI1. Perform noise detection on each 3D light field view CI and its corresponding image CI1 to obtain a score S1.

[0011] 3-2. Construct a global convolutional layer Conv1 to extract the global contour information of image CI1 and form the contour matrix Mat0.

[0012] 3-3. Construct a local convolutional layer Conv2 to extract local contour information of image CI1, obtaining n×n contour matrices mat. i Where n is input by the operator, a suggested value is between 10 and 20. The input size of the convolution kernel for the local convolutional layer Conv2 is... n 2 Matrix mat i The connection calculation forms the contour matrix Mat1.

[0013] 3-4. Construct a pooling layer Pool1 to perform pooling calculations on the contour matrices Mat0 and Mat1, generating a matrix Mat. p .

[0014] 3-5. For the matrix Mat p Significance analysis and quality analysis were performed separately.

[0015] (1) MAt p Decomposed into multiple regions of equal size, each region is represented by a block matrix mat. j express.

[0016] (2) Constructing a block saliency analysis network. The working process of the saliency analysis network is as follows:

[0017] Step 1: For each block matrix mat j Perform a high-pass filter operation to remove the block matrix mat. j The high-frequency signal components are used to obtain the matrix mat. j1 ;

[0018] Step 2: Divide the block matrix mat j and matrix mat j1 Convert from RGB color space to LAB color space.

[0019] Step 3: For matrix mat j Calculate the LAB space chromaticity average vector I m =[L m A m B m Simultaneously calculate the three-channel mean and matrix mat. j1 The sum of the Euclidean distances is used as the significance matrix.

[0020] Step 4: Calculate the average of all elements in the significance matrix to obtain the block weights w. j .

[0021] Step 5: Process all block matrices mat j The obtained block weights w j The normalization process is performed using the following formula:

[0022]

[0023] (3) Constructing the block quality analysis network. The working process of the block quality analysis network is as follows:

[0024] Step 1: Construct fractional compression blocks.

[0025] Step 2: Combine all the partitioned matrices mat j The input is fed into a fractional compression block to obtain the block matrix mat. j mass fraction s j .

[0026] (3) The fraction S2 is calculated as follows:

[0027]

[0028] Among them, w 1j The weights are as shown in equation (9).

[0029] 3-6. Calculate the mass fraction S of the three-dimensional view of the light field. p as follows:

[0030] S p =W s1 ×S1+W s2 ×S2 Equation (10)

[0031] Among them, W s1 W s2 The weights are divided into image viewing experience and image recognizability; W s1 +W s2 =1.

[0032] 3-7. Calculate the mass fraction S for all three-dimensional views of the light field. p The average value is taken as the light field image quality score S of the measured light field image. The higher the light field image quality score S, the higher the quality of the measured light field image.

[0033] As a preferred option, the specific process of step 1 is as follows:

[0034] 1-1. Decompose the measured light field image using a sub-aperture image decomposition algorithm;

[0035] 1-1-1. Extraction of the center point of the original light field image. First, a microlens white image is obtained; then, the OTSU algorithm is used to find the optimal threshold of the white image; based on the optimal threshold, the positions of multiple white image contours are calculated; finally, the gray-scale centroid method is used to calculate the center point of the microlens for each contour, which is taken as the center point of the original light field image.

[0036] 1-1-2. Sort the center points of the original light field image and select a central region. Sort the center points from left to right and from top to bottom. At the same time, expand the center points into a central region, that is, select a central region of a preset size with the center point pixel as the center. The width i of the central region is less than 1 / 2 of the width of the corresponding white image outline.

[0037] 1-1-3. Extract all central regions according to the sorting order of the center points and piece together the sub-aperture image.

[0038] 1-2. By adjusting the parameters of the sub-aperture image decomposition algorithm in step 1-1, multiple sub-aperture images are generated; each sub-aperture image is arranged into a sub-aperture image sequence according to the viewing angle from left to right and from top to bottom.

[0039] Preferably, the microlens white image is obtained by capturing the calibration plate with a light field camera;

[0040] As a preferred option, the specific process of step 2 is as follows:

[0041] 2-1. Pair the sub-aperture image sequences generated in step 1 to obtain multiple left and right view pairs.

[0042] 2-2. For each pair of left and right views, a three-dimensional reconstruction algorithm is used to construct N / 2 light field three-dimensional views.

[0043] Preferably, the pairing process in step 2-1 is as follows: Set a sequence number difference N. d The value, based on the difference N in the sequence number. d Perform pairwise matching on each sub-aperture image; the sequence numbers differ by N. d The two sub-aperture images are grouped together, with the smaller number serving as the left view and the larger number serving as the right view.

[0044] As a preferred embodiment, the 3D reconstruction algorithm described in step 2-2 is as follows:

[0045] (1) Calculate the kernel factor Core(x,y) of each pixel coordinate in the left and right views respectively using the high-pass filtering algorithm, and obtain the kernel factor Core of the left view. L (x,y) and the convolution kernel factor Core of the right view R (x,y).

[0046] (2) Calculate the offset value d between the left and right view pairs.

[0047] (3) Calculate the three-dimensional weighting factor Wei for the left and right views. L (x,y) and Wei R (x,y) is as follows:

[0048]

[0049]

[0050] (4) The calculated three-dimensional view of the light field is as follows:

[0051] CI(x,y)=Wei L (x,y)×I L (x,y)+Wei R (x+d,y)×I R (x+d,y) Formula (3)

[0052] Where CI(x,y) represents the pixel value of the pixel point (x,y) in the 3D view of the light field; I L (x,y) represents the pixel value at (x,y) in the left view; I R (x+d,y) represents the pixel value at (x+d,y) in the right view.

[0053] As a preferred embodiment, the expression for fraction S1 in step 3-1 is as follows:

[0054]

[0055]

[0056] Where W and H represent the width and height of the 3D light field view CI, respectively. MPixel is the number of pixels in the 3D light field view CI; MSE is the mean square error. CI(x,y) and CI1(x,y) are the pixel values ​​of the 3D light field view CI and the image CI1 at pixel (x,y), respectively.

[0057] Preferably, in steps 3-4, the pooling layer Pool1 uses the following PyTorch function:

[0058] max_pool2d(input,kernel_size,stride)

[0059] Wherein, input is the matrix obtained by concatenating the contour matrices Mat0 and Mat1; kernel_size is the kernel size of the pooling layer; and stride is the stride information of the pooling layer.

[0060] Matrix Mat p Size (width) p height p The calculation formula for ) is as follows:

[0061]

[0062]

[0063] In a second aspect, the present invention provides a computer device comprising a memory and at least one processor; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the aforementioned detection method.

[0064] Thirdly, the present invention provides a readable storage medium storing computer instructions; when executed by a processor, the computer instructions are used to implement the detection method as described above.

[0065] The beneficial effects of this invention are as follows:

[0066] 1. This invention effectively improves the quality assessment of light field images by introducing and improving popular methods in the fields of planar images and stereo vision.

[0067] 2. This invention improves popular neural network technology by adapting it to use light field images as model input, which not only simplifies the model's workflow but also improves the accuracy of the model's output.

[0068] 3. By designing a three-dimensional view model, this invention can effectively evaluate the properties of light field images that distinguish them from planar images.

[0069] 4. By designing a mapping scheme between the model and practical applications, this invention achieves the effect of enabling the model to be better applied to specific purposes. Attached Figure Description

[0070] Figure 1 This is a flowchart of the present invention;

[0071] Figure 2 This is a structural diagram of the neural network used in this invention;

[0072] Figure 3 This is a structural diagram of the saliency analysis network used in this invention. Detailed Implementation

[0073] The present invention will be further described below with reference to the accompanying drawings.

[0074] like Figure 1 As shown, a method for quality detection of a light field image includes the following steps:

[0075] Step 1: Decompose the light field image into sub-views

[0076] 1-1. By designing a sub-aperture image decomposition algorithm, the original measured light field image (microlens image) is decomposed into a sequence of sub-aperture images; the sub-aperture image is a planar image decomposed from the light field image, reflecting the planar properties of the light field image. The steps of the sub-aperture image decomposition algorithm are as follows:

[0077] (1) Extraction of the center point of the original light field image. First, a microlens white image is input. The white image can be obtained by taking a picture of a uniform white board or a suitable calibration board with a light field camera. Then, the OTSU algorithm is used to find the optimal threshold of the white image, that is, the gray value of the image is divided into two sets according to the gray level. The difference in gray value between the two sets is the largest, and the difference in gray value within the set is the smallest. Then, according to the optimal threshold, the positions of multiple white image contours are calculated. Finally, the gray centroid method is used to calculate the center point of the microlens for each contour, which is used as the center point of the original light field image.

[0078] (2) Sort the center points of the original light field image and select the central region. Sort the center points from left to right and from top to bottom. At the same time, expand the center points into a central region, that is, select an i×i region with the center point pixel as the center, which is called the central region. i is the preset width of the central region; the value of i is less than 1 / 2 of the contour width calculated in step (1).

[0079] (3) Extract all central regions according to the sorting order of the center points and piece together the sub-aperture image.

[0080] 1-2. By adjusting the parameters of the sub-aperture image decomposition algorithm in step 1-1 (i.e., the gray value threshold and the size of the central region width i), multiple sub-aperture images are generated; each sub-aperture image is arranged into a sub-aperture image sequence according to the viewing angle from left to right and from top to bottom.

[0081] Step 2: Synthesize the 3D view

[0082] 2-1. Pair the sub-aperture image sequences generated in step 1, and select one as the left view and the other as the right view.

[0083] The sub-aperture image sequence generated in step 1 is ordered from smallest to largest, with the image viewing angle gradually and naturally shifting from left to right. Therefore, the pairing method is as follows: set a sequence number difference N. d The value, based on the difference N in the sequence number. d Perform pairwise matching on each sub-aperture image; the sequence numbers differ by N. d Two sub-aperture images are grouped together, with the smaller index serving as the left view and the larger index serving as the right view. Therefore, for the generated N sub-aperture images, a sequence of N / 2 left and right view pairs can be constructed.

[0084] 2-2. For each pair of left and right views, a 3D reconstruction algorithm is applied to construct N / 2 light field 3D views. The specific 3D reconstruction algorithm is as follows:

[0085] (1) Calculate the kernel factor Core(x,y) of each pixel coordinate in the left and right views respectively using the high-pass filtering algorithm, and obtain the kernel factor Core of the left view. L (x,y) and the convolution kernel factor Core of the right view R (x,y); The high-pass filtering algorithm can be implemented using the getGaborKernel method in the Python function library or the OpenCV vision library.

[0086] (2) Based on the difference N between the sequence numbers of the left and right views d Calculate the offset value d between the left and right views. Since step 2-1 mentions that the image viewpoint moves from left to right, the object described by the pixel at position (x, y) in the left view will appear at position (x+d, y) in the right view. Because the left and right views only experience horizontal viewpoint translation, the y-coordinate remains unchanged.

[0087] (3) Based on the calculated kernel factor Core of each pixel in the left and right views L (x,y) and Core R(x, y), calculate the three-dimensional weight factors Wei for the left and right views respectively. L (x,y) and Wei R (x, y). The calculation formulas are shown in formulas (1) and (2):

[0088]

[0089]

[0090] (5) Calculate the three-dimensional view of the light field. The calculation formula is shown in formula (3):

[0091] CI(x,y)=Wei L (x,y)×I L (x,y)+Wei R (x+d,y)×I R (x+d,y) Formula (3)

[0092] Where CI(x,y) represents the pixel value of the 3D view of the light field at (x,y); I L (x,y) represents the pixel value at (x,y) in the left view; I R (x+d,y) represents the pixel value at (x+d,y) in the right view; Wei L (x,y) and Wei R (x+d,y) can be calculated using formulas (1) and (2).

[0093] Step 3: Design the neural network. The structure of the neural network is as follows: Figure 2 As shown.

[0094] 3-1. Construct a network preprocessing layer to denoise N / 2 light field 3D views CI respectively, generating a denoised image CI1. Obtain a score S1 by performing noise detection on the light field 3D views CI and image CI1. Noise detection is performed by calling the OpenCV function library. The score S1 is obtained using the formula (4) for the peak signal-to-noise ratio of the image.

[0095]

[0096]

[0097] Where W and H represent the width and height of the 3D light field view CI, respectively. MPixel is the number of pixels in the 3D light field view CI; MSE is the mean squared error, which is a very common loss function.

[0098] 3-2. Construct a global convolutional layer Conv1 to extract the global contour information of image CI1, forming the contour matrix Mat0. Since global information is extracted, the input size of the convolution kernel of the local convolutional layer Conv1 is the same as the size of image CI1, i.e., W×H. Conv1 can be implemented by encapsulating the PyTorch function torch.nn.conv(W,H,W,H), and the generated matrix Mat0 has a size of W×H.

[0099] 3-3. Construct a local convolutional layer Conv2 to extract local contour information from image CI1. The input size of the convolution kernel of the local convolutional layer Conv2 is... n is input by the operator, with a suggested value between 10 and 20; therefore, the local contour information consists of n×n contour matrices mat. i (1≤i≤n 2 ) constitutes, n 2 Matrix mat i The contour matrix Mat1 is constructed by connecting the components using the Concat function.

[0100] 3-4. Construct a pooling layer Pool1 to perform pooling calculations on the contour matrices Mat0 and Mat1, generating a matrix Mat. p The pooling layer's function is to process the global contour information Mat0 and the local contour information Mat1, outputting a matrix Mat. p This will highlight the common parts of global and local contour information, known as "shared contour" information. The quality of the "shared contour" is an important part of the light field image quality. The output matrix Mat after pooling... p The size is adjusted according to the detection requirements. In this embodiment, the pooling layer Pool1 uses the PyTorch function max_pool2d(input, kernel_size, stride). Here, input is the matrix calculated by concatenating the contour matrices Mat0 and Mat1 using the Concat function; kernel_size is the kernel size of the pooling layer, which determines the size of the "common contours"; and stride is the pooling stride information. In this embodiment, the kernel_size and stride are assigned by the operator based on the size of the original image.

[0101] Matrix Mat p Size (width) p height p The calculation formula for ) is as follows:

[0102]

[0103]

[0104] 3-5. For the matrix Mat calculated in step 3-4 p The steps for conducting significance analysis and quality analysis are as follows:

[0105] (1) Design a block-based strategy to divide Mat p Decomposed into n×n regions of equal size, each region is represented by a block matrix mat p,j It means that 1≤j≤n 2 ;

[0106] The matrix Mat calculated in steps 3-4 p Calculate the dimensions of mat j Size Nm j As in equation (8):

[0107] N mj =(width) p ×height p ) / m 2 Equation (8)

[0108] N mj For the block matrix mat j The number of pixels.

[0109] (2) Figure 3 As shown, a saliency analysis network is constructed as the first branch network. The saliency analysis network adopts the Fourier Transform (FT) algorithm, and its working process is as follows:

[0110] Step 1: For each block matrix mat output in step (1) j As network input, a high-pass filter is performed to remove the block matrix mat. j The high-frequency signal components, the output matrix is ​​mat j1 ;

[0111] Step 2: Convert the matrix mat j and matrix mat j1 Convert from RGB color space to LAB color space. This can be achieved by calling the `rgb2lab` method in the Matlab library.

[0112] Step 3: For matrix mat j Calculate the LAB space chromaticity average vector I m =[L m A m B m Simultaneously calculate the three-channel mean and matrix mat. j1 The sum of the Euclidean distances is used as the significance matrix, and the specific calculation formula is as follows:

[0113] sumj (x,y)=||I m -I m1 (x,y)||

[0114] Where, sum j (x,y) represents matrix mat j The saliency value of the pixel with coordinates (x, y); I m1 (x,y) represents the vector of pixel (x,y) after high-pass filtering.

[0115] Step 4: Process the block matrix mat j The saliency value sum of all pixels within the range j The summation and averaging of (x, y) yields the block weights w. j ,Right now

[0116] (4) Construct a block quality analysis network as the second branch network. The block quality analysis network predicts the quality scores of the multiple sub-regions decomposed in step (1) individually. The specific steps are as follows:

[0117] Step 1: Design the score compression block. The score compression block consists of convolutional layers of a neural network, using a ResNet network. The number of convolutional layers in the score compression block is set by the operator; more layers result in higher accuracy but slower evaluation speed.

[0118] Step 2: Convert each block matrix mat output in step (1) into a single matrix. j The input is fed into the fractional compression block, and the output s of the fractional compression block is... j It is a constant, namely mat j The quality score.

[0119] (3) Calculate the score S2 according to the significance weight calculation formula (9).

[0120] in

[0121] 3-6. Calculate the mass fraction S of the three-dimensional view of the light field. p The calculation formula is as follows:

[0122] S p =W s1 ×S1+W s2 ×S2 Equation (10)

[0123] Among them, W s1 W s2 The weights are divided into image viewing experience and image recognizability; W s1 +W s2=1. Since image quality mainly depends on two dimensions: image recognizability and visual experience, W can be adjusted according to the actual detection scenario and requirements. s1 W s2 Perform the assignment.

[0124] 3-7. Steps 3-1 to 3-6 yielded the quality fraction S of N / 2 three-dimensional views of the light field. p The light field image quality score S for the entire left and right view sequence is derived by averaging the quality scores of all the light field 3D views.

[0125]

[0126] The higher the quality score S of the light field image, the higher the quality of the measured light field image.

[0127] The design training scheme and evaluation criteria for this embodiment are as follows:

[0128] 1. Package the network structure. Package the neural network described in step 3 into a Python script. For the packaged script, you only need to pass the local path of the light field image as a parameter, and calling the script code will output the light field image quality score S;

[0129] 2. Develop the training program. The training program needs to consider the following aspects:

[0130] (1) Selection of training and test sets. A hybrid selection method is used for training, with the training and test sets being approximately 3:1 in size. During training, the network is trained according to the sequence number of the 3D light field view. 3d The result of taking the remainder when divided by 3 determines whether the view is a training set view or a test set view, i.e.: No. 3d %3 = 0, this view is a test set view; No 3d If %3≠0, then the view is the training set view.

[0131] (2) Number of training rounds and step size. Generally speaking, the more training rounds, the better the effect. However, in order to avoid overfitting of the neural network, the number of training rounds should generally not exceed 50. In each training round, a different step size needs to be set to ensure the robustness of the network.

[0132] (3) Define the loss function and threshold. Here, the L2 loss function is used, which is the most commonly used and stable loss function in image processing and pattern recognition. The specific formula is as follows:

[0133]

[0134] Where n represents the number of images selected for backpropagation of the training set, i.e., the loss function is calculated once for every n selected 3D views of the light field; Sri This represents the reference value of the light field image corresponding to the 3D view of the light field. Since in the field of image quality assessment, each image in the dataset provides a reference value, Sr... i This refers to the reference value provided by the light field dataset; S i It comes from the output of the neural network described in step 3.

[0135] During training, we also need to set a loss threshold. When the loss function calculation result exceeds the threshold The results of this training round will not be included in backpropagation. Generally speaking, A smaller threshold range results in more accurate network training, but it can also lead to a large-scale invalidation of the dataset during the initial training phase. Therefore, in practice, different thresholds are typically used in each training round. The threshold to be used in practice is selected by statistically analyzing the failure rate of the dataset during training.

[0136] 3. Design Evaluation Criteria. The evaluation criteria consist of two aspects:

[0137] (1) W in steps 3-6 s1 and W s2 Specific proportion. W s1 The visual experience conveyed by W s2 This reflects the image's recognizability. Generally speaking, if the purpose of quality analysis is to better obtain the image's contour information, such as for terrain observation in military applications, a higher W value is required. s2 Weighting; if the purpose is to optimize the viewing experience, a higher weighting is required. s1 Weights.

[0138] (2) Establish the mapping relationship between network output and actual evaluation metrics. Since the reference values ​​in image quality evaluation datasets are often represented by signal MOS values, and MOS is only used to evaluate the general quality of signals, it cannot provide practical evaluation purposes. Therefore, it is necessary to divide the relationship between MOS values ​​and actual scores according to practical applications. For example, MOS values ​​are generally in the range of -10 to 10, while actual scores are often in the range of 0 to 100. Therefore, by establishing an interval mapping relationship, the MOS values ​​output by the network can be mapped to the actual scores in (0, 100).

Claims

1. A method of quality detection of a light field image, characterized in that, The method comprises the following steps: Step 1, decompose the measured light field image to obtain a sub-aperture image sequence; Step 2, pair the sub-aperture image sequence two by two to obtain a plurality of left-right view pairs; each left-right view pair generates a light field three-dimensional view; Step 3, construct a neural network for quality assessment; and use the neural network to generate a quality score S of the measured light field image; 3-1. Constructing a network preprocessing layer, generating a plurality of light field three-dimensional views from step 2 Respectively, the denoising processing is performed to generate the denoised image ; Respectively, each light field three-dimensional view and the corresponding image Noise detection is performed to obtain a score ; The score According to the pixel value of the light field three-dimensional view and the corresponding image is obtained; 3-2. Constructing global convolutional layers , extract global contour information of the image , and construct a contour matrix ; 3-3. Constructing the local convolution layer , extract the local contour information of the image , and obtain contour matrices ; the input size of the convolution kernel of the local convolution layer is ; W and H represent the width and height of the light field three-dimensional view CI respectively; contour matrices are connected to calculate the contour matrix ; 3-4. Constructing the pooling layer , the contour matrix , and perform the pooling calculation to generate the matrix ; 3-5. On the matrix Significance analysis and quality analysis are performed separately; (1) the following is decomposed into a plurality of regions of the same size, each region being represented by a block matrix ​ (2) Construct a block saliency analysis network; the working process of the block saliency analysis network is as follows: step1: high-pass filtering each block matrix to remove high frequency signal components of the block matrix to obtain matrix ; step2: block matrix and matrix from color space to color space; step3: compute the sum of the Euclidean distances of the matrix compute spatial chrominance mean vector and the sum of the Euclidean distances of the matrix of the three channel means as the saliency matrix; step 4: obtain the average value of all elements in the saliency matrix to obtain the block weight ; Step 5: Normalization of all the block matrices The obtained block weight The normalization processing is performed, and the processing formula is as follows: Equation (9) (3) Construct a block quality analysis network; the working process of the block quality analysis network is as follows: Step 1: Construct a score compression block; step2: input all the block matrices to the fractional compression block to obtain the quality scores of the block matrices ; (3) Calculating the score As follows: Equation (10) wherein is the weight of formula (9), 3-6. Calculating a quality score for a light field three-dimensional view As follows: Equation (11) Wherein, , Divided into image visual experience and image recognition degree of weight; ​ 3-7. Calculate the mass fraction of all 3D views of the light field. The average value is taken as the light field image quality score S of the measured light field image; the higher the light field image quality score S, the higher the quality of the measured light field image.

2. The method of claim 1, wherein: The specific process of step 1 is as follows: 1-1. Decompose the measured light field image using a sub-aperture image decomposition algorithm; 1-1-1. Light field original image center point extraction; first, obtain a microlens white image; then, use the OTSU algorithm to obtain the optimal threshold value of the white image; according to the optimal threshold value, calculate a plurality of white image contour positions; finally, use the gray gravity center method to calculate the microlens center point of each contour as the light field original image center point; 1-1-2. Sort the center points of the original light field image and select the central region; sort the center points from left to right and from top to bottom; simultaneously, expand the center points into a central region, that is, select a central region of a preset size with the center point pixel as the center; the width of the central region... Less than 1 / 2 the width of the corresponding white image outline; 1-1-3. According to the sorting order of the center points, all center regions are taken out to splice sub-aperture images; 1-2. Generate a plurality of sub-aperture images by adjusting the parameters of the sub-aperture image decomposition algorithm in step 1-1; each sub-aperture image is arranged from left to right and from top to bottom to form a sub-aperture image sequence according to the viewing angle.

3. The method of claim 2, wherein: The microlens white image is obtained by shooting a calibration board by a light field camera.

4. The method of claim 1, wherein: The specific process of step 2 is as follows: 2-1. Pair the sub-aperture image sequence generated in step 1 two by two to obtain a plurality of left-right view pairs; 2-2. For each left-right view pair, use a three-dimensional reconstruction algorithm to construct N / 2 light field three-dimensional views.

5. The method of claim 4, wherein: The pairing process in step 2-1 is as follows: Set a sequence number difference N. d The value, based on the difference N in the sequence number. d Perform pairwise matching on each sub-aperture image; the sequence numbers differ by N. d The two sub-aperture images are grouped together, with the smaller number serving as the left view and the larger number serving as the right view.

6. The method of claim 4, wherein: The three-dimensional reconstruction algorithm in step 2-2 is as follows: (1) According to the high-pass filtering algorithm, the convolution kernel factor of each pixel coordinate in the left and right views is calculated respectively , to obtain the convolution kernel factor of the left view and the convolution kernel factor of the right view ; (2) Calculate the offset value d between the left-right view pairs; (3) calculating the three-dimensional weight factors for the left and right views and as follows: Formula (1) Equation (2) (4) Calculate the light field three-dimensional view as follows: Equation (3) in, Pixels representing a 3D view of a light field Pixel values; View in Pixel value at; Indicates the right view is in The pixel value at that location.

7. The method of claim 1, wherein: Fraction in Step 3-1 The expression of the fraction is as follows: Equation (4) Equation (5) wherein, is the number of pixels of the light field three-dimensional view; MSE is the mean square error; , is the number of pixels of the light field three-dimensional view; is the image is the pixel value of the pixel point .

8. The method of claim 1, wherein: In step 3-4, the pooling layer Using The function is as follows: ； wherein, is a contour matrix and connects the calculated matrix; is a kernel size of the pooling layer; is a step information of the pooling; matrix of the dimensions The formula for calculating the size of the matrix is as follows: +1 Equation (6) +1 Equation (7).

Images