A method and system for evaluating the quality of light field images based on 4D wavelet transform

Abstract

This invention provides a method for evaluating the quality of light field images based on 4D wavelet transform. The method includes acquiring sub-aperture image arrays of a reference and distorted light field image, and converting them into corresponding sub-aperture gradient image arrays. Four-dimensional wavelet decomposition is performed on each of the two sub-aperture gradient image arrays to obtain 4D wavelet coefficients for 16 different sub-bands corresponding to each sub-aperture gradient image array. The 4D wavelet coefficients of each sub-band of the two sub-aperture gradient image arrays are combined to obtain 16 similarity error maps. A space-angle weighted strategy is used to aggregate the similarity values ​​of each similarity error map to obtain scores for the 16 sub-bands. All scores are used to construct a one-dimensional feature vector, which is then imported into a pre-trained support vector regression model to obtain the quality score of the distorted light field image. Implementing this invention effectively captures the intrinsic relationship between space and angle in light field images, exhibiting good performance in evaluating light field image quality.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, and in particular to a method and system for evaluating the quality of light field images based on 4D wavelet transform. Background Technology

[0002] Light field images, as a novel imaging technique, contain both spatial and angular information about a scene. They record rich scene information, thus attracting widespread attention from industry and academia. As a 4-dimensional signal, the high-dimensional data representation of a light field image provides powerful capabilities for scene understanding, solving traditional computer vision problems (such as 3D scene reconstruction, refocusing, depth estimation, and semantic segmentation). However, the high-dimensional nature also brings new challenges to light field image processing techniques. Distortion inevitably occurs during the processing, leading to a degradation in the visual quality of the light field image.

[0003] Currently, methods for detecting the quality of light field images can be divided into two types: one is to use traditional image quality assessment methods to predict the quality of sub-aperture images of the light field image and then average the quality of all sub-aperture images as the quality of the light field image; the other is to design methods specifically for light field image quality assessment. However, the former is not suitable for light field image quality assessment because it does not consider the special structure of the light field image. The latter has become the main focus of research at this stage, but it cannot effectively capture the intrinsic relationship between the space and angle of the light field image, resulting in lower performance in light field image quality assessment. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a light field image quality evaluation method and system based on 4D wavelet transform, which can effectively capture the intrinsic relationship between the space and angle of the light field image and has good light field image quality evaluation performance.

[0005] To address the aforementioned technical problems, this invention provides a method for evaluating the quality of light field images based on 4D wavelet transform, the method comprising the following steps:

[0006] Obtain the sub-aperture image array of the reference light field image and the sub-aperture image array of the distorted light field image, and convert them into the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively.

[0007] Four-dimensional wavelet decomposition is performed on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, to obtain the four-dimensional wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the four-dimensional wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the distorted light field image.

[0008] The 4D wavelet coefficients of each sub-band of the reference sub-aperture gradient image array are combined with the 4D wavelet coefficients of the corresponding distorted sub-aperture gradient image array to obtain the similarity error map of the 4D wavelet coefficients of the 16 sub-bands.

[0009] A spatial-angle weighting strategy is adopted to aggregate the similarity values ​​of each pixel in each similarity error map to obtain the scores of the 4-dimensional wavelet coefficients of 16 sub-bands;

[0010] The scores of the 4-dimensional wavelet coefficients of the 16 sub-bands are used to construct a one-dimensional feature vector, which is then imported into a pre-trained support vector regression model to obtain the quality score of the distorted light field image.

[0011] The specific steps of acquiring the sub-aperture image array of the reference light field image and the sub-aperture image array of the distorted light field image, and converting them into the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, include:

[0012] Let the reference light field image be denoted as L. r The distorted light field image is denoted as L. d The sub-aperture image array of the reference light field image is denoted as The sub-aperture image array of the distorted light field image is denoted as Among them, the reference light field image L r And distorted light field image L d Both are 4-dimensional signals, which can be represented as a set of sub-aperture images, i.e., an array of sub-aperture images of the reference light field image. Sub-aperture image arrays with distorted light field images u and v represent the angular planes of the light field image. and The sub-aperture image arrays of the reference light field image and the sub-aperture image arrays of the distorted light field image are respectively represented in the sub-aperture image at the angular coordinates (u,v);

[0013] Through formula and formula Sub-aperture image array for calculating the reference light field image A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). Vertical difference image with reference

[0014] Through formula and formula Sub-aperture image array for calculating distorted light field images A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). and vertical difference image

[0015] Through formula and formula The sub-aperture gradient image of the reference light field image is calculated. Sub-aperture gradient image of distorted light field image

[0016] Sub-aperture gradient image based on reference light field image Sub-aperture gradient image of distorted light field image Sub-aperture gradient image arrays obtained from the reference light field image Sub-aperture gradient image array with distorted light field image

[0017] The specific steps of performing 4D wavelet decomposition on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image include:

[0018] The one-dimensional wavelet transform of a one-dimensional signal f(t) is defined as follows: Wherein, ψ(t) is defined as a is a scale parameter and τ is the shift parameter and j is an integer;

[0019] Perform 1D wavelet transforms along four dimensions on the sub-aperture gradient image arrays of the reference light field image and the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image. The sub-aperture gradient image array corresponding to the distorted light field image has 16 different sub-bands of 4D wavelet coefficients. in, These are the 4-dimensional wavelet subband coefficients of the Lth subband of the sub-aperture gradient image array of the reference light field image. is the 4-dimensional wavelet coefficient of the Lth sub-band of the sub-aperture gradient image array of the distorted light field image, where L = 1, 2, ..., 16.

[0020] Among them, through the formula Calculate the similarity error map of the 4D wavelet coefficients for each sub-band; where T1 is a fixed decimal with a value of 0.00001; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the reference light field image; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the distorted light field image.

[0021] The specific steps of employing a space-angle weighting strategy to aggregate the similarity values ​​of each pixel in each similarity error map to obtain the scores of the 4D wavelet coefficients of the 16 sub-bands include:

[0022] Through formula The spatial weights of each similarity error map are calculated. Where max() represents the maximum value operation;

[0023] Through formula The angle weights of each similarity error map are calculated.

[0024] Through formula The spatial-angular weight values ​​of each similarity error map are calculated.

[0025] Through formula The fraction of the 4-dimensional wavelet coefficients for each sub-band is calculated.

[0026] The specific steps for constructing a one-dimensional feature vector from the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands and importing it into a pre-trained support vector regression model to obtain the quality score of the distorted light field image include:

[0027] The fractions of the 4D wavelet coefficients of all subbands {q L The first eigenvectors are F = {q1, q2, ..., q}. 16 The one-dimensional feature vector is then used as input to a pre-trained support vector regression model to obtain the quality score Q of the distorted light field image.

[0028] Q = model(F)

[0029] Here, model() is a pre-trained support vector regression model.

[0030] This invention also provides a light field image quality assessment system based on 4D wavelet transform, comprising:

[0031] An angle filtering unit is used to acquire sub-aperture image arrays of a reference light field image and sub-aperture image arrays of a distorted light field image, and convert them into sub-aperture gradient image arrays of the reference light field image and sub-aperture gradient image arrays of the distorted light field image, respectively.

[0032] The wavelet decomposition unit is used to perform 4D wavelet decomposition on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the distorted light field image.

[0033] The similarity error calculation unit is used to combine the 4D wavelet coefficients of the sub-band of each reference sub-aperture gradient image array with the 4D wavelet coefficients of the corresponding distorted sub-aperture gradient image array to calculate the similarity error map of the 4D wavelet coefficients of the 16 sub-bands.

[0034] The spatial angle weighting unit is used to aggregate the similarity values ​​of each pixel in each similarity error map using a spatial-angle weighting strategy to obtain the scores of the 4D wavelet coefficients of 16 sub-bands.

[0035] The quality performance evaluation unit is used to construct a one-dimensional feature vector from the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands and import it into the pre-trained support vector regression model to obtain the quality score of the distorted light field image.

[0036] The angle filtering unit includes:

[0037] The assignment module is used to denote the reference light field image as L. r The distorted light field image is denoted as L. d The sub-aperture image array of the reference light field image is denoted as The sub-aperture image array of the distorted light field image is denoted as Among them, the reference light field image L r And distorted light field image L d Both are 4-dimensional signals, which can be represented as a set of sub-aperture images, i.e., an array of sub-aperture images of the reference light field image. Sub-aperture image arrays with distorted light field images u and v represent the angular planes of the light field image. and The sub-aperture image arrays of the reference light field image and the sub-aperture image arrays of the distorted light field image are respectively represented in the sub-aperture image at the angular coordinates (u,v);

[0038] The reference sub-aperture image array calculation module is used to calculate the sub-aperture image array using the formula. and formula Sub-aperture image array for calculating the reference light field image A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). Vertical difference image with reference

[0039] The distortion sub-aperture image array calculation module is used to calculate the distortion using the formula. and formula Sub-aperture image array for calculating distorted light field images A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). and vertical difference image

[0040] Two sub-aperture gradient image acquisition modules are used to obtain images using formulas. and formula The sub-aperture gradient image of the reference light field image is calculated. Sub-aperture gradient image of distorted light field image

[0041] Two sub-aperture gradient image array generation modules are used to generate sub-aperture gradient images based on a reference light field image. Sub-aperture gradient image of distorted light field image Sub-aperture gradient image arrays obtained from the reference light field image Sub-aperture gradient image array with distorted light field image

[0042] The wavelet decomposition unit includes:

[0043] The wavelet signal definition module is used to define the one-dimensional wavelet transform representation of a one-dimensional signal f(t). Wherein, ψ(t) is defined as a is a scale parameter and τ is the shift parameter and j is an integer;

[0044] The wavelet transform module performs 1D wavelet transforms along four dimensions on the sub-aperture gradient image arrays of the reference light field image and the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image. The sub-aperture gradient image array corresponding to the distorted light field image has 16 different sub-bands of 4D wavelet coefficients. in, These are the 4-dimensional wavelet subband coefficients of the Lth subband of the sub-aperture gradient image array of the reference light field image. is the 4-dimensional wavelet coefficient of the Lth sub-band of the sub-aperture gradient image array of the distorted light field image, where L = 1, 2, ..., 16.

[0045] Among them, through the formula Calculate the similarity error map of the 4D wavelet coefficients for each sub-band; where T1 is a fixed decimal with a value of 0.00001; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the reference light field image; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the distorted light field image.

[0046] Implementing the embodiments of the present invention has the following beneficial effects:

[0047] 1. In evaluating the quality performance of distorted light field images, this invention involves reference light field image information, making the evaluation of light field image quality more accurate and comprehensive.

[0048] 2. This invention compares the sub-aperture gradient image of the reference light field image and the gradient image of the distorted light field image in the 4D wavelet domain, effectively characterizing the spatial quality, angular consistency, and intrinsic relationship between space and angle of the light field image, thereby providing a more comprehensive description of the differences between the reference light field image and the distorted light field image, and has good light field image quality evaluation performance.

[0049] 3. This invention converts the sub-aperture image array of the light field image into the sub-aperture gradient image array of the light field image, filters out the angular redundancy of the light field image, and uses a space-angle weighting strategy to further filter out unimportant information in the light field image, thereby further improving the performance of light field image quality evaluation. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0051] Figure 1 A flowchart of a light field image quality evaluation method based on 4D wavelet transform provided in an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the structure of a light field image quality evaluation system based on 4D wavelet transform, provided in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0054] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for evaluating the quality of light field images based on 4D wavelet transform. The method includes the following steps:

[0055] Step S1: Obtain the sub-aperture image array of the reference light field image and the sub-aperture image array of the distorted light field image, and convert them into the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively.

[0056] The specific process is as follows: First, the reference light field image is denoted as L. r The distorted light field image is denoted as L. d The sub-aperture image array of the reference light field image is denoted as The sub-aperture image array of the distorted light field image is denoted as Among them, the reference light field image L r And distorted light field image L d Both are 4-dimensional signals, which can be represented as a set of sub-aperture images, i.e., an array of sub-aperture images of the reference light field image. Sub-aperture image arrays with distorted light field images u and v represent the angular planes of the light field image. and The sub-aperture image arrays of the reference light field image and the sub-aperture image arrays of the distorted light field image are respectively represented in the sub-aperture image at the angular coordinates (u,v);

[0057] Secondly, through the formula and formula Sub-aperture image array for calculating the reference light field image A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). Vertical difference image with reference

[0058] Next, through the formula and formula Sub-aperture image array for calculating distorted light field images A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). and vertical difference image

[0059] Then, through the formula and formula The sub-aperture gradient image of the reference light field image is calculated. Sub-aperture gradient image of distorted light field image

[0060] Finally, based on the sub-aperture gradient image of the reference light field image. Sub-aperture gradient image of distorted light field image Sub-aperture gradient image arrays obtained from the reference light field image Sub-aperture gradient image array with distorted light field image

[0061] Step S2: Perform 4D wavelet decomposition on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the distorted light field image.

[0062] The specific process is as follows: First, the one-dimensional wavelet transform of a one-dimensional signal f(t) is defined as follows: Wherein, ψ(t) is defined as a is a scale parameter and τ is the shift parameter and j is an integer;

[0063] Secondly, one-dimensional wavelet transforms are performed sequentially along four dimensions on the sub-aperture gradient image arrays of the reference light field image and the distorted light field image, respectively, to obtain the four-dimensional wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image. The sub-aperture gradient image array corresponding to the distorted light field image has 16 different sub-bands of 4D wavelet coefficients. in, These are the 4-dimensional wavelet subband coefficients of the Lth subband of the sub-aperture gradient image array of the reference light field image. is the 4-dimensional wavelet coefficient of the Lth sub-band of the sub-aperture gradient image array of the distorted light field image, where L = 1, 2, ..., 16.

[0064] Step S3: Combine the 4D wavelet coefficients of each reference sub-aperture gradient image array with the 4D wavelet coefficients of the corresponding distorted sub-aperture gradient image array to calculate the similarity error map of the 4D wavelet coefficients of the 16 sub-bands.

[0065] The specific process is as follows: through the formula Calculate the similarity error map of the 4D wavelet coefficients for each sub-band; where T1 is a fixed decimal with a value of 0.00001; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the reference light field image; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the distorted light field image.

[0066] Step S4: Using a space-angle weighting strategy, aggregate the similarity values ​​of each pixel in each similarity error map to obtain the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands.

[0067] The specific process is as follows: First, through the formula The spatial weights of each similarity error map are calculated. Where max() represents the maximum value operation;

[0068] Secondly, through the formula The angle weights of each similarity error map are calculated.

[0069] Next, through the formula The spatial-angular weight values ​​of each similarity error map are calculated.

[0070] Then, through the formula The fraction of the 4-dimensional wavelet coefficients for each sub-band is calculated.

[0071] Therefore, the fractions {q1,q2,...,q} of the 4D wavelet coefficients for obtaining 16 subbands are calculated. 16}

[0072] Step S5: Construct a one-dimensional feature vector from the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands and import it into the trained support vector regression model to obtain the quality score of the distorted light field image.

[0073] The specific process involves calculating the fractions {q} of the 4D wavelet coefficients of all sub-bands. L The first eigenvectors are F = {q1, q2, ..., q}. 16 The one-dimensional feature vector is then used as input to a pre-trained support vector regression model to obtain the quality score Q of the distorted light field image.

[0074] Q = model(F)

[0075] Here, model() is a pre-trained support vector regression model.

[0076] It should be noted that the construction, training, and testing of support vector regression models are common techniques in this field and will not be elaborated upon here.

[0077] In the embodiments of the present invention, the superiority of the method of the present invention is demonstrated below through specific examples and data;

[0078] Table 1 compares the experimental results of this invention with other advanced algorithms on the Win5-LID light field database. SSIM, MS-SSIM, VIF, FSIM, IWSSIM, MDFM, Min, and Meng are the names of the advanced algorithms being compared. "Proposed" is defined as the method of this invention. PLCC (Pearson linear correlation coefficient), SROCC (Spearman rank correlation coefficient), and RMSE (root mean square error) are three commonly used standards for evaluating performance in the field of image quality assessment. The closer the values ​​of PLCC and SROCC are to 1, the smaller the RMSE value, and the higher the accuracy of the algorithm's prediction.

[0079] As can be seen from the data in Table 1, the method of the present invention has the highest PLCC and SROCC values ​​and the lowest RMSE value, which indicates that the method of the present invention has the best performance.

[0080] Table 1

[0081]

[0082] like Figure 2 As shown in the figure, an embodiment of the present invention provides a light field image quality assessment system based on 4D wavelet transform, comprising:

[0083] Angle filtering unit 110 is used to acquire the sub-aperture image array of the reference light field image and the sub-aperture image array of the distorted light field image, and convert them into the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively.

[0084] Wavelet decomposition unit 120 is used to perform 4D wavelet decomposition on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the distorted light field image.

[0085] The similarity error calculation unit 130 is used to combine the 4D wavelet coefficients of the sub-band of each reference sub-aperture gradient image array with the 4D wavelet coefficients of the corresponding distorted sub-aperture gradient image array to calculate the similarity error map of the 4D wavelet coefficients of 16 sub-bands.

[0086] The spatial angle weighting unit 140 is used to aggregate the similarity values ​​of each pixel in each similarity error map using a spatial-angle weighting strategy to obtain the scores of the 4-dimensional wavelet coefficients of 16 sub-bands.

[0087] The quality performance evaluation unit 150 is used to construct a one-dimensional feature vector from the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands and import it into the pre-trained support vector regression model to obtain the quality score of the distorted light field image.

[0088] The angle filtering unit 110 includes:

[0089] The assignment module is used to denote the reference light field image as L. r The distorted light field image is denoted as L. d The sub-aperture image array of the reference light field image is denoted as The sub-aperture image array of the distorted light field image is denoted as Among them, the reference light field image L r And distorted light field image L d Both are 4-dimensional signals, which can be represented as a set of sub-aperture images, i.e., an array of sub-aperture images of the reference light field image. Sub-aperture image arrays with distorted light field images u and v represent the angular planes of the light field image. and The sub-aperture image arrays of the reference light field image and the sub-aperture image arrays of the distorted light field image are respectively represented in the sub-aperture image at the angular coordinates (u,v);

[0090] The reference sub-aperture image array calculation module is used to calculate the sub-aperture image array using the formula. and formula Sub-aperture image array for calculating the reference light field image A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). Vertical difference image with reference

[0091] The distortion sub-aperture image array calculation module is used to calculate the distortion using the formula. and formula Sub-aperture image array for calculating distorted light field images A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). and vertical difference image

[0092] Two sub-aperture gradient image acquisition modules are used to obtain images using formulas. and formula The sub-aperture gradient image of the reference light field image is calculated. Sub-aperture gradient image of distorted light field image

[0093] Two sub-aperture gradient image array generation modules are used to generate sub-aperture gradient images based on a reference light field image. Sub-aperture gradient image of distorted light field image Sub-aperture gradient image arrays obtained from the reference light field image Sub-aperture gradient image array with distorted light field image

[0094] The wavelet decomposition unit 120 includes:

[0095] The wavelet signal definition module is used to define the one-dimensional wavelet transform representation of a one-dimensional signal f(t). Wherein, ψ(t) is defined as a is a scale parameter and τ is the shift parameter and j is an integer;

[0096] The wavelet transform module performs 1D wavelet transforms along four dimensions on the sub-aperture gradient image arrays of the reference light field image and the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image. The sub-aperture gradient image array corresponding to the distorted light field image has 16 different sub-bands of 4D wavelet coefficients. in, These are the 4-dimensional wavelet subband coefficients of the Lth subband of the sub-aperture gradient image array of the reference light field image. is the 4-dimensional wavelet coefficient of the Lth sub-band of the sub-aperture gradient image array of the distorted light field image, where L = 1, 2, ..., 16.

[0097] Among them, through the formula Calculate the similarity error map of the 4D wavelet coefficients for each sub-band; where T1 is a fixed decimal with a value of 0.00001; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the reference light field image; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the distorted light field image.

[0098] Implementing the embodiments of the present invention has the following beneficial effects:

[0099] 1. In evaluating the quality performance of distorted light field images, this invention involves reference light field image information, making the evaluation of light field image quality more accurate and comprehensive.

[0100] 2. This invention compares the sub-aperture gradient image of the reference light field image and the gradient image of the distorted light field image in the 4D wavelet domain, effectively characterizing the spatial quality, angular consistency, and intrinsic relationship between space and angle of the light field image, thereby providing a more comprehensive description of the differences between the reference light field image and the distorted light field image, and has good light field image quality evaluation performance.

[0101] 3. This invention converts the sub-aperture image array of the light field image into the sub-aperture gradient image array of the light field image, filters out the angular redundancy of the light field image, and uses a space-angle weighting strategy to further filter out unimportant information in the light field image, thereby further improving the performance of light field image quality evaluation.

[0102] It is worth noting that the various units included in the above system embodiments are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0103] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as ROM / RAM, disk, optical disk, etc.

[0104] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for evaluating the quality of light field images based on 4D wavelet transform, characterized in that, The method includes the following steps: Obtain the sub-aperture image array of the reference light field image and the sub-aperture image array of the distorted light field image, and convert them into the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively. Four-dimensional wavelet decomposition is performed on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, to obtain the four-dimensional wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the four-dimensional wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the distorted light field image. The 4D wavelet coefficients of each sub-band of the reference sub-aperture gradient image array are combined with the 4D wavelet coefficients of the corresponding distorted sub-aperture gradient image array to obtain the similarity error map of the 4D wavelet coefficients of the 16 sub-bands. A spatial-angle weighting strategy is adopted to aggregate the similarity values ​​of each pixel in each similarity error map to obtain the scores of the 4-dimensional wavelet coefficients of 16 sub-bands; The scores of the 4-dimensional wavelet coefficients of the 16 sub-bands are used to construct a one-dimensional feature vector, which is then imported into the trained support vector regression model to obtain the quality score of the distorted light field image. Through formula The similarity error map of the 4-dimensional wavelet coefficients of each sub-band is calculated; where T1 is a fixed decimal with a value of 0.00001. It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the reference light field image; It is the amplitude of the 4D wavelet coefficients of a certain sub-band corresponding to the sub-aperture gradient image array of the distorted light field image. These are the 4-dimensional wavelet subband coefficients of the Lth subband of the sub-aperture gradient image array of the reference light field image. These are the 4-dimensional wavelet coefficients of the Lth sub-band of the sub-aperture gradient image array of the distorted light field image, where L = 1, 2, ..., 16; The specific steps for employing a space-angle weighting strategy to aggregate the similarity values ​​of each pixel in each similarity error map to obtain the scores of the 4D wavelet coefficients of the 16 sub-bands include: Through formula The spatial weights of each similarity error map are calculated. ; where max() represents the maximum value operation; Through formula The angular weights of each similarity error map are calculated. ; Through formula The spatial-angular weight value of each similarity error map is calculated. ; Through formula The fraction of the 4-dimensional wavelet coefficients for each sub-band is calculated. The specific steps for constructing a one-dimensional feature vector from the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands and importing it into a pre-trained support vector regression model to obtain the quality score of the distorted light field image include: Fractions of the 4D wavelet coefficients of all subbands Construct a one-dimensional eigenvector F={q1,q2,...,q 16 The one-dimensional feature vector is then used as input to a pre-trained support vector regression model to obtain the quality score Q of the distorted light field image. Here, model() is a pre-trained support vector regression model.

2. The light field image quality evaluation method based on 4D wavelet transform as described in claim 1, characterized in that, The specific steps of acquiring the sub-aperture image array of the reference light field image and the sub-aperture image array of the distorted light field image, and converting them into the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, include: The reference light field image is denoted as The distorted light field image is denoted as The sub-aperture image array of the reference light field image is denoted as The sub-aperture image array of the distorted light field image is denoted as ; where the reference light field image and distorted light field images Both are 4-dimensional signals, which can be represented as a set of sub-aperture images, i.e., an array of sub-aperture images of the reference light field image. Sub-aperture image arrays with distorted light field images u and v represent the angular planes of the light field image. and The sub-aperture image arrays of the reference light field image and the sub-aperture image arrays of the distorted light field image are respectively represented in the sub-aperture image at the angular coordinates (u,v); Through formula and formula Calculate the sub-aperture image array of the reference light field image. A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). Vertical difference image with reference ; Through formula and formula Sub-aperture image array for calculating distorted light field images A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). and vertical difference image ; Through formula and formula The sub-aperture gradient image of the reference light field image is calculated. Sub-aperture gradient image of distorted light field image ; Sub-aperture gradient image based on reference light field image Sub-aperture gradient image of distorted light field image Sub-aperture gradient image arrays of the reference light field image were obtained respectively. Sub-aperture gradient image array with distorted light field image .

3. The light field image quality evaluation method based on 4D wavelet transform as described in claim 1, characterized in that, The specific steps for performing 4D wavelet decomposition on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image include: Define a 1-dimensional signal One-dimensional wavelet transform is represented as ;in, Defined as ;a is a scale parameter and τ is the shift parameter and j is an integer; Perform 1D wavelet transforms along four dimensions on the sub-aperture gradient image arrays of the reference light field image and the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image. The sub-aperture gradient image array corresponding to the distorted light field image has 16 different sub-bands of 4D wavelet coefficients. .

4. A light field image quality assessment system based on 4D wavelet transform, characterized in that, include: An angle filtering unit is used to acquire sub-aperture image arrays of a reference light field image and sub-aperture image arrays of a distorted light field image, and convert them into sub-aperture gradient image arrays of the reference light field image and sub-aperture gradient image arrays of the distorted light field image, respectively. The wavelet decomposition unit is used to perform 4D wavelet decomposition on the sub-aperture gradient image array of the reference light field image and the sub-aperture gradient image array of the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image and the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the distorted light field image. The similarity error calculation unit is used to combine the 4D wavelet coefficients of the sub-band of each reference sub-aperture gradient image array with the 4D wavelet coefficients of the corresponding distorted sub-aperture gradient image array to calculate the similarity error map of the 4D wavelet coefficients of the 16 sub-bands. The spatial angle weighting unit is used to aggregate the similarity values ​​of each pixel in each similarity error map using a spatial-angle weighting strategy to obtain the scores of the 4D wavelet coefficients of 16 sub-bands. The quality performance evaluation unit is used to construct a one-dimensional feature vector from the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands and import it into the pre-trained support vector regression model to obtain the quality score of the distorted light field image. The specific steps for employing a space-angle weighting strategy to aggregate the similarity values ​​of each pixel in each similarity error map to obtain the scores of the 4D wavelet coefficients of the 16 sub-bands include: Through formula The spatial weights of each similarity error map are calculated. ; where max() represents the maximum value operation; Through formula The angular weights of each similarity error map are calculated. ; Through formula The spatial-angular weight value of each similarity error map is calculated. ; Through formula The fraction of the 4-dimensional wavelet coefficients for each sub-band is calculated. The specific steps for constructing a one-dimensional feature vector from the scores of the 4-dimensional wavelet coefficients of the 16 sub-bands and importing it into a pre-trained support vector regression model to obtain the quality score of the distorted light field image include: Fractions of the 4D wavelet coefficients of all subbands Construct a one-dimensional eigenvector F={q1,q2,...,q 16 The one-dimensional feature vector is then used as input to a pre-trained support vector regression model to obtain the quality score Q of the distorted light field image. Here, model() is a pre-trained support vector regression model.

5. The light field image quality assessment system based on 4D wavelet transform as described in claim 4, characterized in that, The angle filtering unit includes: The assignment module is used to record the reference light field image as... The distorted light field image is denoted as The sub-aperture image array of the reference light field image is denoted as The sub-aperture image array of the distorted light field image is denoted as ; where the reference light field image and distorted light field images Both are 4-dimensional signals, which can be represented as a set of sub-aperture images, i.e., an array of sub-aperture images of the reference light field image. Sub-aperture image arrays with distorted light field images u and v represent the angular planes of the light field image. and The sub-aperture image arrays of the reference light field image and the sub-aperture image arrays of the distorted light field image are respectively represented in the sub-aperture image at the angular coordinates (u,v); The reference sub-aperture image array calculation module is used to calculate the sub-aperture image array using the formula. and formula Calculate the sub-aperture image array of the reference light field image. A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). Vertical difference image with reference ; The distortion sub-aperture image array calculation module is used to calculate the distortion using the formula. and formula Sub-aperture image array for calculating distorted light field images A reference horizontal difference image of the sub-aperture image at angular coordinates (u, v). and vertical difference image ; Two sub-aperture gradient image acquisition modules are used to obtain images using formulas. and formula The sub-aperture gradient image of the reference light field image is calculated. Sub-aperture gradient image of distorted light field image ; Two sub-aperture gradient image array generation modules are used to generate sub-aperture gradient images based on a reference light field image. Sub-aperture gradient image of distorted light field image Sub-aperture gradient image arrays of the reference light field image were obtained respectively. Sub-aperture gradient image array with distorted light field image .

6. The light field image quality assessment system based on 4D wavelet transform as described in claim 4, characterized in that, The wavelet decomposition unit includes: The wavelet signal definition module is used to define 1D signals. One-dimensional wavelet transform is represented as ;in, Defined as ;a is a scale parameter and τ is the shift parameter and j is an integer; The wavelet transform module performs 1D wavelet transforms along four dimensions on the sub-aperture gradient image arrays of the reference light field image and the distorted light field image, respectively, to obtain the 4D wavelet coefficients of 16 different sub-bands corresponding to the sub-aperture gradient image array of the reference light field image. The sub-aperture gradient image array corresponding to the distorted light field image has 16 different sub-bands of 4D wavelet coefficients. .

Images