Full-reference Quality Evaluation Method, System and Storage Medium for Virtual Viewpoint

Abstract

The present invention proposes a full-reference quality evaluation method, system and storage medium for virtual viewpoints, which relates to the technical field of image quality evaluation. First, global motion compensation is performed on the reference image, and the reference image and the virtual viewpoints are subjected to dual-scale discrete wavelet transform to obtain wavelet subbands. The gradient magnitude similarity feature (GMS) maps of each pair of subbands are calculated. Then, morphological opening operation and median filtering are performed on the GMS maps obtained from each pair of subbands. Next, standard deviation pooling is performed on the processed GMS maps of each pair of subbands to obtain the scores of each subband. Finally, linear pooling is performed on the scores of each pair of subbands to obtain the final virtual viewpoint perception quality score, which has a good extraction effect on different distortions caused by different types of DIBR algorithms, making the distortion extraction results more in line with the human visual perception characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of image quality evaluation, and more specifically, to a full-reference quality evaluation method, system, and storage medium for virtual viewpoints. Background Art

[0002] In recent years, three-dimensional related technologies such as virtual reality, multi-viewpoint, and free-viewpoint video have attracted close attention due to their wide applications in many fields such as distance education, medical treatment, and entertainment. However, the realization of these videos requires a large number of viewpoint images. Due to the limitations of network bandwidth and cost, it is impossible to obtain all viewpoints one by one through cameras. Therefore, it is necessary to rely on virtual viewpoint synthesis technology to synthesize new viewpoint images from multiple known viewpoints. Virtual viewpoint synthesis technology is the key to realizing these applications. In existing virtual viewpoint synthesis technologies, the most commonly used is the depth-image-based rendering method (DIBR). The existing DIBR synthesis technology will cause holes or stretching distortions at the foreground or background edges, which will greatly affect the perceptual quality of virtual viewpoints, and the visual quality directly determines whether this technology can be successfully applied. Therefore, it is of great significance to study the quality evaluation for virtual viewpoint synthesis.

[0003] The viewpoints generated by DIBR rendering mainly contain black hole distortions, relatively regular stretching distortions, and irregular stretching distortions. A good full-reference visual quality evaluation method for virtual viewpoint synthesis needs to have a good extraction effect on different distortions caused by different types of DIBR algorithms in order to predict a more human-eye-compliant perceptual quality. In the prior art, a no-reference DIBR-generated image quality evaluation method is disclosed. This method calculates the local binary pattern map LBP of the generated image, binarizes the LBP map, detects and measures the quality of the local hole area and the local stretching area respectively, and comprehensively evaluates the overall quality with the global blur quality metric. However, this scheme has a good extraction effect on black hole distortions and relatively regular stretching distortions, but a poor extraction effect on irregular stretching distortions, resulting in inaccurate perceptual quality predictions for virtual viewpoints containing such distortions. Summary of the Invention

[0004] To solve the problem that the current DIBR image quality evaluation method has a poor extraction effect on distortions, resulting in inaccurate perceptual quality evaluation, the present invention proposes a full-reference quality evaluation method, system, and storage medium for virtual viewpoints, which have a good extraction effect on different distortions caused by different types of DIBR algorithms, making the distortion extraction results more in line with human-eye perception characteristics.

[0005] To achieve the above technical effects, the technical solution of the present invention is as follows:

[0006] A full-reference quality assessment method for virtual perspectives, including:

[0007] S1. Extract the labeled and distinguished reference images and virtual perspectives from the known dataset, obtain the feature matching points of the reference images and virtual perspectives, and perform global object movement compensation on the reference images based on the feature matching points;

[0008] S2. Perform dual-scale discrete wavelet transform on the reference images and virtual perspectives respectively to obtain the wavelet subbands of the reference images and virtual perspectives, pair the subbands of the reference images and virtual perspectives, and generate a Gradient Magnitude Similarity (GMS) feature map based on each pair of subbands;

[0009] S3. Perform morphological opening operation and median filtering on the Gradient Magnitude Similarity (GMS) feature map in sequence;

[0010] S4. Perform standard deviation pooling on the processed GMS map to obtain the scores of each pair of wavelet subbands;

[0011] S5. Perform linear pooling on the scores of each pair of subbands to obtain the final virtual perspective perception quality score.

[0012] In this technical solution, global object movement compensation is performed on the reference image based on the feature points of the reference image and virtual perspective. The wavelet subbands of the reference image and virtual perspective are obtained through dual-scale discrete wavelet transform, and a Gradient Magnitude Similarity (GMS) feature map is generated. By extracting the information of the wavelet subbands through GMS, various different types of distortion situations can be evaluated more accurately.

[0013] Preferably, in step S1, the Speeded Up Robust Features (SURF) algorithm is used to obtain the matching feature points of the virtual perspective I r and the reference image I d ; the matching feature points are input into the RANSAC algorithm model to screen out the correct matching feature points; an affine transformation model is constructed according to the correct matching feature points, and the reference image I d is input into the affine transformation model to obtain the reference image after global object movement compensation

[0014] Preferably, the process of constructing the affine transformation model is as follows:

[0015] Assume that the coordinates of the reference image in a pair of feature matching points are (x0, y0), and the virtual perspective coordinates are (x′0, y′0). The affine transformation formula is:

[0016]

[0017] where (t x , t y ) represents the translation amount, and the parameter ai used to reflect changes such as image rotation and scaling; solve for parameter a i Substitute it into the affine transformation formula later to obtain the constructed affine transformation model.

[0018] Here, through the global object movement compensation for the virtual view, it is avoided that the global object movement in the virtual view is judged as distortion and over - considered.

[0019] Preferably, in step S2, the reference image after global object movement compensation and the virtual view I d are respectively subjected to dual - scale discrete wavelet transform to obtain the reference image and the virtual view I d low - frequency sub - bands and diagonal sub - bands at different scales, and calculate the gradient magnitude similarity feature GMS map for each pair of sub - bands, including the following steps:

[0020] S21. Perform dual - scale wavelet transform on the reference image and the virtual view I d respectively to obtain wavelet sub - bands R S and D S , and the calculation formula is as follows:

[0021]

[0022] D S = DWT(I d )

[0023] where R S represents the wavelet sub - band of the reference image , D S represents the wavelet sub - band of the virtual view I d , DWT() represents the multi - scale wavelet transform operation, and the subscript s represents the scale and category of the wavelet sub - band, s ∈ {LL1, HH1, LL2, HH2}, LL1 represents the low - frequency sub - band from scale 1, HH1 represents the diagonal sub - band from scale 1, LL2 represents the low - frequency sub - band from scale 2, and HH2 represents the diagonal sub - band from scale 2;

[0024] S22. Respectively obtain the gradient magnitude S of R S and D at the image pixel position (i, j), and the calculation formula is as follows:

[0025]

[0026]

[0027] where i and j are the coordinates of the pixel points in the image, represents the convolution operation, H x and H y Represents the horizontal and vertical filters using Prewitt, respectively;

[0028] S23. Reference image and Virtual Perspective I d Subband R S and D S Pair them at the same scale and the same category to generate the gradient amplitude similarity feature GMS map G corresponding to each pair of sub-bands s , the calculation formula is as follows:

[0029]

[0030] Where e is a non-zero constant.

[0031] Preferably, in step S3, each G s Flip and perform an open operation to obtain the processed GMS graph The opening operation formula is as follows:

[0032]

[0033] in, Represents the flipped G s ,!and They represent corrosion and dilation operations respectively, and SE represents the structural element.

[0034] Preferably, in step S3, Perform median filtering to obtain Where φ represents the median filter operation, represent A square window of size n×n in .

[0035] Here, the extracted distortion is further screened using morphological opening operation and median filtering to remove redundant distortion, making the distortion extraction result more consistent with the perception characteristics of the human eye.

[0036] Preferably, in step S4, the processed GMS image is subjected to standard error pooling to obtain the scores Q of each pair of sub-bands respectively. s , the calculation formula is as follows:

[0037]

[0038]

[0039] Among them, s∈{LL1,HH1,LL2,HH2}, W and H represent The width and height of , GM represents the average grayscale of each GMS image.

[0040] Preferably, in step S5, weights α and β are set, and linear pooling is performed on the scores of each pair of subbands to obtain the final virtual view perception quality score S. The calculation formula is as follows:

[0041]

[0042] Among them, α represents the relative contribution for adjusting the low-frequency and high-frequency subbands, and β represents the relative contribution for adjusting the scales 1 and 2.

[0043] Preferably, the present application also proposes a computer storage medium for computer-readable storage. A program for full-reference quality evaluation for virtual views is stored on the computer storage medium. When the program for full-reference quality evaluation for virtual views is executed by a processor, it is used to implement the steps of the useful full-reference quality evaluation for virtual views.

[0044] The present invention also proposes a system for full-reference quality evaluation for virtual views. The system includes:

[0045] A motion compensation module for obtaining feature matching points of a reference image and a virtual view, and performing global object motion compensation on the reference image based on the feature matching points;

[0046] A GMS map generation module for performing two-scale discrete wavelet transform on the reference image and the virtual view, and calculating the gradient magnitude similarity feature (GMS) map of each pair of subbands;

[0047] A distortion region screening module for performing morphological opening operation and median filtering on the gradient magnitude similarity feature (GMS) map obtained from each pair of subbands to remove perceptually unimportant regions;

[0048] A standard deviation pooling module for performing standard deviation pooling on the processed GMS map of each pair of subbands to obtain the score of each subband;

[0049] A linear pooling module for performing linear pooling on the scores of each pair of subbands to obtain the final virtual view perception quality score.

[0050] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0051] The present invention proposes a full-reference quality assessment, system, and storage medium for virtual viewpoints. First, global motion compensation is performed on the reference image, and the reference image and the virtual viewpoint are subjected to dual-scale discrete wavelet transform to obtain wavelet subbands. The gradient magnitude similarity (GMS) feature maps of each pair of subbands are calculated. Then, morphological opening operations and median filtering are performed on the GMS feature maps obtained from each pair of subbands. Next, standard deviation pooling is performed on the processed GMS feature maps of each pair of subbands to obtain the scores of each subband. Finally, linear pooling is performed on the scores of each pair of subbands to obtain the final virtual viewpoint perception quality score, which has a good extraction effect on different distortions caused by different types of DIBR algorithms, making the distortion extraction results more in line with human visual perception characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It represents a schematic flowchart of the full-reference quality assessment method for virtual viewpoints proposed in Embodiment 1 of the present invention;

[0053] Figure 2 It represents a process diagram of the full-reference quality assessment method for virtual viewpoints proposed in Embodiment 1 of the present invention;

[0054] Figure 3 It represents a scatter plot for comparing the virtual viewpoint quality assessment proposed in Embodiment 1 of the present invention;

[0055] Figure 4 It represents a schematic diagram of the computer device proposed in Embodiment 2 of the present invention;

[0056] Figure 5 It represents a schematic structural diagram of the full-reference quality assessment system for virtual viewpoints proposed in Embodiment 3 of the present invention; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] The drawings are only for illustrative purposes and should not be construed as a limitation of this patent;

[0058] For better illustration of this embodiment, some parts of the drawings are omitted, enlarged, or reduced, and do not represent the actual size;

[0059] For those skilled in the art, it is understandable that some well-known content descriptions in the drawings may be omitted.

[0060] The technical solutions of the present invention will be further described below with reference to the drawings and embodiments.

[0061] The description of the positional relationship in the drawings is only for illustrative purposes and should not be construed as a limitation of this patent;

[0062] Embodiment 1

[0063] As Figure 1As shown in the figure, this embodiment proposes a full-reference quality assessment method for virtual perspectives. The method includes the following steps. See Figure 2 :

[0064] S1. Extract the labeled and differentiated reference images and virtual perspectives from the known dataset, obtain the feature matching points of the reference images and virtual perspectives, and perform global object movement compensation on the reference images based on the feature matching points;

[0065] In this embodiment, the labeled and differentiated reference image I d and virtual perspective I r are extracted from the IETR dataset. This dataset contains 10 reference images and 140 virtual perspectives synthesized by 8 new DIBR rendering algorithms. These synthesis algorithms include both single-viewpoint synthesis algorithms and dual-view synthesis algorithms.

[0066] Use the Speeded Up Robust Features (SURF) algorithm to obtain the matching feature points of virtual perspective I r and reference image I d ; input the matching feature points into the RANSAC algorithm model to screen out the correct matching feature points; construct an affine transformation model according to the correct matching feature points. Let the coordinates of the reference image in a pair of feature matching points be (x0, y0), and the coordinates of the virtual perspective be (x′0, y′0). The affine transformation formula is:

[0067]

[0068] where (t x , t y ) represents the translation amount, and the parameter a i is used to reflect changes such as image rotation and scaling;

[0069] Solve for the parameter a i and substitute it into the affine transformation formula to obtain the constructed affine transformation model. Input the reference image I d into the affine transformation model to obtain the reference image after global object movement compensation

[0070] S2. Perform dual-scale discrete wavelet transform on the reference image and the virtual perspective respectively to obtain the wavelet subbands of the reference image and the virtual perspective. Pair the subbands of the reference image and the virtual perspective, and generate a Gradient Magnitude Similarity (GMS) map based on each pair of subbands;

[0071] S21. Perform dual-scale wavelet transform on the reference image and virtual perspective I d respectively to obtain the wavelet subbands R S , D S . The calculation formula is as follows:

[0072]

[0073] D S = DWT(I d )

[0074] where R S represents the wavelet sub - band of the reference image , D S represents the wavelet sub - band of the virtual view I d , DWT() represents the multi - scale wavelet transform operation, the subscript s represents the scale and category of the wavelet sub - band, s ∈ {LL1, HH1, LL2, HH2}, LL1 represents the low - frequency sub - band from scale 1, HH1 represents the diagonal sub - band from scale 1, LL2 represents the low - frequency sub - band from scale 2, and HH2 represents the diagonal sub - band from scale 2;

[0075] S22. Obtain the gradient magnitudes of R S and D S at the image pixel position (i, j) respectively. The calculation formula is as follows: The calculation formula is as follows:

[0076]

[0077]

[0078] where i and j are the coordinates of the pixel points in the image, represents the convolution operation, H x and H y represent the horizontal and vertical filters using Prewitt respectively;

[0079] S23. Pair the sub - bands R and D d of the reference image S and the virtual view I S by the same scale and the same category to generate the gradient magnitude similarity feature GMS map G s , and the calculation formula is as follows:

[0080]

[0081] where e is a non - zero constant.

[0082] S3. Flip each G s and perform the opening operation to obtain the processed GMS map The opening operation formula is as follows:

[0083]

[0084] where Represents the flipped G s ,! and represent erosion and dilation operations respectively, SE represents the structuring element, and the structuring element in this embodiment uses a square window. For scale 1 and scale 2, 2-pixel size and 1-pixel size are selected respectively;

[0085] Perform median filtering operation to obtain where φ represents the median filtering operation, represents a square window of size n×n in

[0086] S4. Perform standard deviation pooling on the processed GMS map to obtain the scores Q of each pair of subbands s , and the calculation formula is as follows:

[0087]

[0088]

[0089] where s∈{LL1, HH1, LL2, HH2}, W and H represent the width and height of

[0090] S5. Set the weights α and β, perform linear pooling on the scores of each pair of subbands to obtain the final virtual view perception quality score S, and the calculation formula is as follows:

[0091]

[0092] where α represents the relative contribution for adjusting the low-frequency and high-frequency subbands, and β represents the relative contribution for adjusting scale 1 and scale 2.

[0093] In addition, in this embodiment, the scores predicted by the full-reference quality evaluation method for virtual views are compared for accuracy using the subjective scoring DMOS algorithm corresponding to the synthetic view. Refer to Figure 3 , and the comparison methods involved are: NIQSV+, SSPD, LOGS, APT, SC-IQA, MNSS, MP-PSNR-reduce. Among them, the larger the PLCC and SRCC, the closer and more relevant the objective score and the subjective score are, and the smaller the RMSE, the closer and more relevant the objective score predicted by the model and the subjective score are. Refer to Table 1.

[0094] Table 1

[0095] Method PLCC SRCC RMSE NIQSV+ 0.2364 0.2902 0.2409 SSPD 0.6837 0.6631 0.1809 LOGS 0.6678 0.6683 0.1845 APT 0.4333 0.4164 0.2234 SC-IQA 0.6766 0.6360 0.1826 MNSS 0.3387 0.2284 0.2333 MP-PSNR-reduce 0.6160 0.5870 0.1953 Ours 0.7910 0.7815 0.1518

[0096] Example 2

[0097] See Figure 4 , this application also proposes a computer device, including a processor, a memory, and a computer program stored on the memory. The processor is labeled as 1, the memory is labeled as 2, and the processor 1 is connected to the memory 2. The processor 1 executes the computer program stored on the memory 2 to implement the full-reference quality evaluation method for virtual viewpoints described in the first embodiment.

[0098] Among them, the memory 2 can be a magnetic disk, a flash memory, or any other non-volatile storage medium. See Figure 3 , the connection between the processor 1 and the memory 2 can be implemented as one or more integrated circuits, specifically a microprocessor or a microcontroller. When executing the computer program stored on the memory, for the global model, it implements the full-reference quality evaluation method for virtual viewpoints.

[0099] This application also proposes a computer-readable storage medium, on which computer program instructions are stored. When these instructions are executed by a processor, the steps of the described method are implemented.

[0100] By executing the above computer program instructions, this computer-readable storage medium saves the collected data in the user terminal itself, preventing the problem of data leakage.

[0101] Example 3

[0102] In this embodiment, as Figure 5 shown, a full-reference quality evaluation system for virtual viewpoints is proposed. The system includes:

[0103] A motion compensation module 101, configured to obtain feature matching points of a reference image and a virtual viewpoint, and perform global object motion compensation on the reference image based on the feature matching points;

[0104] A GMS map generation module 102, configured to perform dual-scale discrete wavelet transform on the reference image and the virtual viewpoint, and calculate the gradient magnitude similarity feature GMS map of each pair of subbands;

[0105] A distortion region screening module 103, configured to perform morphological opening operation and median filtering on the gradient magnitude similarity feature GMS map obtained for each pair of subbands to remove perceptually unimportant regions;

[0106] A standard deviation pooling module 104, which performs standard deviation pooling on the processed GMS map of each pair of subbands to obtain the score of each subband;

[0107] A linear pooling module 105, which performs linear pooling on the scores of each pair of subbands to obtain the final virtual viewpoint perception quality score.

[0108] Overall, the motion compensation module 101 is used to obtain the feature matching points of the reference image and the virtual view, perform global object motion compensation on the reference image based on the feature matching points, and then the GMS generation module 102 performs dual-scale discrete wavelet transform on the reference image and the virtual view respectively to obtain the wavelet subbands of the reference image and the virtual view. The gradient magnitude similarity feature GMS map is generated based on each pair of wavelet subbands of the reference image and the virtual view. Then, the distortion region screening module 103 performs morphological opening operation and median filtering on the gradient magnitude similarity feature GMS map in sequence to remove the perceptually unimportant regions. The standard deviation pooling module 104 performs standard deviation pooling on the processed GMS map of each pair of subbands to obtain the scores of each subband. Finally, the linear pooling module 105 performs linear pooling on the scores of each pair of subbands to obtain the final virtual view perception quality score.

[0109] Obviously, the above embodiments of the present invention are only examples for clearly illustrating the present invention, rather than limiting the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or variations can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A full-reference quality assessment method for virtual viewpoints, characterized in that Including: S1. Extract the annotated and distinguished reference images and virtual viewpoints from the known dataset, obtain the feature matching points of the reference images and virtual viewpoints, and perform global object movement compensation on the reference images based on the feature matching points; S2. Perform dual-scale discrete wavelet transform on the reference images and virtual viewpoints respectively to obtain the wavelet subbands of the reference images and virtual viewpoints, pair the subbands of the reference images and virtual viewpoints, and generate the Gradient Magnitude Similarity (GMS) map based on each pair of subbands; S3. Perform morphological opening operation and median filtering on the Gradient Magnitude Similarity (GMS) map in sequence; S4. Perform standard deviation pooling on the processed GMS map to obtain the scores of each pair of wavelet subbands; S5. Perform linear pooling on the scores of each pair of subbands to obtain the final virtual viewpoint perception quality score.

2. The full-reference quality assessment method for virtual view according to claim 1, wherein In step S1, the Scale-invariant Feature Transform (SURF) algorithm is used to obtain the matching feature points between the virtual view I d and the reference image I r ; the matching feature points are input into the RANSAC algorithm model to filter out the correct matching feature points; an affine transformation model is constructed based on the correct matching feature points, and the reference image I r is input into the affine transformation model to obtain the reference image after global object movement compensation 3. The full-reference quality assessment method for virtual perspective according to claim 2, characterized in that The process of constructing the affine transformation model is as follows: Assume that the coordinates of the reference image in a pair of feature matching points are (x0, y0), and the coordinates of the virtual viewpoint are (x'0, y'0). The affine transformation formula is: Among them, (t x , t y ) represents the translation amount, and the parameter a i is used to reflect the rotation and scaling changes of the image; after solving the parameter a i and substituting it into the affine transformation formula, the constructed affine transformation model is obtained.

4. The full-reference quality evaluation method for virtual view according to claim 2, characterized in that In step S2, the reference image after global object motion compensation and the virtual view I d are respectively subjected to dual-scale discrete wavelet transform to obtain the reference image and the virtual view I d low-frequency subbands and diagonal subbands at different scales, and calculate the gradient magnitude similarity feature GMS map of each pair of subbands, including the following steps: S21. Perform a two-scale wavelet transform on the reference image and the virtual view I d respectively to obtain wavelet subbands R S and D S . The calculation formula is as follows: D S = DWT(I d ) wherein, R S represents the wavelet subband of the reference image , D S represents the wavelet subband of the virtual view I d , DWT() represents the multi-scale wavelet transform operation, the subscript s represents the scale and category of the wavelet subband, s ∈ {LL1, HH1, LL2, HH2}, LL1 represents the low-frequency subband from scale 1, HH1 represents the diagonal subband from scale 1, LL2 represents the low-frequency subband from scale 2, and HH2 represents the diagonal subband from scale 2; S22. Obtain R S and D S at the gradient magnitude of the image pixel position (i, j) The calculation formula is as follows: where i and j are the coordinates of pixels in the image, represents the convolution operation, H x and H y represent the horizontal and vertical Prewitt filters respectively; S23. Take the reference image and the sub-bands R d and D S of the virtual view I S Pair them according to the same scale and the same category to generate the gradient magnitude similarity feature GMS map G s corresponding to each pair of sub-bands. The calculation formula is as follows: where e is a non-zero constant.

5. The full-reference quality evaluation method for virtual view according to claim 4, wherein In step S3, each G s is flipped and morphological opening is performed to obtain the processed GMS diagram The formula for morphological opening is as follows: Among them, represents the flipped G s , and represent erosion and dilation operations respectively, and SE represents the structuring element.

6. The full-reference quality evaluation method for virtual view according to claim 5, characterized in that In step S3, for perform a median filtering operation to obtain where φ represents the median filtering operation, represents a square window of size n×n in 7. The full-reference quality assessment method for virtual view according to claim 6, characterized in that In step S4, standard deviation pooling is performed on the processed GMS graph to obtain the scores Q of each pair of subbands respectively s , and the calculation formula is as follows: where s ∈ {LL1, HH1, LL2, HH2}, W and H represent the width and height of respectively, and GM represents the average gray level of each GMS diagram.

8. The full-reference quality assessment method for virtual view according to claim 7, characterized in that In step S5, set the weights α and β, and perform linear pooling on the scores of each pair of subbands to obtain the final virtual viewpoint perception quality score S. The calculation formula is as follows: where α represents the relative contribution for adjusting the low-frequency and high-frequency subbands, and β represents the relative contribution for adjusting the scales 1 and 2.

9. A computer storage medium for computer-readable storage, characterized in that, The program of the full-reference quality evaluation method for virtual viewpoints is stored on the computer storage medium. When the program of the full-reference quality evaluation method for virtual viewpoints is executed by the processor, it is used to implement the steps of the full-reference quality evaluation method for virtual viewpoints described in any one of claims 1 to 8.

10. A computer system for a full-reference quality assessment method for virtual perspectives, characterized in that, Including: A movement compensation module, which is used to obtain the feature matching points of the reference images and virtual viewpoints, and perform global object movement compensation on the reference images based on the feature matching points; A GMS map generation module, which is used to perform dual-scale discrete wavelet transform on the reference images and virtual viewpoints, and calculate the Gradient Magnitude Similarity (GMS) map of each pair of subbands; A distortion region screening module, which is used to perform morphological opening operation and median filtering on the Gradient Magnitude Similarity (GMS) map obtained from each pair of subbands to remove the perceptually unimportant regions; A standard deviation pooling module, which is used to perform standard deviation pooling on the processed GMS map of each pair of subbands to obtain the scores of each subband; A linear pooling module, which is used to perform linear pooling on the scores of each pair of subbands to obtain the final virtual viewpoint perception quality score.

Images