Light field image quality evaluation method based on rich features
By extracting the color, texture, and angle features of light field images and using a genetic algorithm support vector regression model for light field image quality evaluation, the problem of existing technologies failing to effectively integrate the human visual system is solved, achieving a more accurate and efficient light field image quality evaluation.
Patent Information
- Application Number
- CN202211597866.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-12
- Publication Date
- 2025-10-21
- Estimated Expiration
- 2042-12-12
AI Technical Summary
Existing light field image quality assessment models fail to effectively combine multi-layered information perception of the human visual system and fail to fully consider the color and brightness information of the image, thus failing to accurately evaluate the quality of light field images.
By extracting color, texture, and angle features from light field images, pseudo-videos and epipolar images are generated. A genetic algorithm support vector regression model is then used to evaluate the quality of the light field images, simulating the perception process of the human visual system.
It improves the accuracy and scientific rigor of light field image quality evaluation, reduces information redundancy, increases computational efficiency, and is more in line with human visual perception.
Smart Images
Figure CN115937154B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a light field image quality evaluation method based on rich features, specifically a method for evaluating the quality of light field images by simulating the human visual system by combining color information, texture information, and angle information features. Background Art
[0002] Visual information accounts for approximately 75% of human sensory interaction with multimedia. Compared to text and speech, images are more intuitive, vivid, and easy to understand. With the growing demand for visual services and more perceptual needs, such as immersive scene experiences, traditional two-dimensional imaging, which only records the intensity of light after a two-dimensional integration, is no longer sufficient. Light field images, on the other hand, can capture rich information about real scenes by recording the intensity and direction of the light field in free space, with a large depth of field and wide viewing angle. Most light field image processing tasks, such as acquisition, encoding, compression, storage, transmission, and rendering, inevitably introduce various distortion effects, which negatively impact the visual experience. Therefore, we can develop a light field image quality evaluation system by simulating the human visual system (HVS) to understand the spatial, color, and structural perception of the environment.
[0003] Because light field imaging (LFI) has a relatively recent history, existing models for evaluating light field image quality still need improvement. First, existing light field image quality assessment models mostly start from high-dimensional features, failing to consider the human visual system's perception of multi-layered information. Second, most light field image quality assessment models convert the image into grayscale before evaluating its quality, failing to consider image chromaticity and luminance information. Finally, light field images contain more spatial, angular, and texture information than traditional two-dimensional images, requiring the integration of more features for quality assessment. To address these issues, this paper proposes a feature-rich light field image quality assessment model. This model extracts luminance and chromaticity information from a converted color space, generates a pseudo video to obtain structural information from the sub-aperture image (SAI), and also provides motion perception and parallax information. Finally, it extracts the changing characteristics of texture and angular consistency from the epipolar plane image (EPI). By integrating all these rich features to simulate the human visual system, it can more accurately and scientifically evaluate light field image quality. Summary of the Invention
[0004] In response to the above background and problems, the present invention proposes a light field image quality evaluation method based on rich features, which simulates the human visual system and evaluates the quality of light field images based on color features, texture features and angle features.
[0005] The technical solution adopted by the present invention to solve the problem includes the following steps:
[0006] Step 1: Randomly split the Win5-LID dataset into an 80% training set and a 20% test set. For the light field images, fix the two dimensions representing the angle information and convert them into a 9×9 sub-aperture image array.
[0007] Step 2: Convert each sub-aperture image from RGB space to HSV coordinates and extract brightness, hue and saturation as color information features.
[0008] Step 3: Divide the 9×9 sub-aperture image array into 9 groups by row, perform tensor decomposition on each group, and obtain the first most important slice of each pseudo video. Split the pseudo video sequence into two parts, perform shearlet transform, and extract the disparity structure features.
[0009] Step 4: Obtain the polar plane image by fixing the light field image coordinates, use the Sobel operator to process the horizontal polar plane image and the vertical polar plane image, and then calculate their horizontal gradient and vertical gradient respectively to obtain the image angular texture features.
[0010] Step 5: All features (color information features, parallax structure features, image angle texture features) are standardized, then spliced and fused, and the light field image prediction score is obtained using the support vector model based on genetic algorithm (GA-SVR).
[0011] The beneficial effects of the present invention are as follows:
[0012] The present invention uses high-dimensional singular value decomposition to greatly reduce the information redundancy between multiple sub-aperture images and improve computational efficiency. At the same time, it combines a variety of features such as color features, parallax structure features, and angular texture features to better simulate the human visual system and make a more scientific evaluation of light field images. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 Flowchart of the implementation of the present invention
[0014] Figure 2 This is a complete segmentation flowchart DETAILED DESCRIPTION
[0015] The present invention will be further described below with reference to the accompanying drawings and examples.
[0016] like Figure 1 As shown in FIG, the light field image quality evaluation method based on rich features specifically includes the following steps:
[0017] Step 1. Randomly split the Win5-LID dataset into an 80% training set and a 20% test set. A light field image representing a scene has dimensions (u, v, s, t), where (u, v) is the angular coordinate. By fixing (u, v), it can be divided into a 9×9 sub-aperture image array.
[0018] Step 2. Convert each sub-aperture image from RGB space to HSV coordinates and extract brightness, hue and saturation as color information features.
[0019] The brightness, chroma and saturation matrices of each sub-aperture image are represented as T x , where x represents brightness, chroma, and saturation. The chroma, brightness, and saturation matrices of the final light field image can be calculated using the following formula:
[0020]
[0021] Where M and N (M=N=9) represent the number of sub-aperture images in each horizontal and vertical direction of a light field image. Finally, the required color information features [R1, R2, R3] are obtained.
[0022] Step 3. Divide the 9×9 sub-aperture image array into 9 groups by row, perform tensor decomposition on each group, and obtain the first most important slice of each pseudo video. Split the pseudo video sequence into two parts, perform shearlet transform, and extract the disparity structure features.
[0023] 3.1 Pseudo video grouping is performed on the 9×9 sub-aperture images separated from the light field image. Each row is recorded as a group, denoted as p1, p2, p3..., p M , where M = 9 and the dimensions of the pseudo video group are (M, S, T). A high-order singular value decomposition is performed on each pseudo video group. The decomposition formula is as follows:
[0024] p i =X×1F (1) ×2F (2) ×3F (3)
[0025] Where x is the core tensor N is the length of the third dimension, F (i) is a three-factor matrix. × i is the modular product of tensor i, F (i) is an orthogonal matrix. Here K1 = S, K2 = T, K3 = N.
[0026] The decomposition components of the third dimension can be calculated as follows:
[0027] ξ=p i ×3(F(3) ) -1
[0028] Among them, (F (3) ) -1 F (3) The inverse matrix of C1 represents the first slice of ξ, which contains 80% of the energy of slice ξ, and the energy gradually decreases as the slice number increases.
[0029] 3.2 The first slice extracted from each set of pseudo video sequences is denoted as C m1 , where m = 1, 2..., M, perform shearlet transform, and the shearlet calculation formula is as follows:
[0030]
[0031] in, D a is the shear matrix, A a is the anisotropic expansion matrix, a represents the scale, s represents the direction, and t represents the location parameter.
[0032]
[0033] Each scale contains 6 shear coefficients in different directions, which are spliced into the parallax structure feature S.
[0034] Step 4. Obtain the polar plane image by fixing the light field image coordinates, use the Sobel operator to process the horizontal polar plane image and the vertical polar plane image, and then calculate their horizontal gradient and vertical gradient respectively to obtain the image angular texture features.
[0035] By fixing the u and s coordinates of the light field image, the vertical polar plane image is obtained as follows: Fix v, t coordinates to get the horizontal polar plane image: Then the gradient of the vertical polar plane image is obtained by filtering with the Sobel operator The calculation process is as follows:
[0036]
[0037]
[0038] in,
[0039]
[0040] Finally, we get the gradient feature Similarly, the horizontal polar plane image is processed as above. Finally, the features of the horizontal polar plane image and the vertical polar plane image are processed by mean and entropy to obtain the image angle texture feature [G mean , G entropy ].
[0041] Step 5: Normalize all features, concatenate and fuse them, and use the support vector regression method based on genetic algorithm to train the model to obtain the light field image prediction score. The specific implementation is as follows:
[0042] 5-1. After normalizing the obtained color information feature R, parallax structure feature S, and image angle texture feature G, they are spliced together to obtain the sample training set X = {(x1, y1), (x2, y2), ..., (x n ,y n )}, where x i represents the i-th feature, y i represents the i-th label, and n is the number of training samples.
[0043] 5-2. Set the initial population size to 20, the maximum genetic generation to 100, and the single-point crossover probability to 0.7.
[0044] 5-3. Using the real number coding method, randomly initialize the SVR model parameter pair (c, g) and perform binary coding. c is the penalty factor and g is the radial basis function.
[0045] 5-4. Substitute the initialized parameter pair into SVR and train it with the training set. Use cross-validation as the parameter optimization criterion. The mean square error between the test sample label and the predicted value obtained by SVR is used as the fitness value function. The smaller the value, the higher the fitness and the greater the probability of inheritance to the next generation. The formula is as follows:
[0046]
[0047] 5-5. Select N1 individuals according to the roulette rule or other selection methods, and calculate the average fitness value f of the group ave And the maximum fitness value f max .
[0048] 5-6. Randomly pair individuals in the population and calculate the adaptive crossover probability p for each pair of individuals c and mutation probability p m , with p c and p m Perform crossover and mutation operations for each probability.
[0049] 5-7. Calculate the fitness of the new individuals generated by crossover and mutation. The new individuals together with the parents constitute the new generation population.
[0050] 5-8. If the termination condition is met, the optimal parameter pair (c, g) is substituted into the SVR model for prediction. Otherwise, return to step (4).
[0051] 5-9 The light field image quality scores are predicted by the GA-SVR model, where Figure 2 This is the specific flow chart of the GA-SVR model.
Claims
1. A light field image quality evaluation method based on rich features, characterized by The steps include: Step 1: Randomly split the Win5-LID dataset into an 80% training set and a 20% test set. For the light field images, fix the two dimensions representing the angle information and convert them into a 9×9 sub-aperture image array. Step 2: Convert each sub-aperture image from RGB space to HSV coordinates, and extract brightness, hue, and saturation as color information features; Step 3: Divide the 9×9 sub-aperture image array into 9 groups by rows, perform tensor decomposition on each group, and obtain the first most important slice of each pseudo video group; divide the pseudo video sequence into two parts, perform shearlet transform, and extract the disparity structure features; Step 4: Obtain the polar plane image by fixing the light field image coordinates, use the Sobel operator to process the horizontal polar plane image and the vertical polar plane image, and then calculate their horizontal gradient and vertical gradient respectively to obtain the image angular texture features; Step 5: Normalize the color information features, parallax structure features, and image angle texture features, then splice and fuse them, and use the support vector model based on genetic algorithm to obtain the light field image prediction score.
2. The light field image quality evaluation method based on rich features according to claim 1 is characterized in that Step 1: Randomly split the Win5-LID dataset into an 80% training set and a 20% test set. Assume that the dimension of a light field image representing a scene is (u, v, s, t), where (u, v) is the angular coordinate. By fixing (u, v), it can be divided into a 9×9 sub-aperture image array.
3. The light field image quality assessment method based on rich features according to claim 1 is characterized in that Step 2 is implemented as follows: The brightness, chroma and saturation matrix of each sub-aperture image is represented as T x , where x represents brightness, chroma, and saturation; the chroma, brightness, and saturation matrices of the final light field image are calculated by the following formula: Where M and N represent the number of sub-aperture images in each horizontal and vertical direction of a light field image, where M = N = 9, and finally the required color information features [R1, R2, R3] are obtained.
4. The light field image quality evaluation method based on rich features according to claim 2 or 3, characterized in that Step 3: Extracting disparity structure features is specifically implemented as follows: 3.1 Pseudo video grouping is performed on the 9×9 sub-aperture images separated from the light field image. Each row is recorded as a group, denoted as p1, p2, p3…, p M , where M = 9 and the dimension of the pseudo video group is (M, S, T); perform high-order singular value decomposition on each pseudo video group, and the decomposition formula is as follows: p i =X×1F (1) ×2F (2) ×3F (3) Where X is the core tensor N is the length of the third dimension, F (i) are three factor matrices; × i is the modular product of tensor i, F (i) is an orthogonal matrix; here K1 = S, K2 = T, K3 = N; The decomposition components of the third dimension are calculated as follows: ξ=p i ×3(F (3) ) -1 Among them, (F (3) ) -1 F (3) The inverse matrix, ξ={C1,…,C N }, C1 represents the first slice of ξ, which contains 80% of the energy of slice ξ, and the energy gradually decreases as the slice number increases; 3.2 The first slice extracted from each set of pseudo video sequences is denoted as C m1 , where m = 1, 2…, M, perform shear wave transform, and the shear wave calculation formula is as follows: in, D a is the shear matrix, A a is the anisotropic expansion matrix, a represents the scale, s represents the direction, and t represents the location parameter; Each scale contains 6 shear coefficients in different directions, which are spliced into the parallax structure feature S.
5. The light field image quality assessment method based on rich features according to claim 4 is characterized in that Step 4 obtains the image angle texture feature and is specifically implemented as follows: By fixing the u and s coordinates of the light field image, the vertical polar plane image is obtained as follows: Fixed v, t coordinates to get the horizontal polar plane image: Then the gradient of the vertical polar plane image is obtained by filtering with the Sobel operator The calculation process is as follows: in, Finally, we get the gradient feature Similarly, the horizontal polar plane image is processed as above; finally, the features of the horizontal polar plane image and the vertical polar plane image are processed by mean and entropy to obtain the image angle texture feature [G mean ,G entropy ].