Tone mapping image quality evaluation method based on mixing of multiple image features

Abstract

The invention provides a tone mapping image quality evaluation method and system based on mixing of multiple image features. The method comprises the following steps: extracting texture features of a tone mapping image to measure the divergence degree and local detail distortion of an edge region of the tone mapping image; extracting color moment and color gradient features of the tone mapping image, performing trisection segmentation on the tone mapping image from two sides based on trisection composition, and extracting mean values and standard deviations of three channels of tone, saturation and brightness from each of the nine generated regions to jointly serve as aesthetic features; and combining the features into an integral feature vector, training a support vector regression device by taking the corresponding subjective quality score as a label, and testing and evaluating the tone mapping image by adopting the trained support vector regression device to obtain a predicted image objective quality score. According to the invention, the performance and subjective consistency of the non-reference tone mapping image quality evaluation technology are improved, and the evaluation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing evaluation, and in particular to a tone mapping image quality evaluation method based on a mixture of multiple image features (texture, color and aesthetic features), and also to a corresponding system, terminal and medium. Background Art

[0002] High dynamic range (HDR) has advantages in capturing, manipulating and displaying real-world lighting, while consumer-level terminal display devices only have 8-bit or 10-bit standard dynamic range (SDR), which makes it difficult to accurately represent images with higher dynamic range. Tone mapping operators (TMOs) are widely used in the acquisition, compression, transmission and display of HDR images, aiming to render HDR scenes in SDR display devices. However, after using different TMOs, HDR images not only suffer from distortion of naturalness attributes, but also distortion of aesthetic attributes in terms of color and composition. In actual application scenarios, in order to measure the performance of different TMOs and select the best tone mapping image, a large amount of subjective evaluation with human participation is required, which is difficult to meet the actual application requirements. At the same time, the original HDR image is difficult to obtain. Therefore, objective tone quality evaluation methods without reference are more valuable for research. They can be used to automatically select the best tone quality evaluation index from candidate tone quality evaluation indicators generated by different tone quality evaluation indicators online, and can also be used to seek optimal parameters in the tone mapping process. Therefore, there is an urgent need to provide a quality evaluation technology for tone mapping images with better performance.

[0003] Gu et al. extracted features from aspects such as information volume, naturalness and structure, ignoring color distortion. Yue et al. measured the distortion of tone mapping images from exposure, color and naturalness, but ignored the changes in aesthetic perception caused by quality, and the performance was not high. The Chinese invention patent "A blind reference tone mapping image quality evaluation method" applied by Jiang Gangyi et al. extracts the detail information of local and global areas from the highlight and low-light areas of the image to measure the detail information. The time complexity is high and it is not suitable for real-time evaluation after terminal tone mapping. Based on the above analysis, the existing methods are not comprehensive enough and their subjective consistency needs to be improved. Therefore, it is necessary to develop a more efficient tone mapping evaluation method to meet the requirements of standard dynamic range terminal display devices for high dynamic range imaging. At present, no description or report of similar technology to the present invention has been found, and similar information at home and abroad has not been collected. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a tone mapping image quality evaluation method based on a mixture of multiple image features, and also provides a corresponding system, terminal and medium.

[0005] According to one aspect of the present invention, a tone mapping image quality assessment method based on a mixture of multiple image features is provided, comprising:

[0006] Extracting texture structural features of the tone mapping image, wherein the texture structural features are used to measure the degree of divergence of edge regions of the tone mapping image and local detail distortion;

[0007] Extract the color moment and gradient features of each color channel of the tone mapping image as color features;

[0008] Extracting a tone feature of the tone-mapped image as a first aesthetic feature;

[0009] extracting composition features of the tone-mapped image as a second aesthetic feature;

[0010] The above-extracted features are combined into an overall feature vector, and a support vector regressor is trained based on a corresponding subjective quality score as a label. The trained support vector regressor is used to test and evaluate the tone mapping image to obtain the predicted objective quality score of the image.

[0011] Preferably, extracting texture structural features of the tone mapping image comprises:

[0012] The Sobel operator is used to extract the edge map EM of each tone-mapped image; the mean μ of the brightness block of a preset size around each pixel in the obtained edge map EM is calculated. L , variance σ L , skewness s L and peak value k L , to measure the brightness of the image edge and the degree of divergence of the edge area;

[0013] A centrosymmetric local binary gradient operator is provided to perform local texture measurement on the edge map, and each tone mapping image generates a local structural feature vector of 16; wherein the centrosymmetric local binary gradient operator XCS-LBP is:

[0014]

[0015] ρ(a,b,c)=a*2 2 +b*2 1 +c*2 0 ;

[0016]

[0017] Among them, N is the total number of neighbors of the pixel involved, m c is the gray value of the center pixel, m i(i=0,1,…,(N / 2)-1) is the gray value of adjacent pixels, ρ(a,b,c) is the binary encoding of each element in high-low order weighting, and τ(x,y) is the difference between the central pixel and the adjacent pixels.

[0018] Preferably, extracting the color moment and gradient features of each color channel of the tone mapping image comprises:

[0019] The tone-mapped image RGB space is converted to the HSV color space, and the mean, standard deviation, and skewness of the saturation (S) and value (V) channels are extracted as color moment features;

[0020] Calculate the vector gradient VG and the corresponding angle array VA of each color channel of the tone-mapped RGB image; wherein VG and VA are:

[0021] A=0.5·(atan(2*gxy. / (gxy-gyy+eps)));

[0022] G1=0.5*((gxx+gyy)+(gxx-gyy).*cos(2*A)+2*gxy.*sin(2*A));

[0023] A=A+π / 2;

[0024] G2=0.5*((gxx+gyy)+(gxx-gyy).*cos(2*A)+2*gxy.*sin(2*A));

[0025] VG = mat2gray(max(G1,G2));

[0026]

[0027] Among them, G1 and G2 are the edge map EM and its transpose obtained for each color channel in the HSV color space; gxx, gyy, gxy are the parameters of the vector gradient respectively; A is the angle calculated based on multiple vector gradients, which is accumulated by repeatedly calculating the angle π / 2;

[0028] The composite gradient PPG per plane is obtained by summing the gradients RG, GG, and BG of each color plane; wherein the composite gradient PPG is:

[0029] PPG=mat2gray(RG+GG+BG).

[0030] Preferably, extracting the tone feature of the tone mapping image comprises:

[0031] Normalize the brightness value L to between [0 1] and sort them in ascending order, and calculate the two hue components, which are denoted as c uand c o ; wherein the c u and c o They are:

[0032] c u =min(0.5,p 30 (L)-p 5 (L)) / 0.5;

[0033] c o =min(0.5,p 95 (L)-p 70 (L)) / 0.5;

[0034] Among them, p 30 、p 5 、p 95 、p 70 represent the 30th, 5th, 95th, and 70th percentile values ​​in L respectively.

[0035] Preferably, extracting the composition features of the tone mapping image comprises:

[0036] Based on the rule of thirds, the tone-mapped image is divided into three equal parts from both sides to form nine image blocks;

[0037] Calculate the mean A of the hue (H), saturation (S), and value (V) channels of each block respectively m and standard deviation A s , as the second aesthetic feature.

[0038] Preferably, the subjective quality score is provided by a public tone-mapped image database.

[0039] Preferably, the method further comprises:

[0040] The predicted objective quality score of the image is compared with the corresponding subjective quality score provided in the image database to verify the prediction accuracy.

[0041] According to another aspect of the present invention, a tone mapping image quality assessment system based on a mixture of multiple image features is provided, comprising:

[0042] A texture feature extraction module, which is used to extract texture structural features of the tone mapping image, wherein the texture features are used to measure the degree of divergence of the edge area of ​​the tone mapping image and the distortion of local details;

[0043] A color feature extraction module, which is used to extract the color moment and gradient features of each color channel of the tone mapping image as color features;

[0044] An aesthetic feature extraction module, which is used to extract tone features and composition features of the tone mapping image as aesthetic features;

[0045] The tone mapping image quality assessment module is used to combine the features extracted by the texture feature extraction module, the color feature extraction module and the aesthetic feature extraction module into an overall feature vector, train a support vector regressor based on a corresponding subjective quality score as a label, and use the trained support vector regressor to test and evaluate the tone mapping image to obtain a predicted objective image quality score.

[0046] Preferably, the system further comprises:

[0047] A verification module is used to compare the predicted objective quality score of the image with the corresponding subjective quality score provided in the image database to verify the prediction accuracy.

[0048] According to a third aspect of the present invention, there is provided a quality assessment terminal for tone-mapped images, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor is used to execute any one of the methods described above in the present invention, or to execute any one of the systems described above in the present invention when executing the computer program.

[0049] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is used to execute any of the above-mentioned methods of the present invention, or to run any of the above-mentioned systems of the present invention.

[0050] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0051] The tone mapping image quality evaluation method based on a mixture of multiple image features provided by the present invention can accurately and effectively predict the quality of the tone mapping image by extracting texture features, color features and aesthetic features.

[0052] The tone mapping image quality assessment method based on the mixture of multiple image features provided by the present invention further improves the performance and subjective consistency of the reference-free tone mapping image (TMI) quality assessment technology by measuring attributes such as overall exposure, texture, and chromaticity.

[0053] The tone mapping image quality evaluation method based on a mixture of multiple image features provided by the present invention improves the efficiency of tone mapping image quality evaluation by accurately measuring the correlation of local spatial distribution of different exposure areas and the difference of regional color distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0055] Figure 1 The flowchart is a method for evaluating the quality of a tone mapping image based on a mixture of multiple image features in one embodiment of the present invention.

[0056] Figure 2 The figure is a schematic diagram of the components of a tone mapping image quality assessment system based on a mixture of multiple image features in one embodiment of the present invention.

[0057] Figure 3 The figure is a workflow diagram of a tone mapping image quality assessment method based on a mixture of multiple image features in a specific application example of the present invention.

[0058] Figure 4 (a) to (c) are texture structure feature maps extracted by the Sobel operator in a specific application example of the present invention. DETAILED DESCRIPTION

[0059] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0060] The tone mapping image quality assessment technology used in the prior art usually has the technical problem of high feature extraction complexity. In view of the above problem, an embodiment of the present invention provides a tone mapping image quality assessment method based on a mixture of multiple image features. The method includes texture measurement, color measurement and aesthetic feature measurement. By extracting edge map texture features, color features and two aesthetic features of hue and three-dimensional composition, the performance and subjective consistency of the reference-free tone mapping image (TMI) quality assessment technology are improved, and the evaluation efficiency is improved.

[0061] Specifically, Figure 1 As shown, the tone mapping image quality assessment method based on a mixture of multiple image features provided in this embodiment may include the following operations:

[0062] S100, extracting texture structural features of the tone mapping image, where the texture structural features are used to measure the degree of divergence of edge regions of the tone mapping image and local detail distortion;

[0063] S200, extracting the color moment and gradient features of each color channel of the tone mapping image as color features to measure color distortion;

[0064] S300, extracting a tone feature of the tone mapping image as a first aesthetic feature;

[0065] S400, extracting a composition feature of the tone mapping image as a second aesthetic feature;

[0066] S500, combining texture features, color features, first aesthetic features and second aesthetic features into an overall feature vector, training a support vector regressor based on a corresponding subjective quality score as a label, and using the trained support vector regressor to test and evaluate the tone mapping image to obtain a predicted objective image quality score.

[0067] In some preferred embodiments, the above method may further include the following operations:

[0068] S600, comparing the predicted objective quality score of the image with the corresponding subjective quality score provided in the image database to verify the prediction accuracy.

[0069] In some preferred implementations, the above S100, extracting the texture structure features of the tone mapping image, may further include the following operations:

[0070] S101, using the Sobel operator to extract the edge map EM of each tone mapping image; calculating the mean value μ of the brightness block of a preset size around each pixel in the obtained edge map EM L , variance σ L , skewness s L and peak value k L , to measure the brightness of the image edge and the degree of divergence of the edge area;

[0071] S102, design a centrosymmetric local binary gradient operator to perform local texture measurement on the edge image, and generate 16 local structure feature vectors for each tone mapping image; wherein the centrosymmetric local binary gradient operator XCS-LBP is:

[0072]

[0073] ρ(a,b,c)=a*2 2 +b*2 1 +c*2 0 ;

[0074]

[0075] Among them, N is the total number of neighbors of the pixel involved, m c is the gray value of the center pixel, m i(i=0,1,…,(N / 2)-1) is the grayscale value of adjacent pixels, ρ(a,b,c) is the binary coded weighting of each element in order, and τ(x,y) is the difference between the central pixel and the adjacent pixels.

[0076] The extracted texture features, such as Figure 4 As shown in (a) to (c).

[0077] In some preferred embodiments, the above S200, extracting the color features of the tone mapping image, may further include the following operations:

[0078] S201, converting the tone mapping image RGB space into HSV color space, extracting the mean, standard deviation and skewness of saturation (S) and value (V) channels as color moment features;

[0079] S202, calculate the vector gradient VG and the corresponding angle array VA of each color channel of the tone-mapped RGB image. Wherein, VG and VA are:

[0080] A=0.5·(atan(2*gxy. / (gxy-gyy+eps)));

[0081] G1=0.5*((gxx+gyy)+(gxx-gyy).*cos(2*A)+2*gxy.*sin(2*A));

[0082] A=A+π / 2;

[0083] G2=0.5*((gxx+gyy)+(gxx-gyy).*cos(2*A)+2*gxy.*sin(2*A));

[0084] VG = mat2gray(max(G1,G2));

[0085]

[0086] Among them, G1 and G2 are the edge map EM and its transpose obtained for each color channel in the HSV color space; gxx, gyy, gxy are the parameters of the vector gradient respectively; A is the angle calculated based on multiple vector gradients, which is accumulated by repeatedly calculating the angle π / 2;

[0087] S203: Sum the gradients RG, GG, and BG of each color plane to obtain a composite gradient PPG for each plane. The composite gradient PPG is:

[0088] PPG=mat2gray(RG+GG+BG).

[0089] In some preferred implementations, the above S300, extracting the tone features of the tone mapping image, may further include the following operations:

[0090] S301, normalize the brightness value L to between [0 1] and sort them in ascending order, calculate two hue components, denoted as c u and c o Among them, c u and c o They are:

[0091] c u =min(0.5,p 30 (L)-p 5 (L)) / 0.5;

[0092] c o =min(0.5,p 95 (L)-p 70 (L)) / 0.5;

[0093] Among them, p 30 , p 5 , p 95 , p 70 represent the 30th, 5th, 95th, and 70th percentile values ​​in L respectively.

[0094] In some preferred implementations, the above S400, extracting the composition features of the tone mapping image, may further include the following operations:

[0095] The tone mapping image is equally divided into multiple regions, and the mean of the hue, saturation and brightness channels of each region is extracted. The following steps may be further performed:

[0096] S401, dividing the tone mapping image into three equal parts from both sides based on the rule of thirds composition to form nine image blocks;

[0097] S402, respectively calculate the mean value A of the hue (H), saturation (S) and value (V) channels of each block m and standard deviation A s , as the second aesthetic feature.

[0098] In some preferred implementations, the above S500 mainly includes the following two parts:

[0099] S501: training model: combining the features extracted from S100 to S400 into an overall feature vector, using the corresponding subjective quality scores provided in the public authoritative tone mapping image database as labels, and training the support vector regressor through continuous iterations;

[0100] S502: Testing: Input the feature values ​​of the test data in the data set into the trained support vector regressor to obtain the predicted objective quality score of the image.

[0101] In the above embodiment, S100 to S400 do not need to be executed strictly in the above order, but can be executed in any order.

[0102] The tone mapping image quality assessment method based on a mixture of multiple image features provided by the above-mentioned embodiment of the present invention can accurately and effectively predict the quality of the tone mapping image by extracting texture features, color features and aesthetic features.

[0103] An embodiment of the present invention provides a tone mapping image quality assessment system based on a mixture of multiple image features.

[0104] like Figure 2 As shown, the tone mapping image quality assessment system based on a mixture of multiple image features provided in this embodiment may include the following modules:

[0105] Texture feature extraction module, which is used to extract the texture structure features of the tone mapping image. The texture features are used to measure the degree of divergence of the edge area of ​​the tone mapping image and the distortion of local details;

[0106] A color feature extraction module, which is used to extract the color moment and gradient features of each color channel of the tone mapping image as color features;

[0107] An aesthetic feature extraction module, which is used to extract tone features and composition features of the tone mapping image as aesthetic features;

[0108] The tone mapping image quality evaluation module is used to combine the features extracted by the texture feature extraction module, the color feature extraction module and the aesthetic feature extraction module into an overall feature vector, train a support vector regressor based on a corresponding subjective quality score as a label, and use the trained support vector regressor to test and evaluate the tone mapping image to obtain the predicted objective image quality score.

[0109] In some preferred embodiments, the above system may further include the following modules:

[0110] The verification module is used to compare the predicted objective quality score of the image with the corresponding subjective quality score provided in the image database to verify the prediction accuracy.

[0111] In some preferred embodiments, the aesthetic feature extraction module further comprises:

[0112] A first aesthetic feature extraction module, used to extract a tone feature of the tone mapping image as a first aesthetic feature;

[0113] The second aesthetic feature extraction module is used to extract the composition feature of the tone mapping image as the second aesthetic feature.

[0114] In some preferred implementations, the tone mapping image quality assessment module mainly includes the following three units:

[0115] The model training unit is used to combine the features extracted by the texture feature extraction unit, the color feature extraction unit, and the aesthetic feature extraction unit into an overall feature vector, and use the corresponding subjective quality scores provided in the public authoritative tone mapping image database as labels to train the support vector regressor through continuous iterations.

[0116] A testing unit, which is used to synthesize the feature values ​​of the test data in the data set into an overall feature vector, and train the support vector regressor through continuous iterations using the corresponding subjective quality scores provided in the image database as labels;

[0117] The prediction unit is used to input the feature values ​​of the test data in the data set into the support vector regressor trained by the model training unit to obtain the predicted objective quality score of the image.

[0118] It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system, that is, the embodiments in the method can be understood as preferred examples for constructing the system, which will not be elaborated here.

[0119] The technical solution provided by the above embodiment of the present invention is further described in detail below in conjunction with a specific application example.

[0120] In this specific application example, the tone mapping image quality evaluation method based on a mixture of multiple image features is involved, such as Figure 3 As shown, the following steps are included:

[0121] S1: Extract the texture structure features of the tone mapping image. The texture structure features are used to measure the degree of divergence in the edge area of ​​the tone mapping image and the distortion of local details.

[0122] S2: extract the color moment and gradient features of each color channel of the tone mapping image as color features, which are used to measure color distortion;

[0123] S3-S4, extraction of two aesthetic features: hue and composition. Aesthetic features are used to measure the balance of image color and geometry (including geometric structures such as composition, lines, and shapes):

[0124] S3: extracting the tone feature of the tone mapping image as the first aesthetic feature;

[0125] S4: extracting the composition features of the tone-mapped image as the second aesthetic features;

[0126] S5, train the model and test: combine the features extracted by S1 to S4 into an overall feature vector, use the corresponding subjective quality scores provided in the public authoritative tone mapping image database as labels, train the support vector regressor, use the trained support vector regressor for test evaluation, and obtain the predicted objective image quality score.

[0127] In this method, after the above steps, the following steps may also be included:

[0128] S6, comparing the objective quality score of the image with the corresponding subjective quality score provided in the image database to verify the prediction accuracy.

[0129] In this specific application example, the tone mapping image quality assessment system based on a mixture of multiple image features for implementing the above method includes the following modules:

[0130] Texture feature extraction module, which is used to extract the texture structure features of the edge map of the tone mapping image, and is used to measure the degree of divergence of the edge area of ​​the tone mapping image and the distortion of local details;

[0131] A color feature extraction module is used to extract the color moment and gradient features of each color channel of the tone mapping image as color features for measuring color distortion;

[0132] Aesthetic feature extraction module, which is used to extract tone features and composition features as aesthetic features;

[0133] The tone mapping image quality assessment module, whose model training unit is used to combine the features extracted by the texture feature extraction module, the color feature extraction module, and the aesthetic feature extraction module into an overall feature vector, and use the corresponding subjective quality scores provided in the image database as labels to train the support vector regressor through continuous iterations; its testing unit is used to synthesize the feature values ​​of the test data in the data set into an overall feature vector, and use the corresponding subjective quality scores provided in the image database as labels to train the support vector regressor through continuous iterations; the testing unit is used to input the feature values ​​of the test data in the data set into the support vector regressor trained by the model training unit to obtain the predicted objective quality score of the image.

[0134] In order to verify the performance of the above methods and systems, the following implementation was carried out on the ESPL-LIVE HDR database. This database is the largest TMI database released so far, with 1811 tone-mapped HDR images generated by tone mapping, multi-exposure fusion algorithms and post-processing methods. In the experiment, several excellent natural image and TMI quality evaluation algorithms were selected as comparison algorithms. The ones in italics are natural image quality evaluation algorithms. Three commonly used evaluation indicators were used to measure algorithm performance, namely: Pearson linear correlation coefficient (PLCC), Spearman rank correlation coefficient (SRCC), and mean square error (RMSE). Among them, the larger the values ​​of PLCC and SROCC, the smaller the value of RMSE, indicating that the objective image quality evaluation algorithm is better.

[0135] In order to improve the accuracy of the method, the experiment adopted the training and testing process to be executed 1000 times on the support vector machine, and finally the average of the 1000 results was taken to represent the performance results of the algorithm. Table 1 shows the overall performance of the present invention and other excellent algorithms on the ESPL-LIVE HDR database. It can be seen that the overall performance of the method of the present invention is significantly better than the performance of several other algorithms.

[0136] Table 1 Overall performance comparison of the present invention with several mainstream no-reference algorithms on the ESPL-LIVE HDR database

[0137]

[0138] In addition, the performance of different distortion types is evaluated (see Table 2). The results with the best performance are marked in bold. It can be seen that the algorithm proposed in the present invention performs best in tone mapping and post-processing, and ranks third in multi-exposure fusion distortion. Its overall performance is better than other algorithms. Therefore, the algorithm of the present invention can evaluate the quality of TMI more accurately and effectively.

[0139] Table 2 Performance evaluation of the present invention and other no-reference quality evaluation algorithms on different distortion types

[0140]

[0141] An embodiment of the present invention provides a quality assessment terminal for tone-mapped images, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to execute the method of any one of the above-mentioned embodiments of the present invention when executing the computer program, or to execute the system of any one of the above-mentioned embodiments of the present invention.

[0142] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0143] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0144] The processor is used to execute the computer program stored in the memory to implement the various steps of the method or various modules of the system involved in the above embodiments. For details, please refer to the relevant descriptions in the above method and system embodiments.

[0145] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0146] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is used to execute the method of any one of the above embodiments of the present invention, or to run the system of any one of the above embodiments of the present invention.

[0147] The quality assessment method and system of tone mapping images based on a mixture of multiple image features provided by the above-mentioned embodiments of the present invention can accurately and effectively predict the quality of tone mapping images by extracting texture features, color features and aesthetic features; by measuring local texture, local color, aesthetics and other attributes, the performance and subjective consistency of the reference-free tone mapping image (TMI) quality assessment technology are further improved; by accurately measuring the correlation of local texture spatial distribution and the difference in local color distortion, the efficiency of tone mapping image quality assessment is improved.

[0148] Those skilled in the art know that, in addition to implementing the system and its various devices provided by the present invention in a purely computer-readable program code, the system and its various devices provided by the present invention can be made to implement the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices provided by the present invention can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component; the devices for implementing various functions can also be considered as both software modules for implementing the method and structures within the hardware component.

[0149] All matters not covered in the above embodiments of the present invention are well known in the art.

[0150] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A tone mapping image quality assessment method based on a mixture of multiple image features, characterized in that: include: Extracting texture structural features of the tone mapping image, wherein the texture structural features are used to measure the degree of divergence of edge regions of the tone mapping image and local detail distortion; Extracting color moment and gradient features of each color channel of the tone mapping image as color features, wherein the color features are used to measure color distortion; Extracting a tone feature of the tone-mapped image as a first aesthetic feature; extracting composition features of the tone-mapped image as a second aesthetic feature; The texture structure feature, color feature, first aesthetic feature and second aesthetic feature are combined into an overall feature vector, a support vector regressor is trained based on a corresponding subjective quality score as a label, and the trained support vector regressor is used to test and evaluate the tone mapping image to obtain a predicted objective image quality score.

2. The tone mapping image quality assessment method based on a mixture of multiple image features according to claim 1, characterized in that: The extracting of texture structure features of the tone mapping image comprises: The Sobel operator is used to extract the edge map EM of each tone-mapped image; the mean μ of the brightness block of a preset size around each pixel in the obtained edge map EM is calculated. L , variance σ L , skewness s L and peak value k L , to measure the brightness of the image edge and the degree of divergence of the edge area; A centrosymmetric local binary gradient operator is provided to perform local texture measurement on the edge map, and each tone mapping image generates a local structural feature vector of 16; wherein the centrosymmetric local binary gradient operator XCS-LBP is: ρ(a,b,c)*2 2 +b*2 1 +c*2 0 100. Among them, N is the total number of neighbors of the pixel involved, m c is the gray value of the center pixel, m i (i=0,1,…,(N / 2)-1) is the grayscale value of adjacent pixels, ρ(z,b,c) is the binary coded weighting of each element in order, and τ(x,y) is the difference between the central pixel and the adjacent pixels.

3. The tone mapping image quality assessment method based on a mixture of multiple image features according to claim 1, characterized in that: The method of extracting the color moment and gradient features of each color channel of the tone mapping image comprises: The tone-mapped image RGB space is converted to the HSV color space, and the mean, standard deviation, and skewness of the saturation (S) and value (V) channels are extracted as color moment features; Calculate the vector gradient VG and the corresponding angle array VA of each color channel of the tone-mapped RGB image; wherein VG and VA are: A=0.5·*atan(2*gxy. / (gxy-gyy+eps))); G1=0.5*((gxx+gyy)+(gxx-gyy).*cos(2*A)+2*gxy.*sin(2*A)); A=A+π / 2; G2=0.5*((gxx+gyy)+(gxx-gyy).*cos(2*A)+2*gxy.*sin(2*A)); VG = mat2gray(max(G1,G2)); Among them, G1 and G2 are the edge map EM and its transpose obtained for each color channel in the HSV color space; gxx, gyy, gxy are the parameters of the vector gradient respectively; A is the angle calculated based on multiple vector gradients, which is accumulated by repeatedly calculating the angle π / 2; The composite gradient PPG per plane is obtained by summing the gradients RG, GG, and BG of each color plane; wherein the composite gradient PPG is: PPG=mat2gray(RG+GG+BG).

4. The quality assessment method of a tone-mapped image with mixed texture, color and aesthetic features according to claim 1, characterized in that: The step of extracting the tone feature of the tone mapping image comprises: Normalize the brightness value L to between [0 1] and sort them in ascending order, and calculate the two hue components, which are denoted as c u and c o ; wherein the c u and c o They are: c u =min(0.5,p 30 (L)-p 5 (L)) / 0.5; c o =min(0.5,P 95 (L)-P 70 (L)) / 0.5; Among them, p 30 、p 5 、p 95 、p 70 represent the 30th, 5th, 95th, and 70th percentile values ​​in L respectively.

5. The quality assessment method of tone mapping images based on a mixture of multiple image features according to claim 1, characterized in that: The extracting composition features of the tone mapping image comprises: Based on the rule of thirds, the tone-mapped image is divided into three equal parts from both sides to form nine image blocks; Calculate the mean A of the hue (H), saturation (S), and value (V) channels of each block respectively m and standard deviation A s , as the second aesthetic feature.

6. The quality assessment method of tone mapping image based on a mixture of multiple image features according to any one of claims 1 to 5, characterized in that: Also includes any one or more of the following: - the subjective quality scores are provided by a public database of tone-mapped images; - Comparing the predicted objective quality score of the image with the corresponding subjective quality score provided in the image database to verify the prediction accuracy.

7. A tone mapping image quality assessment system based on a mixture of multiple image features, characterized in that: include: A texture feature extraction module, which is used to extract texture structural features of the tone mapping image, wherein the texture features are used to measure the degree of divergence of the edge area of ​​the tone mapping image and the distortion of local details; A color feature extraction module, which is used to extract the color moment and gradient features of each color channel of the tone mapping image as color features; An aesthetic feature extraction module, which is used to extract tone features and composition features of the tone mapping image as aesthetic features; The tone mapping image quality assessment module is used to combine the features extracted by the texture feature extraction module, the color feature extraction module and the aesthetic feature extraction module into an overall feature vector, train a support vector regressor based on a corresponding subjective quality score as a label, and use the trained support vector regressor to test and evaluate the tone mapping image to obtain a predicted objective image quality score.

8. The tone mapping image quality assessment system based on a mixture of multiple image features according to claim 1, characterized in that: Also includes: A verification module is used to compare the predicted objective quality score of the image with the corresponding subjective quality score provided in the image database to verify the prediction accuracy.

9. A quality assessment terminal for tone mapping images, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to execute the method described in any one of claims 1 to 6, or to execute the system described in any one of claims 7 to 8 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it is used to perform the method according to any one of claims 1 to 6, or to run the system according to any one of claims 7 to 8.

Images