An image quality assessment method, apparatus, device, and storage medium

Abstract

This invention discloses an image quality assessment method, apparatus, device, and storage medium. The method includes: acquiring an image to be assessed; inputting the image to be assessed into an image quality assessment network to obtain an image quality assessment result; the image quality assessment network is used to: extract a first image feature from the input image; perform a shift operation on the first image feature to obtain one or more second image features; and combine the first image feature and the obtained second image features to determine the image quality assessment result; the second image feature has the same size as the first image feature, and there are regions with the same feature value at different positions between the first image feature and the second image feature.

Description

Technical Field

[0001] This invention relates to the field of computer application technology, and in particular to an image quality assessment method, apparatus, device, and storage medium. Background Art

[0002] Currently, image quality assessment is often conducted manually. For example, people score the quality of an image, and the image quality is determined based on these scores.

[0003] Specifically, the image can be a video frame, and the quality of the video can be determined by evaluating the quality of the video frames. However, this manual evaluation method is inefficient. Summary of the Invention

[0004] This invention provides an image quality assessment method, apparatus, device, and storage medium to address the shortcomings of related technologies.

[0005] According to a first aspect of the present invention, an image quality assessment method is provided, comprising:

[0006] Acquire the image to be evaluated;

[0007] The image to be evaluated is input into the image quality evaluation network to obtain the image quality evaluation result;

[0008] The image quality assessment network is used to: extract a first image feature from an input image, perform a shift operation on the first image feature to obtain one or more second image features, and combine the first image feature and the obtained second image feature to determine the image quality assessment result;

[0009] The second image feature has the same size as the first image feature, and there are regions at different locations between the first image feature and the second image feature where the feature values ​​are the same.

[0010] Optionally, determining the image quality assessment result by combining the first image features and the acquired second image features includes:

[0011] By combining the first image features and the acquired second image features, one or more image attribute evaluation results are determined;

[0012] Based on the determined image attribute evaluation results, the image quality evaluation results are determined.

[0013] Optionally, the image quality assessment network includes N cascaded preset modules; N≥2; for the i-th preset module, 1≤i≤N-1, its output is cascaded to the input of the (i+1)-th preset module;

[0014] The preset module is used to perform a shift operation on the input image features to obtain one or more third image features, and to further extract and output image features by combining the input image features and the obtained third image features; the third image features have the same size as the input image features, and there are regions where the feature values ​​are the same between the input image features and the third image features at different positions.

[0015] The step of performing a shift operation on the first image feature to obtain one or more second image features, and combining the first image feature and the obtained second image features to determine the image quality assessment result includes:

[0016] The first image feature is input into the first preset module of the N cascaded preset modules to obtain the image feature output by the Nth preset module;

[0017] The image quality assessment result is determined based on the image features output by the Nth preset module.

[0018] Optionally, the step of performing a shift operation on the first image feature to obtain one or more second image features includes:

[0019] A shift operation is performed on the features of a preset region in the first image features to obtain one or more shift results;

[0020] For each shift result, fill the default part with a preset value to obtain a second image feature with the same size as the first image feature.

[0021] Optionally, determining the image quality assessment result by combining the first image features and the acquired second image features includes:

[0022] By combining the first image features and the acquired second image features, spatial feature information of the image is further extracted, and the image quality assessment result is determined based on the extracted spatial feature information.

[0023] Optionally, the first image feature includes at least one of the following: image detail features, image noise features, and image global features.

[0024] Optionally, extracting the first image features from the input image includes:

[0025] Obtain preset information from the input image, and further extract image features based on the preset information;

[0026] Add the extracted image features to the first image features;

[0027] The preset information includes at least one of the following: original image, image detail information, image noise information, image brightness information, image saturation information, and image tone information.

[0028] Optionally, obtaining preset information from the input image includes:

[0029] Edge filtering is applied to the input image to extract image detail information; and / or

[0030] The input image is subjected to guided filtering to obtain a denoised blurred image. The input image and the blurred image are combined to extract image noise information.

[0031] Optionally, further extracting image features based on the preset information includes:

[0032] Different feature extraction networks are used to extract image features for different preset information.

[0033] Optionally, the method for generating the training set of the image quality assessment network includes:

[0034] Obtain unlabeled image samples;

[0035] For the unlabeled image samples, calculate the corresponding image attribute values ​​and determine the corresponding image quality assessment labels based on the calculation results.

[0036] According to a second aspect of the present invention, an image quality assessment apparatus is provided, comprising:

[0037] The acquisition unit is used to acquire the image to be evaluated;

[0038] The evaluation unit is used to input the image to be evaluated into the image quality evaluation network to obtain the image quality evaluation result;

[0039] The image quality assessment network is used to: extract a first image feature from an input image, perform a shift operation on the first image feature to obtain one or more second image features, and combine the first image feature and the obtained second image feature to determine the image quality assessment result;

[0040] The second image feature has the same size as the first image feature, and there are regions at different locations between the first image feature and the second image feature where the feature values ​​are the same.

[0041] Optionally, the image quality assessment network is used to: integrate the first image features and the acquired second image features to determine one or more image attribute assessment results; and determine the image quality assessment result based on the determined image attribute assessment results.

[0042] Optionally, the image quality assessment network includes N cascaded preset modules; N≥2; for the i-th preset module, 1≤i≤N-1, its output is cascaded to the input of the (i+1)-th preset module;

[0043] The preset module is used to perform a shift operation on the input image features to obtain one or more third image features, and to further extract and output image features by combining the input image features and the obtained third image features; the third image features have the same size as the input image features, and there are regions where the feature values ​​are the same between the input image features and the third image features at different positions.

[0044] The image quality assessment network is used to: input the first image feature into the first preset module of the N cascaded preset modules to obtain the image feature output by the Nth preset module; and determine the image quality assessment result based on the image feature output by the Nth preset module.

[0045] Optionally, the image quality assessment network is used to: perform a shift operation on the features of a preset region in the first image features to obtain one or more shift results; fill the default part in each shift result with a preset value to obtain a second image feature with the same size as the first image feature.

[0046] Optionally, the image quality assessment network is used to: integrate the first image features and the acquired second image features to further extract spatial feature information of the image, and determine the image quality assessment result based on the extracted spatial feature information.

[0047] Optionally, the first image feature includes at least one of the following: image detail features, image noise features, and image global features.

[0048] Optionally, the image quality assessment network is used to: acquire preset information in the input image, and further extract image features based on the preset information; add the extracted image features to the first image features; the preset information includes at least one of the following: original image, image detail information, image noise information, image brightness information, image saturation information, and image hue information.

[0049] Optionally, the image quality assessment network is used for:

[0050] Edge filtering is applied to the input image to extract image detail information; and / or

[0051] The input image is subjected to guided filtering to obtain a denoised blurred image. The input image and the blurred image are combined to extract image noise information.

[0052] Optionally, the image quality assessment network is used to extract image features using different feature extraction networks for different preset information.

[0053] Optionally, the method for generating the training set of the image quality assessment network includes:

[0054] Obtain unlabeled image samples;

[0055] For the unlabeled image samples, calculate the corresponding image attribute values ​​and determine the corresponding image quality assessment labels based on the calculation results.

[0056] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0057] At least one processor; and,

[0058] A memory communicatively connected to the at least one processor; wherein,

[0059] The memory stores instructions that can be executed by the one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any of the above method embodiments.

[0060] According to a fourth aspect of the present invention, a computer-readable storage medium storing a computer program is provided, which, when executed by a processor, implements any of the above-described method embodiments.

[0061] As can be seen from the above embodiments, the quality of input images can be automatically evaluated by utilizing a pre-trained image quality evaluation network, thereby improving the efficiency of quality evaluation and reducing manual costs.

[0062] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0064] Figure 1 This is a flowchart illustrating an image quality assessment method according to an embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram illustrating the principle of an image quality assessment network according to an embodiment of the present invention;

[0066] Figure 3 This is a flowchart illustrating an edge filtering feature extraction method according to an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of a guided filter feature extraction process according to an embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram of the structure of a feature extraction network according to an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram illustrating the principle of an image global feature extraction network according to an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram illustrating the principle of a first image feature extraction method according to an embodiment of the present invention;

[0071] Figure 8 This is a schematic diagram illustrating the principle of image feature shifting according to an embodiment of the present invention;

[0072] Figure 9 This is a schematic diagram of the structure of a spatial feature extraction network according to an embodiment of the present invention;

[0073] Figure 10 This is a schematic diagram of the structure of a ShiftNet layer according to an embodiment of the present invention;

[0074] Figure 11 This is a schematic diagram of the structure of a regression network according to an embodiment of the present invention;

[0075] Figure 12 This is a schematic diagram of the structure of an image quality assessment network according to an embodiment of the present invention;

[0076] Figure 13 This is a schematic diagram of the structure of an image quality assessment device according to an embodiment of the present invention;

[0077] Figure 14 This is a schematic diagram of the hardware structure of a computer device configured according to an embodiment of the present invention. Detailed Implementation

[0078] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0079] Currently, image quality assessment is often conducted manually. For example, people score the quality of an image, and the image quality is determined based on these scores.

[0080] Specifically, the image can be a video frame, and the quality of the video can be determined by evaluating the quality of the video frames. However, this manual evaluation method is inefficient.

[0081] To address the aforementioned problems, embodiments of the present invention provide an image quality assessment method.

[0082] In this method, a pre-trained image quality assessment network can be used to automatically assess the quality of input images. Compared with manual image quality assessment, this method can improve the efficiency of quality assessment and reduce labor costs.

[0083] Furthermore, in this method, image features related to image quality can be introduced into the image quality assessment network for prediction, thereby improving the accuracy of image quality assessment.

[0084] For example, noise features of the image can be introduced. Generally, the greater the noise in an image, the lower its quality. By introducing noise features, the accuracy of the image quality assessment network can be improved.

[0085] For example, detailed feature information of the image can be introduced. Generally, the more details an image has, the higher its quality. By introducing detailed feature information of the image, the accuracy of the image quality assessment network can be improved.

[0086] like Figure 1 As shown, Figure 1 This is a schematic flowchart illustrating an image quality assessment method according to an embodiment of the present invention.

[0087] The method does not limit the execution entity; it can be either a server or a client.

[0088] The method process may include the following steps.

[0089] S101: Obtain the image to be evaluated.

[0090] S102: Input the image to be evaluated into the image quality evaluation network to obtain the image quality evaluation result.

[0091] In this process, an image quality assessment network can be trained in advance based on image samples and corresponding quality assessment labels.

[0092] Optionally, the image quality assessment network can be used to: extract first image features from the input image, perform a shift operation on the first image features to obtain one or more second image features, and combine the first image features and the obtained second image features to determine the image quality assessment result.

[0093] The second image feature has the same size as the first image feature, and there are regions where the feature values ​​are the same in different locations between the first image feature and the second image feature.

[0094] This method can automatically assess the quality of input images by utilizing a pre-trained image quality assessment network, thereby improving the efficiency of quality assessment and reducing manual costs.

[0095] Furthermore, in the above method, the shifting result of the first image feature can be used for quality assessment, thereby improving the comprehensiveness and accuracy of image quality assessment.

[0096] Since the new image obtained by shifting the original image has the same or similar quality as the original image, multiple new images with the same or similar quality as the original image can be obtained through shifting operations. These new images and the original image can then be used for quality assessment from multiple different perspectives, improving the comprehensiveness and accuracy of the quality assessment.

[0097] It should be noted that, in one optional embodiment, since the video contains several video frame images, the video quality can also be reflected by the quality of the several video frame images.

[0098] Optionally, several video frames in the video can be evaluated for quality using the above-described method, and then the quality evaluation results of the several video frames can be combined to determine the quality evaluation result of the video.

[0099] Specifically, for the entire video, the quality assessment results obtained from each frame can be averaged to obtain the final quality assessment result for the entire video.

[0100] The above method and process will be explained in detail below.

[0101] I. About Image Quality Assessment Networks

[0102] This method does not limit the specific structure or algorithm of the image quality assessment network. Optionally, the image quality assessment network can be an image convolutional network or a deep learning model.

[0103] This method does not limit the training method of the image quality assessment network.

[0104] Optionally, an image quality assessment network can be trained using image samples and corresponding quality assessment labels.

[0105] This embodiment does not limit the specific form of the quality assessment labels corresponding to the image samples.

[0106] Optionally, the quality assessment label may take the form of a rating or quality grade, etc.

[0107] Optionally, the quality assessment label may include the results of quality assessments from multiple different perspectives.

[0108] Alternatively, the quality assessment label can be determined by combining the results of quality assessments from multiple different perspectives. Specifically, it can be a final quality score determined by combining quality scores from multiple different perspectives.

[0109] Since image attributes have a significant impact on image quality, quality assessment labels can specifically include at least one of the following aspects: image blurriness, compression blockiness, noise level, brightness, contrast, color richness, and exposure. Specifically, quality assessment labels can include quality scores or quality levels from multiple perspectives.

[0110] In this embodiment, the attributes of an image can affect its quality to a certain extent. Therefore, conducting quality assessment from multiple different perspectives can improve the comprehensiveness and accuracy of the quality assessment.

[0111] Optionally, based on the image samples and quality assessment labels mentioned above, the output of the image quality assessment network can include quality assessment results from multiple perspectives, making it easy to see from which angles the image quality is specifically reflected.

[0112] For ease of understanding, such as Figure 2 The diagram shown is a schematic representation of an image quality assessment network provided in an embodiment of the present invention.

[0113] The image quality assessment network can output quality assessment results from seven aspects for a single input image: blur, compression blockiness, noise level, brightness, contrast, color richness, and exposure.

[0114] Optionally, determining the image quality assessment result by combining the first image features and the acquired second image features may include: determining one or more image attribute assessment results by combining the first image features and the acquired second image features; and determining the image quality assessment result based on the determined image attribute assessment results.

[0115] The image attribute evaluation results may include at least one of the following image attribute evaluation results: image blur, compression blockiness, noise level, brightness, contrast, color richness, and exposure.

[0116] Optionally, the determined image quality assessment results may include one or more image attribute assessment results.

[0117] Regarding the annotation of quality assessment labels, this method does not limit the annotation method for the quality assessment labels corresponding to the image samples.

[0118] Specifically, manual labeling can be used to improve the accuracy of quality assessment labels, or the labels can be automatically labeled by machines, or a combination of machine and manual labeling can be used.

[0119] Alternatively, since image quality can be related to a variety of image attributes, such as image blur, compression blockiness, noise level, brightness, contrast, color richness, and exposure, etc.

[0120] The values ​​of these attributes can be directly calculated using formulas. Therefore, the machine can automatically calculate the various attribute values ​​of the image, and then determine the quality assessment label based on the calculated image attribute values.

[0121] Optionally, the quality assessment label is determined based on the calculated image attribute values. This can be done by determining the quality assessment label based on preset quality assessment rules, or by manually reviewing the calculated image attributes and considering factors such as the image's content and the actual display effect to manually determine the quality assessment label.

[0122] Compared to manually examining images and determining quality assessment labels, machine-calculated image attributes allow for more convenient and accurate label determination, improving efficiency, reducing labor costs, and enhancing the accuracy of quality assessment labels.

[0123] This embodiment does not limit the preset quality assessment rules. Optionally, the preset quality assessment rules may include: image quality is negatively correlated with image blur, and image quality is negatively correlated with noise level.

[0124] Optionally, the quality assessment label can be automatically determined based on the image attribute values ​​of the image sample, or it can be manually determined based on the image attribute values ​​of the image sample.

[0125] Therefore, optionally, the training set generation method for the image quality assessment network may include: acquiring unlabeled image samples; calculating the corresponding image attribute values ​​for the acquired unlabeled image samples; and determining the corresponding image quality assessment labels based on the calculation results.

[0126] Optionally, the image quality assessment labels can be determined manually or automatically based on the calculation results.

[0127] This embodiment can automatically calculate the image attribute values ​​of image samples by machine, which helps to label the image quality assessment labels corresponding to the image samples, thereby reducing manual costs, improving training efficiency, and improving the training effect of the image quality assessment network.

[0128] In one specific embodiment, several images can be acquired, and then more image samples can be obtained by adjusting the image attributes.

[0129] Specifically, adjustments can be made to different degrees to attributes of an image, such as brightness, contrast, color saturation, highlights, shadows, and blur, to obtain more image samples.

[0130] When annotating image samples, you can annotate manually, automatically, or a combination of both.

[0131] (1) Manual annotation: This can be done by manually evaluating the image quality. Specifically, it can be done by manually evaluating the quality of each attribute index of the image.

[0132] Specifically, the quality can be assessed manually for each of the seven indicators (blurriness, blockiness, noise, brightness, contrast, color richness, and exposure).

[0133] Of course, the same image sample can be evaluated multiple times by different people to improve the accuracy of the quality assessment.

[0134] (2) Automatic annotation.

[0135] Since image attributes can be calculated using formulas and numerically mapped, automatic annotation based on image quality can be achieved.

[0136] The blur, blockiness, noise, brightness, contrast, color richness, and exposure of an image can be calculated using formulas and numerically mapped to achieve automatic annotation.

[0137] Specifically, the following explains how to use formulas to calculate the various attributes of an image.

[0138] Blur: The grayscale image is obtained using the formulas Y = 0.2126R + 0.7252G + 0.0722B (BT.709 data) and Y = 0.2627R + 0.678G + 0.0593B (BT.2020 data). Then, the edge map e of the grayscale image is obtained using the Laplacian operator, and the formula is used... The blur level of the image is obtained. Here, μ is the mean of e, i,j are the pixel coordinates of the image, and N is the sum of the number of all pixels.

[0139] For videos that meet the standard, the calculated blur value is mapped to 1, and the original video value is mapped to 0. For other videos, the blur value is automatically calculated, and the resulting values ​​are used to obtain the corresponding labeled values ​​using the mapping relationship. For example, if the standard video's blur calculation result is S1 (mapped to 1), and the original video's blur calculation result is S2 (mapped to 0), then the mapping formula is: Where X is the calculated ambiguity and Y is the mapped ambiguity label value.

[0140] Blockiness: First, convert the RGB image to grayscale. Then, divide the grayscale image into LxL blocks, where L is 8 by default. Calculate the absolute sum of the differences between the last column and the second-to-last column of each block (VIS), the absolute sum of the differences between the last column and the first column of the next block (VOS), the absolute sum of the differences between the last row and the second-to-last row of each block (HIS), and the absolute sum of the differences between the last row of each block and the first row of the next block (HOS). The final blockiness calculation formula is: The mapping relationship is the same as above.

[0141] Noise: First, convert the RGB image to grayscale, then apply guided filtering to the grayscale image to obtain the filtered image. Calculate the difference between the original image and the filtered image to obtain the residual image c. Then, use the formula... The noise level of the image is obtained. The mapping relationship is the same as above.

[0142] Brightness: Y = 0.2126R + 0.7252G + 0.0722B (BT.709 data), Y = 0.2627R + 0.678G + 0.0593B (BT.2020 data), and the mean value of Y is calculated to obtain the brightness level. The mapping relationship is the same as above.

[0143] Contrast: First, convert the RGB image to grayscale, using the formula... The contrast information of the image is obtained, where M and N are the width and height dimensions of x, and i and j are the coordinate positions of the pixels. The mapping relationship is the same as above.

[0144] Color richness: Calculated using the following formula:

[0145] RG = |RG|

[0146] YB = |0.5*(R+G)-B|

[0147]

[0148]

[0149] color = stdR + 0.3 * meanR

[0150] Here, color represents the color richness of the image, and the mapping relationship is the same as above.

[0151] Exposure: First, obtain the grayscale image Y, calculate its mean Y, and then normalize it to obtain the value. `bit` represents the bit depth of the image. 8-bit video is divided by 255, and 10-bit data is divided by 1023. The exposure calculation formula is... The mapping relationship is the same as above.

[0152] (3) Combining automatic annotation with manual annotation.

[0153] For multiple image samples obtained by adjusting the same image, image attribute values ​​can be calculated using formulas and directly mapped to image quality assessment results.

[0154] However, it is difficult to directly use image attribute values ​​as the result of image quality assessment for images with different content.

[0155] For example, image 1 was taken at night, and image 2 was taken during the day. Image 2 is brighter than image 1. However, the quality of image 1 may be greater than that of image 2.

[0156] Therefore, the image quality assessment result can be determined by comprehensively considering the multiple calculated image attribute values.

[0157] In addition, multiple calculated image attribute values ​​can be provided to humans, allowing them to comprehensively determine the image quality assessment results based on these values, thereby improving the accuracy of image quality assessment.

[0158] The image quality annotation results obtained by the above method can be used to train an image quality assessment network.

[0159] The structure of the image quality assessment network is not specifically limited in this method. Details will be explained later.

[0160] II. Regarding the first image feature.

[0161] This method does not limit the specific content of the first image features.

[0162] Optionally, the first image feature may include at least one of the following: image noise features, image detail features, image global features, etc.

[0163] In this embodiment, the accuracy of image quality assessment can be improved by using one or more image features included in the first image features.

[0164] This method does not limit the way the first image features are extracted.

[0165] Alternatively, feature extraction can be performed directly on the input image. Specifically, a convolutional network can be used.

[0166] Since the first image features can include one or more feature information, different feature extraction networks can be used to extract features from the input image, and then the extracted features can be added to the first image features.

[0167] To facilitate the extraction of specific feature information, alternatively, the corresponding features in the image can be highlighted first, and then the corresponding feature information can be extracted.

[0168] For example, images can be processed to enhance edge information and further extract edge features, thereby improving the accuracy of image quality assessment.

[0169] Therefore, optionally, extracting first image features from the input image may include: obtaining preset information in the input image and further extracting image features based on the preset information; and adding the extracted image features to the first image features.

[0170] Optionally, the preset information may include at least one of the following: original image, image detail information, image noise information, image brightness information, image saturation information, and image tone information. The image detail information may include image edge information.

[0171] Image detail information can be used to highlight image details, facilitating subsequent extraction of image detail features. Image noise information can be used to highlight image noise, facilitating subsequent extraction of image noise features. Furthermore, the original image, image brightness information, image saturation information, and image hue information all belong to the global features of the image; therefore, they can be used to highlight the global features of the image from multiple perspectives, facilitating subsequent extraction of global image features.

[0172] Optionally, in order to improve the image feature extraction effect, different feature extraction networks can be used for feature extraction for different preset information, thereby reducing the influence between different image feature extraction processes during training and use.

[0173] Optionally, further extracting image features based on preset information may include: extracting image features using different feature extraction networks for different preset information.

[0174] Regarding image detail information, this method does not limit the acquisition method. Optionally, edge filtering can be used to extract image detail information.

[0175] Therefore, optionally, obtaining preset information from the input image may include: performing edge filtering on the input image to extract image detail information.

[0176] In one specific embodiment, edge filtering can be performed on the input image to extract edge detail information, which is beneficial for better judging the blurriness / sharpness of the image.

[0177] Edge filtering can be performed using Sobel-x, Sobel-y, and Laplacian operators to obtain the edge image. For example... Figure 3 The diagram shown is a flowchart of an edge filtering feature extraction method provided in an embodiment of the present invention.

[0178] The Sobel-x filter kernel can be: Sobel-y's filter kernel can be: The Laplacian filter kernel can be:

[0179] Regarding image noise information, this method does not limit the acquisition method. Optionally, guided filtering can be used to extract image noise information.

[0180] Therefore, optionally, obtaining preset information from the input image may include: performing guided filtering on the input image to obtain a denoised blurred image, and combining the input image and the blurred image to extract image noise information.

[0181] In one specific embodiment, guided filtering can be performed on the input image to obtain a denoised blurred image. By subtracting the input image from the denoised image, noise information can be extracted, which is beneficial for better judging the noise level of the image.

[0182] like Figure 4 The diagram shown is a flowchart illustrating a guided filtering feature extraction method according to an embodiment of the present invention.

[0183] Specifically, the input image can be subjected to guided filtering to obtain a denoised image, and then the input image can be subtracted from the denoised image to obtain noise information.

[0184] The guided filtering is a linearly variable filtering process, comprising a guide image I, an input image p, and an output image q. The guide image I needs to be pre-defined based on the specific application, or it can be directly taken as the input image p. The guide image I and the input image p can be the same image. For the i-th pixel in the output image, its calculation method can be expressed as:

[0185] In this formula, i and j are pixel labels, and W ij The filter kernel function is defined as follows:

[0186]

[0187] In this formula, w k Let |w| be the number of pixels within the k-th kernel window, and μ be the number of pixels within the window. k and ε represents the mean and variance of the guiding image I within the window, and ε is the smoothing factor.

[0188] The above formula can be used to obtain the smoothed image after guided filtering. The noise information can be obtained by subtracting the smoothed image from the input image.

[0189] This method does not limit the specific structure of the feature extraction network.

[0190] Different feature extraction networks can be used to extract image detail features and image noise features respectively, which can then be used as part of the first image features.

[0191] Optionally, a detail extraction network can be used to extract detailed information from an image, which helps determine the image's blurriness; a noise extraction network can be used to extract noise information from an image, which helps determine the image's noise level. Both the detail extraction network and the noise extraction network can use the same model architecture.

[0192] like Figure 5 The diagram shown is a schematic representation of a feature extraction network provided in an embodiment of the present invention.

[0193] Feature extraction networks can include cascaded convolutional layers, activation function layers, normalization layers, and multiple residual networks.

[0194] Specifically, the activation function layer can be a ReLU layer, and the normalization layer can be an instance normalization (IN) layer. ResNet is a residual network, and the ellipsis can include multiple omitted ResNet networks.

[0195] Optionally, a feature extraction network used to extract global image features, or other feature extraction networks used to extract image features, can also be used. Figure 5 The network structure in [the context].

[0196] For global image features, this method does not limit the form or extraction method of global image features.

[0197] Optionally, at least one of the following can be obtained from the input image: the original image, image brightness information, image saturation information, and image hue information. The obtained information is then used to further extract global image features, which are then added to the first image features.

[0198] Optionally, the original image, image brightness information, image saturation information, and image hue information can be combined to facilitate subsequent extraction of global image features.

[0199] Optionally, since the image brightness information, image saturation information, and image hue information are all extracted from the original image, in order to facilitate the combination of features, features can be extracted from the original image first, and then the extracted features can be combined with the image brightness information, image saturation information, and image hue information to facilitate the subsequent extraction of global image features.

[0200] In one specific embodiment, a global feature extraction network can be used to extract global features of an image.

[0201] like Figure 6 The diagram shown is a schematic representation of the principle of an image global feature extraction network provided in an embodiment of the present invention.

[0202] For an image input to a global feature extraction network, the image brightness, saturation, and hue information can be extracted first.

[0203] Where L represents the brightness information of the image, S represents the saturation information of the image, and H represents the hue information of the image.

[0204] The calculation formula is as follows (input image is RGB):

[0205] (1) Brightness information:

[0206] L = 0.2126R + 0.7252G + 0.0722B (BT.709 data)

[0207] L = 0.2627R + 0.678G + 0.0593B (BT.2020 data)

[0208] (2) Saturation information:

[0209] S = max(R,G,B) – min(R,G,B)

[0210] (3) Hue information:

[0211] H = R – 0.5G – 0.5B

[0212] L, S, and H are obtained through the above formulas, and features can be extracted from the input image through convolutional layers. The extracted features are then combined with L, S, and H for subsequent extraction of global image features.

[0213] Specifically, this can be achieved by extracting global features of an image through the structure of a feature extraction network.

[0214] Since brightness, saturation, and hue information are global features of an image, incorporating them into a global feature extraction network can help the network learn the global features of the image more effectively.

[0215] In an optional embodiment, given the image detail features, image noise features, and image global features of the input image, the image detail features, image noise features, and image global features can be stacked and combined to obtain the first image feature.

[0216] For ease of understanding, such as Figure 7 The diagram shown is a schematic representation of the principle of a first image feature extraction method provided in an embodiment of the present invention.

[0217] Specifically, for the input image, image noise information, image detail information, image brightness information, image saturation information, and image tone information can be obtained respectively.

[0218] Then, based on the image noise information, a noise feature extraction network is used to extract the image noise features.

[0219] To extract image detail information, a detail feature extraction network is used to extract image detail features.

[0220] Global feature extraction networks are used to extract global features of an image, taking into account its brightness, saturation, and hue information.

[0221] Finally, the image noise features, image detail features, and image global features are stacked and combined to obtain the first image feature.

[0222] III. Second Image Features.

[0223] In one alternative embodiment, the second image feature may be obtained by shifting the first image feature.

[0224] First, based on the first image feature, multiple features of the input image can be used for quality assessment, such as image detail features and image noise features, thereby improving the accuracy of image quality assessment.

[0225] Adding a second image feature to the first image feature for quality assessment can also improve the accuracy of image quality assessment.

[0226] Specifically, since the second image feature is obtained by shifting the first image feature, the second image feature may include some features from the first image feature, but the positions are not the same.

[0227] Therefore, the second image feature can be the first image feature at different positions and angles. Adding a second image feature for quality assessment can determine image features from multiple angles, thereby improving the comprehensiveness and accuracy of image quality assessment.

[0228] For example, when evaluating the image quality of the original image, shifting it down by 3 rows of pixels or to the right by 3 columns of pixels has little impact on image quality. The shifted image has the same or similar quality as the original image, thus allowing for image quality evaluation from multiple perspectives, improving the comprehensiveness and accuracy of image quality assessment.

[0229] Similarly, relative to the extracted first image features, one or more second image features obtained through shifting operations can be first image features at different positions and angles. The evaluated image quality is the same or similar, thus allowing image quality to be evaluated from multiple positions and angles, improving the comprehensiveness and accuracy of image quality evaluation.

[0230] Optionally, the second image feature can have the same size as the first image feature, which facilitates subsequent stacking and combination to determine the image quality assessment result and clarifies the positional relationship of the values ​​of each feature in the image feature.

[0231] Optionally, the feature values ​​may be the same in regions at different locations between the first image feature and the second image feature.

[0232] In this context, the region where the feature values ​​are the same between the first image feature and the second image feature can be the region retained by shifting the first image feature. Therefore, the position of the feature value in this region in the first image feature is different from its position in the second image feature after shifting, thus providing first image features (i.e., second image features) at different positions and angles, which facilitates the improvement of the comprehensiveness and accuracy of subsequent image quality assessment.

[0233] For shifting operations, this method does not limit the specific operation method, nor does it limit the direction or distance of the shift operation, as long as the feature values ​​are the same in different regions between the first image feature and the second image feature.

[0234] Optionally, the direction of the shift operation can be leftward, rightward, upper leftward, lower rightward, etc.

[0235] Optionally, the shift distance of the shift operation can be either moving one feature point or moving three feature points.

[0236] Optionally, in order to improve the comprehensiveness and accuracy of image quality assessment, the shift distance of the shift operation can be limited, thereby reducing the impact of the shift operation on image quality.

[0237] Optionally, the shift distance of the shift operation can be limited to less than a preset distance. Specifically, it can be less than 3 feature points, so that the acquired second image features can retain most of the feature values ​​in the first image features, thereby making the image quality evaluated for the second image features the same or similar to the image quality evaluated for the first image features.

[0238] Optionally, different shift operations can be performed on the first image features to obtain different second image features.

[0239] The specific shift operations can differ in either the shift direction or the shift distance.

[0240] For example, a second image feature can be obtained by moving one feature point to the left of the first image feature; a second image feature can be obtained by moving three feature points to the left of the first image feature; and a second image feature can be obtained by moving two feature points to the right of the first image feature.

[0241] This embodiment does not limit the number of second image features acquired.

[0242] Optionally, as the number of second image features increases, first image features from multiple position angles can be obtained, thereby improving the comprehensiveness and accuracy of image quality assessment.

[0243] The following explains the specific shifting operation.

[0244] Since the shift operation is performed on the first image features, the shift result may include some feature values ​​from the first image features, and the positions of these feature values ​​in the shift result are different from their positions in the first image features.

[0245] Furthermore, the shift result can also include default values ​​generated by the shift operation. Specifically, after shifting some feature values ​​in the first image features, the shift result may contain some feature values ​​that are shifted, thus resulting in default values.

[0246] For ease of understanding, such as Figure 8 The diagram shown is a schematic representation of the principle of image feature shifting provided in an embodiment of the present invention.

[0247] The first image feature can be a white rectangle, while the gray rectangle can be the position of the first image feature after it has been shifted.

[0248] The first image feature can be shifted by 3 feature points in 8 directions, specifically up, down, left, right, and top left, bottom left, top right, and bottom right. It is evident that after shifting, the result contains missing parts.

[0249] The gray area inside the white rectangle represents a portion of the feature values ​​of the first image feature in the shift result. The white area inside the white rectangle represents the default part of the shift result.

[0250] This method does not limit the processing of these default parts.

[0251] Optionally, the shift result containing the default portion can be directly used as the second image feature. This is because when using the second image feature, the focus should be on determining the image quality at different positions and angles based on the shifted feature values ​​of the first image feature within the second image feature.

[0252] Optionally, a preset value can be used to fill the missing portion of the shift result as a second image feature, thereby facilitating subsequent calculations. Specifically, the preset value can be 0.

[0253] Alternatively, image inpainting algorithms or other algorithms can be used to predict the missing parts in the shift result, serving as a second image feature. Since the inpainted image features are of the same or similar quality to the original image features, the comprehensiveness and accuracy of image quality assessment can be improved.

[0254] In one optional embodiment, performing a shift operation on a first image feature to obtain one or more second image features may include: performing a shift operation on features in a preset region of the first image feature to obtain one or more shift results; filling the default part of each shift result with a preset value to obtain a second image feature with the same size as the first image feature.

[0255] Different shift operations can correspond to different shift operations.

[0256] IV. Based on the combined features of the first and second images, determine the image quality assessment result.

[0257] The method described herein does not limit the specific form of the image quality assessment results. For a detailed explanation, please refer to the embodiments described above.

[0258] Optionally, the image quality assessment result may include one or more quality assessment results for image attributes, or may be determined based on one or more quality assessment results for image attributes.

[0259] For example, image quality assessment results can include quality assessment results from seven aspects: image blur, compression blockiness, noise level, brightness, contrast, color richness, and exposure.

[0260] Optionally, determining the image quality assessment result by combining the first image features and the acquired second image features may include: determining one or more image attribute assessment results by combining the first image features and the acquired second image features; and determining the image quality assessment result based on the determined image attribute assessment results.

[0261] In this embodiment, quality assessment can be performed on image attributes, thereby facilitating the analysis of the correlation between image quality and image attributes.

[0262] Furthermore, quality assessment can be performed from the perspective of multiple image attributes, which can improve the comprehensiveness and accuracy of quality assessment.

[0263] This method does not limit the way image quality assessment results are determined.

[0264] Alternatively, the image quality assessment result can be predicted directly based on the first and second image features. Specifically, the prediction can be performed using a fully connected network or a regression network.

[0265] Optionally, further image features can be extracted based on the first image features and the acquired second image features, and then the image quality assessment result can be predicted based on the extracted image features.

[0266] Optionally, spatial feature information can be further extracted by combining the first image features and the first image features from multiple different positions and angles, so that the spatial feature information of the image can be used to predict the image quality assessment result.

[0267] Specifically, the spatial feature information can be features extracted from the first image features at multiple different positions and angles. By introducing different positions and angles, spatial information of the image can be learned through feature extraction during model training.

[0268] Specifically, the model can be trained by using image features from multiple different positions and angles as input, along with the output image quality.

[0269] Since the image quality corresponding to multiple image features from different positions and angles is the same or similar, and the content contained in these image features is the same or similar, the feature representation ability can be further improved by performing feature extraction on image features from multiple different positions and angles during the model training process. This can better represent the correlation between image features and image quality, and improve the stability and efficiency of model training.

[0270] Therefore, the spatial image features extracted from the first image features at multiple different positions and angles can better characterize the correlation between the first image features and image quality, thereby improving the accuracy of image quality assessment.

[0271] Therefore, optionally, determining the image quality assessment result by combining the first image features and the acquired second image features may include: further extracting...

[0272] The spatial feature information of the image is used to determine the image quality assessment result.

[0273] In one alternative embodiment, for the first image feature, the first image feature at different position angles can be obtained through a shift operation.

[0274] Optionally, the image quality assessment result can be determined directly based on the first image features and the first image features acquired at different positions and angles.

[0275] Alternatively, features can be further extracted from the first image features and the first image features acquired at different positions and angles, and the same steps can be performed serially and continuously multiple times for the extracted features, thereby improving the feature extraction effect.

[0276] Optionally, the image quality evaluation network includes N cascaded preset modules; N≥2; for the i-th preset module, 1≤i≤N-1, its output is cascaded to the input of the (i+1)-th preset module.

[0277] Optionally, the preset module can be used to perform a shift operation on the input image features to obtain one or more third image features, and combine the input image features and the obtained third image features to further extract and output image features; the third image features have the same size as the input image features, and there are regions where the feature values ​​are the same between the input image features and the third image features.

[0278] The form of the image features extracted and output by the preset module is not limited; in particular, it can be spatial feature information.

[0279] Optionally, in the preset module, the input image features need to be expanded to obtain one or more third image features; and there are multiple cascaded preset modules. In order to reduce the calculation columns of the preset modules, the size of the image features extracted and output by the preset modules can be limited.

[0280] Specifically, the input image features of the preset module can be limited to the same size as the output image features.

[0281] Optionally, the size of the image features can be adjusted by normalization, specifically by batch normalization (BN), instance normalization (IN), or feature normalization (FN).

[0282] Because the preset module directly performs a shift operation on the input image features.

[0283] Therefore, optionally, performing a shift operation on the first image feature to obtain one or more second image features, and combining the first image feature and the obtained second image feature to determine the image quality assessment result, may include: inputting the first image feature into the first preset module of N cascaded preset modules to obtain the image feature output by the Nth preset module; and determining the image quality assessment result based on the image feature output by the Nth preset module.

[0284] In this embodiment, multiple rounds of shifting operations and feature extraction can be performed on the first image features through a series of preset modules, thereby improving the feature extraction effect.

[0285] This embodiment does not specifically limit the structure of the preset module. Optionally, it can be a ShiftNet structure.

[0286] In one specific embodiment, for the first image features, a spatial feature extraction network can be used to integrate image detail information, image noise information, and global image features in the first image features, and make full use of the spatial features of the image to better extract image features.

[0287] like Figure 9 The diagram shown is a schematic representation of a spatial feature extraction network provided in an embodiment of the present invention.

[0288] This can include convolutional layers, activation function layers, normalization layers, multiple ShiftNet layers, and multiple residual networks. The ellipsis may include omitted ShiftNet layers and ResNet networks.

[0289] The ShiftNet layer can perform shift operations and feature extraction on the input image features, and the residual network can further extract features.

[0290] like Figure 10 The diagram shown is a schematic representation of the structure of a ShiftNet layer provided in an embodiment of the present invention.

[0291] This can include shift layers, convolutional layers, activation function layers, and normalization layers.

[0292] Shift layers can be used to shift input image features to obtain image features at multiple different positions and angles.

[0293] Therefore, when the input of the shift layer is an image feature with F channels, the output of the shift layer can be an image feature with F*9 channels, where 8 shift operations can be performed on the image feature, and the shift results are stacked and combined with the input image feature.

[0294] Features can then be further extracted through convolutional layers, activation function layers, and normalization layers. The normalization layer can normalize the image features with F*9 channels to the image features with F channels.

[0295] The main function of ShiftNet is to move feature information by a small number of pixels in different directions, thereby obtaining the continuity of feature information between adjacent rows and columns, which is beneficial for better extraction of spatial feature information of the image.

[0296] The spatial feature information output by the spatial feature extraction network can be further used to predict image quality assessment results using a regression network.

[0297] like Figure 11 The diagram shown is a schematic representation of a regression network provided in an embodiment of the present invention.

[0298] The regression network can include multiple cascaded convolutional layers, activation function layers, pooling layers, and normalization layers, as well as a final pooling layer and a fully connected layer. The ellipsis may include omitted sets of cascaded convolutional layers, activation function layers, pooling layers, and normalization layers.

[0299] In a regression network, the final pooling layer can be a global max pooling layer, while the preceding pooling layers can be specific max pooling layers.

[0300] The role of the regression network is to further output the previously extracted image feature information as image quality assessment results.

[0301] For the structure of the regression network, if the image quality assessment results include quality assessment results from multiple image attribute angles, the number of outputs of the fully connected layers in the regression network can be determined.

[0302] If you need to update the number of image attribute angles in the image quality assessment results, you can directly update the number of outputs of the fully connected layers in the regression network.

[0303] For ease of understanding, such as Figure 12 The diagram shown is a structural schematic of an image quality assessment network provided in an embodiment of the present invention.

[0304] This may include an input layer, an information extraction layer, a detail feature extraction network, a global feature extraction network, a noise feature extraction network, a spatial feature extraction network, and a regression network.

[0305] For an input image, the information extraction layer can extract image noise information, image detail information, image brightness information, image saturation information, and image tone information.

[0306] Then, the detail feature extraction network, global feature extraction network, and noise feature extraction network can be used to extract the image detail features, image global features, and image noise features of the input image, respectively.

[0307] Furthermore, the image detail features, global image features, and image noise features of the input image can be integrated into the spatial feature extraction network, and spatial feature information can be output through shifting operations.

[0308] Then, a regression network can predict the image quality assessment result based on the input spatial feature information.

[0309] Corresponding to the above method embodiments, this invention also provides a corresponding device embodiment.

[0310] like Figure 13 The diagram shown is a structural schematic of an image quality assessment device provided in an embodiment of the present invention.

[0311] It may include the following units.

[0312] Acquisition unit 201 is used to acquire the image to be evaluated.

[0313] Evaluation unit 202 is used to input the image to be evaluated into the image quality evaluation network to obtain the image quality evaluation result.

[0314] The image quality assessment network is used to: extract first image features from an input image, perform a shift operation on the first image features to obtain one or more second image features, and combine the first image features and the obtained second image features to determine the image quality assessment result.

[0315] The second image feature has the same size as the first image feature, and there are regions where the feature values ​​are the same in different locations between the first image feature and the second image feature.

[0316] Optionally, the image quality assessment network is used to: integrate the first image features and the acquired second image features to determine one or more image attribute assessment results; and determine the image quality assessment result based on the determined image attribute assessment results.

[0317] Optionally, the image quality evaluation network includes N cascaded preset modules; N≥2; for the i-th preset module, 1≤i≤N-1, its output is cascaded to the input of the (i+1)-th preset module.

[0318] The preset module is used to perform a shift operation on the input image features to obtain one or more third image features, and to further extract and output image features by combining the input image features and the obtained third image features; the third image features have the same size as the input image features, and there are regions where the feature values ​​are the same between the input image features and the third image features.

[0319] The image quality assessment network is used to: input the first image feature into the first preset module of N cascaded preset modules to obtain the image feature output by the Nth preset module; and determine the image quality assessment result based on the image feature output by the Nth preset module.

[0320] Optionally, the image quality assessment network is used to: perform a shift operation on the features of a preset region in the first image features to obtain one or more shift results; fill the default part in each shift result with a preset value to obtain a second image feature with the same size as the first image feature.

[0321] Optionally, the image quality assessment network is used to: integrate the first image features and the acquired second image features, further extract spatial feature information of the image, and determine the image quality assessment result based on the extracted spatial feature information.

[0322] Optionally, the first image feature includes at least one of the following: image detail features, image noise features, and image global features.

[0323] Optionally, the image quality assessment network is used to: acquire preset information in the input image and further extract image features based on the preset information; add the extracted image features to the first image features; the preset information includes at least one of the following: original image, image detail information, image noise information, image brightness information, image saturation information, and image tone information.

[0324] Optionally, the image quality assessment network is used to: perform edge filtering on the input image to extract image detail information; and / or perform guided filtering on the input image to obtain a denoised blurred image, and combine the input image and the blurred image to extract image noise information.

[0325] Optionally, the image quality assessment network is used to extract image features using different feature extraction networks for different preset information.

[0326] Optionally, the method for generating the training set of the image quality assessment network includes: obtaining unlabeled image samples; calculating the corresponding image attribute values ​​for the unlabeled image samples, and determining the corresponding image quality assessment labels based on the calculation results.

[0327] For an explanation of the above device embodiments, please refer to the above method embodiments.

[0328] This invention also provides a computer device, which includes at least a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above-described method embodiments.

[0329] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above method embodiments.

[0330] Figure 14 This is a schematic diagram of the hardware structure of a computer device configured according to an embodiment of the present invention. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are internally connected to each other via the bus 1050.

[0331] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0332] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of the present invention are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0333] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0334] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0335] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0336] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of the present invention, and not necessarily all the components shown in the figures.

[0337] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described method embodiments.

[0338] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the above-described method embodiments.

[0339] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0340] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0341] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0342] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. In implementing the embodiments of the present invention, the functions of each module can be implemented in one or more software and / or hardware. Alternatively, some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0343] The above description is merely a specific implementation of the embodiments of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the embodiments of the present invention, and these improvements and modifications should also be considered as protection of the embodiments of the present invention.

[0344] In this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0345] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. The invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0346] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. An image quality assessment method, characterized in that, include: Acquire the image to be evaluated; The image to be evaluated is input into the image quality evaluation network to obtain the image quality evaluation result; The image quality assessment network is used to: extract a first image feature from an input image, perform a shift operation on the first image feature to obtain one or more second image features, and combine the first image feature and the obtained second image feature to determine the image quality assessment result; The second image feature has the same size as the first image feature, and there are regions with the same feature value at different locations between the first image feature and the second image feature; The first image feature includes: image detail features, image noise features, and image global features; The step of extracting the first image feature from the input image includes: Obtain preset information from the input image, and further extract image features based on the preset information; Add the extracted image features to the first image features; The preset information includes image detail information, image noise information, and at least one of the following: original image, image brightness information, image saturation information, and image hue information; The global image features are extracted from at least one of the following: the original image, image brightness information, image saturation information, and image hue information. The step of obtaining preset information from the input image includes: Edge filtering is applied to the input image to extract image detail information; The input image is subjected to guided filtering to obtain a denoised blurred image. The input image and the blurred image are combined to extract image noise information. The image quality assessment network comprises N cascaded preset modules; N≥2; for the i-th preset module, 1≤i≤N-1, its output is cascaded to the input of the (i+1)-th preset module; The preset module is used to perform a shift operation on the input image features to obtain one or more third image features, and to further extract and output image features by combining the input image features and the obtained third image features; the third image features have the same size as the input image features, and there are regions where the feature values ​​are the same between the input image features and the third image features.

2. The method according to claim 1, characterized in that, The step of combining the first image features and the acquired second image features to determine the image quality assessment result includes: By combining the first image features and the acquired second image features, one or more image attribute evaluation results are determined; Based on the determined image attribute evaluation results, the image quality evaluation results are determined.

3. The method according to claim 1, characterized in that, The step of performing a shift operation on the first image feature to obtain one or more second image features, and combining the first image feature and the obtained second image features to determine the image quality assessment result includes: The first image feature is input into the first preset module of the N cascaded preset modules to obtain the image feature output by the Nth preset module; The image quality assessment result is determined based on the image features output by the Nth preset module.

4. The method according to claim 1, characterized in that, The step of shifting the first image feature to obtain one or more second image features includes: A shift operation is performed on the features of a preset region in the first image features to obtain one or more shift results; For each shift result, fill the default part with a preset value to obtain a second image feature with the same size as the first image feature.

5. The method according to claim 1, characterized in that, The step of combining the first image features and the acquired second image features to determine the image quality assessment result includes: By combining the first image features and the acquired second image features, spatial feature information of the image is further extracted, and the image quality assessment result is determined based on the extracted spatial feature information.

6. The method according to claim 1, characterized in that, The step of further extracting image features based on the preset information includes: Different feature extraction networks are used to extract image features for different preset information.

7. The method according to claim 1, characterized in that, The method for generating the training set of the image quality assessment network includes: Obtain unlabeled image samples; For the unlabeled image samples, calculate the corresponding image attribute values ​​and determine the corresponding image quality assessment labels based on the calculation results.

8. An image quality assessment device, characterized in that, include: The acquisition unit is used to acquire the image to be evaluated; The evaluation unit is used to input the image to be evaluated into the image quality evaluation network to obtain the image quality evaluation result; The image quality assessment network is used to: extract a first image feature from an input image, perform a shift operation on the first image feature to obtain one or more second image features, and combine the first image feature and the obtained second image feature to determine the image quality assessment result; The second image feature has the same size as the first image feature, and there are regions with the same feature value at different locations between the first image feature and the second image feature; The first image feature includes: image detail features, image noise features, and image global features; The image quality assessment network is used to: acquire preset information in the input image, and further extract image features based on the preset information; add the extracted image features to the first image features; the preset information includes image detail information, image noise information, and at least one of the following: original image, image brightness information, image saturation information, and image hue information; The global image features are extracted from at least one of the following: the original image, image brightness information, image saturation information, and image hue information. The image quality assessment network is used for: Edge filtering is applied to the input image to extract image detail information; The input image is subjected to guided filtering to obtain a denoised blurred image. The input image and the blurred image are combined to extract image noise information. The image quality assessment network comprises N cascaded preset modules; N≥2; for the i-th preset module, 1≤i≤N-1, its output is cascaded to the input of the (i+1)-th preset module; The preset module is used to perform a shift operation on the input image features to obtain one or more third image features, and to further extract and output image features by combining the input image features and the obtained third image features; the third image features have the same size as the input image features, and there are regions where the feature values ​​are the same between the input image features and the third image features.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the method of any one of claims 1 to 7.

Images