Image quality evaluation method and device, equipment, storage medium and product

Abstract

The invention discloses an image quality evaluation method and device, equipment, a storage medium and a product. The method comprises the following steps: acquiring a reference feature of a reference image and a distortion feature of a distorted image; inputting the reference feature into a degradation network comprising T degradation units to obtain M first degradation features; wherein T > = M; based on the distortion feature and the M first degradation features, obtaining an aggregated degradation feature; wherein the aggregated degradation features are simulation of distortion features; and inputting the aggregated degradation features and reference features into a regression network to obtain a quality evaluation result of the distorted image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image quality assessment method and apparatus, device, storage medium, and product. Background Art

[0002] In recent years, digital images have been widely used in our daily lives and various professional fields. The scan function on mobile phones allows us to quickly scan and identify QR codes, text, charts and even documents. In actual applications, different scanning environments and different types of devices have different scanned image quality. After a series of processing steps such as rendering, compression and transmission, the loss of image quality is inevitable, which seriously affects the user's visual experience. Image quality has a direct impact on the subsequent artificial intelligence function recognition performance. Therefore, it is particularly important to evaluate the quality of images. Summary of the invention

[0003] The present application provides an image quality assessment method and apparatus, device, storage medium, and product.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, a method for evaluating image quality is provided, the method comprising:

[0006] Obtaining reference features of the reference image and distortion features of the distorted image;

[0007] Inputting the reference feature into a degradation network including T degradation units to obtain M first degradation features; wherein T≥M;

[0008] Based on the distortion feature and the M first degradation features, an aggregated degradation feature is obtained; wherein the aggregated degradation feature is a simulation of the distortion feature;

[0009] The aggregated degradation features and the reference features are input into a regression network to obtain a quality assessment result of the distorted image.

[0010] In a second aspect, an image quality assessment device is provided, the device comprising:

[0011] An acquisition module, used for acquiring reference features of a reference image and distortion features of a distorted image;

[0012] A degradation network including T degradation units, used to process the reference features to obtain M first degradation features; wherein T≥M;

[0013] A processing module, configured to obtain an aggregated degradation feature based on the distortion feature and the M first degradation features; wherein the aggregated degradation feature is a simulation of the distortion feature;

[0014] The regression network is used to process the aggregated degradation features and the reference features to obtain a quality assessment result of the distorted image.

[0015] According to a third aspect, an electronic device is provided, comprising: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the method of the first aspect when running the computer program.

[0016] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program implements the steps of the method according to the first aspect when executed by a processor.

[0017] According to a fifth aspect, a computer program product is provided, comprising a computer program, wherein the computer program implements the steps of the method according to the first aspect when executed by a processor.

[0018] The present application provides an image quality assessment method and apparatus, device, storage medium, and product, which obtain reference features of a reference image and distortion features of a distorted image; input the reference features into a degradation network including T degradation units to obtain M first degradation features; wherein T≥M; based on the distortion features and the M first degradation features, obtain aggregated degradation features; wherein the aggregated degradation features are a simulation of the distortion features; input the aggregated degradation features and the reference features into a regression network to obtain a quality assessment result of the distorted image. In this way, the degradation features of the reference image in the layer-by-layer degradation process are learned through the degradation network, and then combined with the distortion features, so that the aggregated degradation features obtained are closer to the degradation expression of the distorted image, and then the aggregated degradation features and the reference features are processed through the regression network to obtain the quality assessment result of the distorted image. This method of using the reference image as an evaluation standard can more accurately capture subtle changes in image quality, thereby improving the accuracy of the image quality assessment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 1 ;

[0020] Figure 2 This is a schematic diagram of the network structure of the degenerate network in an embodiment of the present application;

[0021] Figure 3 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 2 ;

[0022] Figure 4 This is a schematic diagram of the network structure of the conversion network in the embodiment of the present application;

[0023] Figure 5 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 5 ;

[0024] Figure 6 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 6 ;

[0025] Figure 7 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 7 ;

[0026] Figure 8 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 8 ;

[0027] Fig. 9 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 9 ;

[0028] Fig.10 This is a flowchart of the image quality assessment process in an embodiment of the present application;

[0029] Fig.11 This is a schematic diagram of a similarity matrix in an embodiment of the present application;

[0030] Fig.12 This is a schematic diagram of the structure of the image quality assessment device in an embodiment of the present application;

[0031] Fig.13 It is a schematic diagram of the structure of the electronic device in the embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this embodiment and are not intended to limit this application.

[0034] In the following description, references to “some embodiments,” “this embodiment,” “this embodiment,” and examples, etc., describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0035] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the present embodiment described here can be implemented in an order other than that illustrated or described here.

[0036] In this embodiment, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, object A and / or object B may represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0037] Image Quality Assessment (IQA) is an important task to evaluate image quality in simulating human subjective perception. It plays a vital role in the application field of digital images and can detect and identify problems such as distortion, blur and noise in images. Only by identifying and detecting these problems can targeted measures be taken to improve image quality, thereby further improving the accuracy and success rate of scanning.

[0038] The present application provides an image quality assessment method. Figure 1 The process diagram of the image quality assessment method in the embodiment of the present application is as follows: Figure 1 ,like Figure 1 As shown, the image quality assessment method includes the following steps:

[0039] S101: Acquire reference features of a reference image and distortion features of a distorted image.

[0040] In the embodiments of the present application, the reference image can be understood as a non-distorted image. A distorted image refers to an image that deviates from the reference image due to various factors that cause the image quality to deteriorate. These deviations may include color distortion, reduced contrast, reduced clarity, blur, increased noise, geometric distortion, etc. Distorted images may be caused by a variety of reasons, such as equipment problems during shooting (such as lens stains, improper focus adjustment), compression loss during image transmission, improper application of image processing algorithms, etc.

[0041] The image in the embodiment of the present application may be a visual QR code image.

[0042] In some embodiments, S101 includes the following steps: inputting the reference image into a feature extraction network to obtain the reference feature; inputting the distorted image into the feature extraction network to obtain the distorted feature.

[0043] In the embodiment of the present application, a feature extraction network is used to extract features from a reference image to obtain reference features. And a feature extraction network is used to extract features from a distorted image to obtain distorted features. Here, when extracting features from the reference image and the distorted image, the same feature extraction network is used, so the feature dimension (or dimensionality) of the reference feature is the same as the feature dimension of the distorted feature.

[0044] Exemplarily, the feature extraction network can be ResNet50 and VGG16.

[0045] S102: Inputting the reference feature into a degradation network including T degradation units to obtain M first degradation features; wherein T≥M.

[0046] In the embodiment of the present application, the degeneration network (DNet) is composed of T identical degeneration units, each of which includes multiple linear layers, an activation (ReLU) layer and a dropout layer (such as Figure 2 As shown in Figure 1). T degradation units are connected through residuals, and the features are input to a degradation unit for processing and output, which is then used as the input of the next degradation unit. Residual connections enable the degradation network to better learn and propagate features, achieve effective parameter sharing, and enhance the network's expressiveness. The degradation network can extract useful features from the reference image and encode them into corresponding degradation features, which can more accurately simulate the image degradation process and learn key features related to distortion. The degradation network provides more comprehensive and accurate information for IQA, and provides a new vision for improving and optimizing image restoration models.

[0047] In the embodiment of the present application, a reference feature is input into a degradation network including T degradation units, and first degradation features output by M degradation units are obtained, that is, M first degradation features are obtained.

[0048] Exemplarily, the T degenerate units refer to the first degenerate unit to the Tth degenerate unit, and the M degenerate units may be the TMth degenerate unit to the Tth degenerate unit, or may be the middle M degenerate units, etc. No specific limitation is given here.

[0049] S103: Obtaining aggregated degradation features based on the distortion features and the M first degradation features; wherein the aggregated degradation features are a simulation of the distortion features.

[0050] In the embodiment of the present application, a first degradation feature with a high similarity to the distortion feature is selected from the M first degradation features, and then all the selected first degradation features are aggregated to obtain an aggregated degradation feature. Therefore, the aggregated degradation feature can be understood as a simulation of the distortion feature, or in other words, the aggregated degradation feature is very similar to the distortion feature and can be considered to be the same feature.

[0051] S104: Inputting the aggregated degradation features and reference features into a regression network to obtain a quality assessment result of the distorted image.

[0052] In the embodiment of the present application, the aggregated degradation features can be understood as the distortion features of the simulated distorted image of the reference image. After the aggregated degradation features and the reference features are processed by the regression network, the quality assessment result of the simulated distorted image of the reference image is obtained. Here, the quality assessment result of the simulated distorted image is used as the quality assessment result of the real distorted image.

[0053] In the embodiment of the present application, the degradation characteristics of the reference image in the layer-by-layer degradation process are learned through the degradation network, and then combined with the distortion characteristics, so that the obtained aggregated degradation characteristics are closer to the degradation expression of the distorted image, and then the aggregated degradation characteristics and reference characteristics are processed through the regression network to obtain the quality evaluation result of the distorted image. This use of the reference image as the evaluation standard can more accurately capture the subtle changes in image quality, thereby improving the accuracy of the image quality evaluation result.

[0054] In some embodiments, S103 may include the following steps S301 to S304:

[0055] S301: Inputting M first degenerate features into a conversion network to obtain M first mapping features.

[0056] In the embodiment of the present application, the conversion network consists of I linear layers (such as Figure 4 As shown). I can be 8. The M first degradation networks are input into the conversion network, and are linearly transformed by weighted and biased operations through I linear layers to obtain the transformed new features, i.e., the M first mapping features.

[0057] It should be noted that since the structure of each mapping space (i.e., linear layer) in the conversion network is the same, they share the same parameters and learning capabilities. Therefore, each mapping space can learn a specific feature representation, allowing different perspectives to complete the expression of feature information. This feature shows higher flexibility and adaptability in capturing various aspects of the input features. Therefore, the conversion network not only performs well in IQA, but also shows certain applicability and versatility for video quality assessment (Video Quality Assessment, VQA).

[0058] S302: Determine the similarity between the distortion feature and the M first mapping features respectively.

[0059] It should be noted that the mechanism of image degradation usually exhibits a high degree of complexity and diversity. In order to accurately evaluate the degree of image degradation, a similarity matrix between distortion features and mapping features is introduced to quantify their correlation. By calculating this similarity matrix, we can identify and quantify subtle changes between different feature dimensions. This method not only reveals the relationship between mapping features and distortion features, but also captures multi-level and multi-dimensional feature information during the degradation process. To achieve this, cosine similarity is used as a comparison metric. The M first mapping features and distortion features are input into the similarity module to calculate the similarity between them. The calculation formula (1) of cosine similarity is:

[0060]

[0061] Where 0≤S m ≤1, S m is the similarity, f m is the first mapping feature, m is greater than or equal to 1 and less than or equal to M, d i is the distortion feature. When there are N distorted images, i is greater than or equal to 1 and less than or equal to N. ⊙ and ‖·‖ represent the dot product and norm, respectively.

[0062] S303: Based on the M similarities, select K second mapping features from the M first mapping features.

[0063] It should be noted that the higher the similarity, the closer the image quality between the simulated distorted image of the reference image and the real distorted image. When a higher similarity is obtained, the mapping features are not only very similar to the features of the real distorted image, but also can accurately capture the underlying content of the real distorted image. On this basis, the K similarities with the highest similarity are selected from the M similarities, and the first mapping features corresponding to the K similarities are used as the second mapping features to promote a high level of matching between the mapping features and the distortion features. This approach helps us capture these complex degradation patterns more comprehensively, thereby enhancing our understanding and simulation capabilities of the image degradation process.

[0064] S304: Based on the similarities corresponding to the K second mapping features, aggregate the K second mapping features to obtain aggregated degenerate features.

[0065] In some embodiments, S303 may include the following steps S501 to S502:

[0066] S501: Sort the M similarities in descending order to obtain a sorting result.

[0067] In the embodiment of the present application, a ranking function is used to sort the M similarities in descending order, and the ranking of each similarity in the M similarities is calculated to obtain a ranking result.

[0068] S502: The first mapping features corresponding to the first K similarities in the sorting results are used as K second mapping features.

[0069] Here, the top K similarities with the highest similarities are screened out and subsequently aggregated with the K second mapping features to maximize the expressiveness of the features.

[0070] In some embodiments, S304 may include the following steps S601 to S602:

[0071] S601: normalize the similarities corresponding to the K second mapping features to obtain K normalized results.

[0072] Here, the sum of the K normalized results is 1.

[0073] S602: Based on the K normalization results, aggregate the K second mapping features to obtain aggregated degraded features.

[0074] In the embodiment of the present application, the formula (2) corresponding to the aggregation process is:

[0075]

[0076] Among them, f k is the second mapping feature, G k is the normalized result, and Mf is the degradation feature after aggregation. k =1.

[0077] In some embodiments, S103 may include the following steps S701 to S703:

[0078] S701: Determine similarities between the distortion feature and the M first degradation features respectively.

[0079] Substituting the distortion feature and the first degradation feature into the above formula (1), the similarity between the distortion feature and the first degradation feature is obtained.

[0080] S702: Based on the M similarities, select K second degradation features from the M first degradation features.

[0081] In some embodiments, S702 may include the following steps: sorting the M similarities in descending order to obtain a sorting result; and using the first degradation features corresponding to the first K similarities in the sorting result as K second degradation features.

[0082] S703: Based on the similarities corresponding to the K second degenerate features, aggregate the K second degenerate features to obtain aggregated degenerate features.

[0083] In some embodiments, S703 may include the following steps: normalizing the similarities corresponding to the K second degradation features to obtain K normalized results; and aggregating the K second degradation features based on the K normalized results to obtain aggregated degradation features.

[0084] In some embodiments, the method further includes the following steps S801 to S802:

[0085] S801: Construct a prediction loss function based on the actual results and evaluation results of the image quality.

[0086] It should be noted that in order to measure the prediction quality and optimize the model's capabilities, the prediction error loss is introduced. Generally speaking, the difference between the image quality assessment result (i.e., the predicted value) and the true result (i.e., the label value) well reflects the prediction effect of the image quality assessment model on the quality of the distorted image. Therefore, the mean square error (MSE) is used as the metric for error loss. The formula (3) of the prediction loss function is:

[0087]

[0088] Where Q represents the number of distorted samples, y represents the true result, and y pre Indicates the evaluation result.

[0089] S802: Based on the prediction loss function, the initial image quality assessment model is trained to obtain a trained image quality assessment model; wherein the trained image quality assessment model at least includes: a degradation network and a regression network.

[0090] In the embodiment of the present application, by minimizing the prediction loss function, the gap between the evaluation result of image quality and the actual result can be reduced, thereby improving the prediction ability of the image quality evaluation model for image quality. During the training process, the prediction loss function is minimized. In this way, the image quality evaluation model can more accurately predict the quality level of the image.

[0091] In some embodiments, the method further includes the following steps S901 to S903:

[0092] S901: Construct a prediction loss function based on the actual results and evaluation results of the image quality.

[0093] S902: constructing a degradation loss function based on the reference feature and the first degradation feature.

[0094] S903: Based on the prediction loss function and the degradation loss function, the initial image quality assessment model is trained to obtain a trained image quality assessment model; wherein the trained image quality assessment model at least includes: a degradation network and a regression network.

[0095] It should be noted that in order to achieve the best performance of the visual degradation quality assessment model, a hybrid loss function L is proposed, which consists of the degradation loss function L d And the prediction loss function L p The two parts are defined as their sum, expressed as formula (4):

[0096] L=L d +L p (4)

[0097] Among them, the degradation loss function L d The degradation error from the reference image to the simulated distorted image is measured to ensure that the degradation network model can accurately reflect the degree of degradation of the reference image. p It is the difference between the evaluation result and the real result based on image quality, and is used to measure the accuracy of the regression network model in predicting image quality. During the training process, optimization algorithms such as gradient descent are used to minimize the hybrid loss function to adjust the parameters of each network model, thereby improving the accuracy and robustness of image quality assessment. In addition, the design of this comprehensive loss function can help us evaluate image quality more comprehensively, enable the model to better adapt to different types of distortion, and provide more accurate evaluation results.

[0098] Based on the above embodiment, the present application specifically illustrates a flowchart of an image quality assessment process. Fig.10 is a flowchart of the image quality assessment process in the embodiment of the present application, such as Fig.10As shown, it includes a feature extraction stage 11, a degradation stage 22, an aggregation stage 33 and an evaluation stage 44;

[0099] Feature extraction stage 11:

[0100] Get N reference images (denoted as I ref ) and N distorted images (denoted as I dstr ).

[0101] Input N reference images into the Convolutional Neural Network (CNN), i.e., the feature extraction network, to obtain the reference feature set r i Input N distorted images into the feature extraction network to obtain the distorted feature set d i . Where i is greater than or equal to 1 and less than or equal to N. The feature block sizes of the reference feature set and the distorted feature set are the same. The feature extraction network can be ResNet50 or VGG16.

[0102] Degeneration stage 22:

[0103] During transmission, compression, processing, or other operations applied to the reference image, the distorted image will gradually degrade. In order to capture the perceptual degradation content in the distorted image, the concept of degradation space and the design of degradation network are introduced. Taking into account the image degradation process and understanding the key features related to distortion, the image degradation process is described as follows (5):

[0104] Df i =D z (r i ;w) (5)

[0105] Among them, Df i Denotes the degenerate feature, D z represents the degenerate network, z represents the degenerate space, r i represents the reference features of the input, and w represents the parameters of the visual degradation network.

[0106] To simulate the degradation of the reference image, the reference features are input into Figure 2 The degenerate network shown includes T degenerate units, and the first degenerate feature (denoted as Df i ). For example, the degradation network uniformly resizes and crops N reference images to 224×224×3, where r i ∈RN×L, N represents the batch size, L represents the feature dimension, Df i ∈R H×W×C, H, W, C represent the height, width and size of the channel respectively. The degradation network can extract useful features from the reference image and encode them into corresponding degradation features, which can more accurately simulate the degradation process of the image and learn key features related to distortion. It provides more comprehensive and accurate information for IQA and provides a new vision for improving and optimizing image restoration models.

[0107] Next, the M first degenerate features are input into the converter (Convertor, C) (i.e., the conversion network), and are linearly transformed by weighted and biased operations through I linear layers to obtain the transformed new features, i.e., the M first mapping features, represented as {f1, f2, ..., f M-1 ,f M}.

[0108] It should be noted that since the structure of each mapping space (i.e., linear layer) within the conversion network is the same, they share the same parameters and learning capabilities. Therefore, each mapping space can learn a specific feature representation, allowing different perspectives to complete the expression of feature information. This feature shows higher flexibility and adaptability in capturing various aspects of the input features. Therefore, the conversion network not only performs well in IQA, but also shows certain applicability and versatility for video quality assessment.

[0109] Next, the reference feature r i The corresponding M first mapping features are {f1,f2,...,f M-1 ,f M} and distortion characteristics d i Input to the Similarity Module (SIM), output M similarities, denoted as: {S1, S2, ..., S M-1 ,S M}.

[0110] Here, the cosine similarity formula is used to calculate the similarity between the first mapping feature and the distortion feature. The similarity calculation formula (1) is:

[0111]

[0112] Where 0≤S m ≤1, S m is the similarity, f m is the first mapping feature, m is greater than or equal to 1 and less than or equal to M, d i is the distortion feature. When there are N distorted images, i is greater than or equal to 1 and less than or equal to N. ⊙ and ‖·‖ represent the dot product and norm, respectively.

[0113] By calculating the similarity value between the first mapping feature and the distorted feature, this value represents the similarity relationship between the first mapping feature and the distorted feature. The larger the similarity value, the more similar they are and the closer the content is. The positive sample pair represents the distorted feature and the first mapping feature with the same content, and the similarity between them is 1. This means that they are very close in the feature space and have highly similar features. On the other hand, the negative sample pair represents that the similarity between the features is 0. This means that these features are relatively far apart in the feature space and have a large difference. In order to intuitively describe the generation process of positive and negative sample pairs, as shown in Fig.11 , which is a schematic diagram of a similarity matrix with a dimension of (N, NM), wherein N represents the number of distorted features, NM represents the number of first mapping features, and blocks of different colors represent the similarity between the distorted features and the first mapping features.

[0114] By analyzing the correlation coefficients in this matrix, we can obtain similarity patterns between features. Images with higher correlation coefficients indicate that they are closer in feature space and may have similar appearance, structure, or content. Images with lower correlation coefficients indicate that they are relatively far away in feature space and may have different appearance, structure, or content. By using the correlation coefficients in this matrix, we can sort, cluster, and classify features to better understand their relationships and similarities.

[0115] Aggregation stage 33:

[0116] As we all know, the higher the similarity, the closer the image quality between the simulated distorted image of the reference image and the real distorted image. When a higher similarity is obtained, the mapped features are not only very similar to the features of the real distorted image, but also can accurately capture the underlying content of the real distorted image. On this basis, the K similarities with the highest similarity are selected from the M similarities, and the first mapped features corresponding to the K similarities are used as the second mapped features to promote a high level of matching between the mapped features and the distorted features. This approach helps us capture these complex degradation patterns more comprehensively, thereby enhancing our understanding and simulation capabilities of the image degradation process.

[0117] The features of the reference image obtained by the degradation network and the converter C are: {f1,f2,...,f M-1 ,f M}, distortion feature d i , and the similarity between them {S1,S2,...,S M-1 ,S M Then, the sampling sorting function is used to sort the M similarities from large to small, and the ranking of each similarity in the M similarities is calculated to obtain the sorting result, and the first K similarities S of the sorting result are kThe corresponding first mapping features are used as K second mapping features f k .

[0118] Next, the first K similarities S of the sorted results are k Perform normalization (softmax) to obtain K normalized results G k . G1+G2+...+G k =1.

[0119] Next, the K normalized results G k and K second mapping features f k Substitute into formula (2) and we can get the degradation characteristics after aggregation.

[0120]

[0121] That is, using the information of the reference image to better restore the key features of the distorted image. The reference image contains complete feature information, which helps to compensate for the missing and distorted information in the distorted image. This method not only captures basic information and features more effectively, enhancing the model's understanding of the image, but also provides a more comprehensive and precise evaluation, thereby further improving the accuracy of the evaluation.

[0122] Evaluation Phase 44:

[0123] The reference feature r i The aggregated degradation features are input into the regression network, and the quality assessment result of the distorted image is output.

[0124] In the above scheme, the concept of degradation space is introduced, and the degraded image of the reference image in the layer-by-layer degradation process is learned instead of the restored image. This can better capture the characteristics of various types of distorted images and improve the performance of the evaluation algorithm at different distortion levels.

[0125] In the above scheme, the degradation network is used to learn the degradation content of the reference image, and the converter is used to map it to multiple spaces for evaluation. In addition to using the degradation network to obtain a degradation expression that is closer to the distorted image, similarity calculation is also used to compare the similarity between the degradation feature and the distortion feature. This can more accurately measure the degree of image degradation and improve the accuracy of the evaluation.

[0126] It should be noted that in order to measure the prediction quality and optimize the model's capabilities, the prediction error loss is introduced. Generally speaking, the difference between the image quality assessment result (i.e., the predicted value) and the true result (i.e., the label value) well reflects the prediction effect of the image quality assessment model on the quality of the distorted image. Therefore, the mean square error (MSE) is used as the metric for error loss. The formula (3) of the prediction loss function is:

[0127]

[0128] Where Q represents the number of distorted samples, y represents the true result, and y pre Indicates the evaluation result.

[0129] Image quality assessment models include Fig.10 The feature extraction network, degradation network, conversion network and regression network in .

[0130] By minimizing the prediction loss function, the gap between the image quality assessment result and the actual result can be reduced, thereby improving the image quality prediction ability of the image quality assessment model. During the training process, the prediction loss function is minimized. In this way, the image quality assessment model can more accurately predict the quality level of the image.

[0131] In order to achieve the best performance of visual degradation quality assessment model, a hybrid loss function L is proposed, which is composed of the degradation loss function L d And the prediction loss function L p The two parts are defined as their sum, expressed as formula (4):

[0132] L=L d +L p (4)

[0133] Among them, the degradation loss function L d The degradation error from the reference image to the simulated distorted image is measured to ensure that the degradation network model can accurately reflect the degree of degradation of the reference image. p It is the difference between the evaluation result and the real result based on image quality, and is used to measure the accuracy of the regression network model in predicting image quality. During the training process, optimization algorithms such as gradient descent are used to minimize the hybrid loss function to adjust the parameters of each network model, thereby improving the accuracy and robustness of image quality assessment. In addition, the design of this comprehensive loss function can help us evaluate image quality more comprehensively, enable the model to better adapt to different types of distortion, and provide more accurate evaluation results.

[0134] The beneficial effects of the above scheme are summarized as follows:

[0135] First, the full-reference image quality evaluation method based on visual degradation space can accurately evaluate the quality of the image. Traditional image quality evaluation methods may not be able to capture small changes in image quality, while the above scheme can comprehensively consider multiple image quality factors and provide more comprehensive and accurate evaluation results.

[0136] 2. The above scheme is applicable to the evaluation of the quality of QR code images. By accurately evaluating the quality of QR code images, image quality problems can be discovered and corrected in advance, thereby improving the reliability and stability of QR codes. This helps to improve the recognition rate and decoding accuracy of QR codes in various application fields.

[0137] 3. The accurate image quality evaluation results of the above scheme can provide guidance for the performance improvement of image-related applications. For example, in the fields of digital image processing, image transmission and image compression, optimization and adjustment can be made based on the image quality evaluation results to improve the visual effect and transmission efficiency of the image.

[0138] Fourth, by ensuring accurate evaluation of image quality, the above solution can provide a better user experience. In image-related applications, users often expect to obtain high-quality image content. By using the above solution to evaluate image quality, it can ensure that users obtain clear, distortion-free images, improving user satisfaction and usage experience.

[0139] 5. The above scheme is based on the full reference image quality evaluation method of visual degradation space, which provides new ideas and methods for the research and development of image quality evaluation technology. By continuously improving and optimizing this scheme, the practical application and promotion of image quality evaluation technology in a wider range of application fields can be promoted.

[0140] In order to implement the method of the embodiment of the present application, based on the same inventive concept, an image quality assessment device is also provided in the embodiment of the present application. Fig.12 FIG. 1 is a schematic diagram of the structure of the image quality assessment device in an embodiment of the present application. Fig.12 As shown, the image quality assessment device 120 includes:

[0141] An acquisition module 1201 is used to acquire a reference feature of a reference image and a distortion feature of a distorted image;

[0142] A degradation network 1202 including T degradation units, used to process the reference features to obtain M first degradation features; wherein T≥M;

[0143] The processing module 1203 is used to obtain an aggregated degradation feature based on the distortion feature and the M first degradation features; wherein the aggregated degradation feature is a simulation of the distortion feature;

[0144] The regression network 1204 is used to process the aggregated degradation features and the reference features to obtain a quality assessment result of the distorted image.

[0145] In the embodiment of the present application, the degradation characteristics of the reference image in the layer-by-layer degradation process are learned through the degradation network, and then combined with the distortion characteristics, so that the obtained aggregated degradation characteristics are closer to the degradation expression of the distorted image, and then the aggregated degradation characteristics and reference characteristics are processed through the regression network to obtain the quality evaluation result of the distorted image. This use of the reference image as the evaluation standard can more accurately capture the subtle changes in image quality, thereby improving the accuracy of the image quality evaluation result.

[0146] In some embodiments, the processing module 1203 is further used to input the M first degradation features into the conversion network to obtain M first mapping features; determine the similarities between the distortion features and the M first mapping features respectively; based on the M similarities, select K second mapping features from the M first mapping features; based on the similarities corresponding to the K second mapping features, aggregate the K second mapping features to obtain the aggregated degradation features.

[0147] In some embodiments, the processing module 1203 is further used to sort the M similarities in descending order to obtain a sorting result; and use the first mapping features corresponding to the first K similarities of the sorting result as the K second mapping features.

[0148] In some embodiments, the processing module 1203 is further used to normalize the similarities corresponding to the K second mapping features to obtain K normalized results; based on the K normalized results, aggregate the K second mapping features to obtain the aggregated degraded features.

[0149] In some embodiments, the processing module 1203 is further used to determine the similarities between the distortion features and the M first degradation features respectively; based on the M similarities, select K second degradation features from the M first degradation features; based on the similarities corresponding to the K second degradation features, aggregate the K second degradation features to obtain the aggregated degradation features.

[0150] In some embodiments, a training module is also included, which is used to construct a prediction loss function based on the actual results and evaluation results of image quality; based on the prediction loss function, the initial image quality assessment model is trained to obtain a trained image quality assessment model; wherein the trained image quality assessment model at least includes: the degradation network and the regression network.

[0151] In some embodiments, the training module is further used to construct a degradation loss function based on the reference feature and the first degradation feature; and to train the initial image quality assessment model based on the prediction loss function and the degradation loss function to obtain a trained image quality assessment model.

[0152] In some embodiments, a feature extraction network is also included, which is used to extract features from a reference image to obtain reference features; and is also used to extract features from a distorted image to obtain distorted features.

[0153] The present application also provides another electronic device, Fig.13 Schematic diagram of the structure of the electronic device in the embodiment of the present application. Fig.13 As shown, the electronic device 130 includes: a processor 1301 and a memory 1302 configured to store a computer program that can be run on the processor;

[0154] The processor 1301 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0155] Of course, in practical applications, Fig.13 As shown, the various components in the electronic device 130 are coupled together via a bus system 1303. It is understood that the bus system 1303 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1303 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.13 Various buses are labeled as bus system 1303.

[0156] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application do not specifically limit this.

[0157] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0158] In an exemplary embodiment, the present application also provides a computer-readable storage medium for storing a computer program.

[0159] Optionally, the computer-readable storage medium can be applied to any one of the methods in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the processor in each method in the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0160] Illustratively, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by a processor of an electronic device to complete the steps of any of the aforementioned methods.

[0161] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0162] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0163] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art may understand that all or part of the steps of implementing the above method embodiments may be completed by hardware related to program instructions, and the above program may be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0164] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0165] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0166] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0167] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for evaluating image quality, characterized in that: The method comprises: Obtaining reference features of the reference image and distortion features of the distorted image; Inputting the reference feature into a degradation network including T degradation units to obtain M first degradation features; wherein T≥M; Based on the distortion feature and the M first degradation features, an aggregated degradation feature is obtained; wherein the aggregated degradation feature is a simulation of the distortion feature; The aggregated degradation features and the reference features are input into a regression network to obtain a quality assessment result of the distorted image.

2. The method according to claim 1, characterized in that The obtaining, based on the distortion feature and the M first degradation features, an aggregated degradation feature includes: Inputting the M first degradation features into a conversion network to obtain M first mapping features; Determining the similarity between the distortion feature and the M first mapping features respectively; Based on the M similarities, selecting K second mapping features from the M first mapping features; Based on the similarities corresponding to the K second mapping features, the K second mapping features are aggregated to obtain the aggregated degraded features.

3. The method according to claim 2, characterized in that The selecting K second mapping features from the M first mapping features based on the M similarities includes: Sorting the M similarities in descending order to obtain a sorting result; The first mapping features corresponding to the first K similarities of the sorting results are used as the K second mapping features.

4. The method according to claim 2, characterized in that: The aggregating the K second mapping features based on the similarities corresponding to the K second mapping features to obtain the aggregated degraded features includes: Normalizing the similarities corresponding to the K second mapping features to obtain K normalized results; Based on the K normalized results, the K second mapping features are aggregated to obtain the aggregated degraded features.

5. The method according to claim 1, characterized in that The obtaining, based on the distortion feature and the M first degradation features, an aggregated degradation feature includes: Determining similarities between the distortion feature and the M first degradation features respectively; Based on the M similarities, selecting K second degradation features from the M first degradation features; Based on the similarities corresponding to the K second degenerate features, the K second degenerate features are aggregated to obtain the aggregated degenerate features.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Based on the real results and evaluation results of image quality, a prediction loss function is constructed; Based on the prediction loss function, the initial image quality assessment model is trained to obtain a trained image quality assessment model; wherein the trained image quality assessment model at least includes: the degradation network and the regression network.

7. The method according to claim 6, characterized in that The initial image quality assessment model is trained based on the prediction loss function to obtain a trained image quality assessment model, including: Constructing a degradation loss function based on the reference feature and the first degradation feature; Based on the prediction loss function and the degradation loss function, the initial image quality assessment model is trained to obtain a trained image quality assessment model.

8. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the reference feature of the reference image and the distortion feature of the distorted image includes: Inputting the reference image into a feature extraction network to obtain the reference features; The distorted image is input into the feature extraction network to obtain the distortion feature.

9. An image quality assessment device, characterized in that: The device comprises: An acquisition module, used for acquiring reference features of a reference image and distortion features of a distorted image; A degradation network including T degradation units, used to process the reference features to obtain M first degradation features; wherein T≥M; A processing module, configured to obtain an aggregated degradation feature based on the distortion feature and the M first degradation features; wherein the aggregated degradation feature is a simulation of the distortion feature; The regression network is used to process the aggregated degradation features and the reference features to obtain a quality assessment result of the distorted image.

10. An electronic device, characterized in that: The electronic device comprises: a processor and a memory configured to store a computer program capable of running on the processor, Wherein, the processor is configured to execute the steps of the method according to any one of claims 1 to 8 when running the computer program.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images