Image quality evaluation method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides an image quality evaluation method, and the method comprises the steps: obtaining a to-be-evaluated image and a reference image related to the to-be-evaluated image; the to-be-evaluated image and the reference image are input into a trained image quality evaluation model, the perception distance between the to-be-evaluated image and the reference image is output through the trained image quality evaluation model, and the perception distance is used for representing the image quality of the to-be-evaluated image; wherein the trained image quality evaluation model is used for determining a first image distribution parameter of the to-be-evaluated image and determining a second image distribution parameter of the reference image; determining a sensing distance between the to-be-evaluated image and the reference image based on the first image distribution parameter and the second image distribution parameter; the image distribution parameters represent feature distribution conditions of a plurality of image scales and / or a plurality of pixel blocks of the image. The embodiment of the invention further provides a device, electronic equipment, a storage medium and a computer program product.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an image quality evaluation method, apparatus, device, storage medium, and computer program product. Background Art

[0002] Image Quality Assessment (IQA) is a key problem in the field of computer vision, which aims to simulate the image quality evaluation in human subjective perception. In today's wide range of digital image applications, IQA plays a vital role. IQA can evaluate the quality of images and detect problems such as distortion, blur or noise in the image. By detecting and identifying these problems, corresponding measures can be taken to improve the image quality, thereby improving the accuracy and success rate of scanning.

[0003] Currently, Full Reference-IQA (FR-IQA) methods usually compare the image to be evaluated with the reference image and determine the image quality based on the difference between the two. However, most FR-IQA methods tend to model specific types of distortion and lack the ability to comprehensively characterize the human visual system (HVS), resulting in limited versatility. Summary of the invention

[0004] Embodiments of the present application provide an image quality method, apparatus, device, storage medium, and computer program product.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image quality evaluation method, the method comprising:

[0007] Acquire an image to be evaluated and a reference image associated with the image to be evaluated;

[0008] Inputting the image to be evaluated and the reference image into a trained image quality assessment model, and outputting a perceptual distance between the image to be evaluated and the reference image through the trained image quality assessment model, wherein the perceptual distance is used to characterize the image quality of the image to be evaluated;

[0009] Among them, the trained image quality assessment model is used to determine the first image distribution parameter of the image to be evaluated, and to determine the second image distribution parameter of the reference image; and based on the first image distribution parameter and the second image distribution parameter, the perceptual distance between the image to be evaluated and the reference image is determined; the image distribution parameter characterizes the characteristic distribution of multiple image scales and / or multiple pixel blocks of the image.

[0010] In a second aspect, an embodiment of the present application provides an image quality assessment device, the device comprising:

[0011] an acquisition unit, configured to acquire the image to be evaluated and a reference image associated with the image to be evaluated;

[0012] a processing unit configured to input the image to be evaluated and the reference image into a trained image quality evaluation model, and output a perceptual distance between the image to be evaluated and the reference image through the trained image quality evaluation model, wherein the perceptual distance is used to characterize the image quality of the image to be evaluated;

[0013] Among them, the trained image quality assessment model is used to determine the first image distribution parameter of the image to be evaluated, and to determine the second image distribution parameter of the reference image; and based on the first image distribution parameter and the second image distribution parameter, the perceptual distance between the image to be evaluated and the reference image is determined; the image distribution parameter characterizes the characteristic distribution of multiple image scales and / or multiple pixel blocks of the image.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method described in the first aspect.

[0016] In a fifth aspect, the present application provides a computer program product, comprising computer program instructions, which, when executed by a processor, implement the method described in the first aspect.

[0017] The embodiment of the present application provides an image quality assessment method, wherein an electronic device can obtain an image to be evaluated and a reference image associated with the image to be evaluated; the image to be evaluated and the reference image are input into a trained image quality assessment model, and the perceptual distance between the image to be evaluated and the reference image is output through the trained image quality assessment model, and the perceptual distance is used to characterize the image quality of the image to be evaluated; wherein the trained image quality assessment model is used to determine a first image distribution parameter of the image to be evaluated, and to determine a second image distribution parameter of the reference image; and the perceptual distance between the image to be evaluated and the reference image is determined based on the first image distribution parameter and the second image distribution parameter; the image distribution parameter characterizes the characteristic distribution of multiple image scales and / or multiple pixel blocks of the image. It can be seen that the trained image quality assessment model can represent the image to be evaluated and the reference image as image distribution parameters. The complex relationship between the characteristic distribution of different scales and / or regions in the image is captured by the image distribution parameters, and then a more accurate image quality prediction is achieved by measuring the perceptual distance between the first image distribution parameter of the image to be evaluated and the second image distribution parameter of the reference image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a QR code provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of a method for evaluating image quality provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a processing flow of a trained image quality assessment model provided in an embodiment of the present application Figure 1 ;

[0021] Figure 4 A schematic diagram of a processing flow of a trained image quality assessment model provided in an embodiment of the present application Figure 2 ;

[0022] Figure 5 A schematic diagram of a phased measurement framework provided for an embodiment of the present application;

[0023] FIG. 6A to FIG. 6C A schematic diagram of a cutting rule provided in an embodiment of the present application;

[0024] 7A to 7F A schematic diagram of a graph convolution feature extraction method provided in an embodiment of the present application;

[0025] Figure 8 A schematic diagram of a model training process provided in an embodiment of the present application;

[0026] Fig. 9 A schematic diagram of a model architecture provided for an embodiment of the present application;

[0027] Fig.10 A schematic diagram of the structure of an image quality assessment device provided in an embodiment of the present application;

[0028] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0031] It should also be understood that the "indication" mentioned in the embodiments of the present application can be a direct indication, an indirect indication, or an indication of an association relationship. For example, A indicates B, which can mean that A directly indicates B, for example, B can be obtained through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; it can also mean that there is an association relationship between A and B.

[0032] It should also be understood that the "correspondence" mentioned in the embodiments of the present application may indicate a direct or indirect correspondence between the two, or an association between the two, or a relationship of indication and being indicated, configuration and being configured, etc.

[0033] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0034] In recent years, digital images have been widely used in daily life and professional fields, and scanning QR codes has become a common task. The following uses QR codes in different environments as examples to illustrate. The environmental and device conditions for scanning QR codes vary, including different lighting, angles, distances, and device types. For example, people may scan codes indoors, outdoors, in dim light, or in bright sunshine. These changes in conditions may result in differences in the quality of the scanned images. In addition, the loss of image quality after multiple processing steps such as rendering, compression, and transmission is unavoidable, which will inevitably seriously affect the user's final visual experience. Image quality directly affects the recognition performance of the scanned code. For reference, Figure 1 As shown in the QR code diagram, low-quality images may contain noise, blur, distortion or lighting problems, all of which will reduce the accuracy of the code scanning algorithm. Therefore, for code scanning tasks, ensuring image quality is crucial.

[0035] Image Quality Assessment (IQA) is a key problem in the field of computer vision, which aims to simulate the image quality evaluation in human subjective perception. In today's wide range of digital image applications, IQA plays a vital role. IQA methods can evaluate the quality of images and detect problems such as distortion, blur or noise in the image. By detecting and identifying these problems, corresponding measures can be taken to improve the image quality, thereby improving the accuracy and success rate of scanning.

[0036] Currently, Full Reference-IQA (FR-IQA) methods are usually performed by comparing pixels or blocks in the distorted image with the reference image. However, most methods tend to model specific distortion types and lack the ability to comprehensively characterize the human visual system (HVS), resulting in limited versatility.

[0037] In a FR-IQA method, the system extracts the structural features, texture features and exposure of each distorted image to obtain the feature vector of each image, and obtains the image feature vector space based on the feature vectors of all distorted images; supports vector regression is used to establish a functional relationship between the score vector space and the image feature vector space to obtain an image quality evaluation model; the image to be evaluated is input, and the image quality evaluation model is used to obtain the quality score of the image to be evaluated.

[0038] In another image quality evaluation method, the color similarity, contrast similarity and gradient similarity of the visualized QR code are calculated respectively, and the visual QR code image quality evaluation result is obtained by the final weighted average.

[0039] It can be seen that the relevant image quality assessment methods all model specific types of distortion (such as structural features, texture features, exposure, color, and contrast). However, in the context of IQA, the model's ability to respond to the distribution of image content indicates its ability to perceive objects and distortions. The spatial features within the image are inherently non-stationary, and the impact of superimposed noise information on perceived visual quality is also non-stationary. Therefore, the aggregation of spatial local features may not consistently correspond to the overall perceptual quality of the distorted image. In addition, the form of objects in images is often highly irregular, which makes it difficult to effectively represent their correlations using traditional grid or sequence representations.

[0040] In a FR-IQA method, the system extracts the structural features, texture features and exposure of each distorted image to obtain the feature vector of each image, and obtains the image feature vector space based on the feature vectors of all distorted images; supports vector regression is used to establish a functional relationship between the score vector space and the image feature vector space to obtain an image quality evaluation model; the image to be evaluated is input, and the image quality evaluation model is used to obtain the quality score of the image to be evaluated.

[0041] In another image quality evaluation method, the color similarity, contrast similarity and gradient similarity of the visualized QR code are calculated respectively, and the visual QR code image quality evaluation result is obtained by the final weighted average.

[0042] It can be seen that the relevant image quality evaluation methods all model specific distortion types (such as structural features, texture features, exposure, color, and contrast). However, due to the diversity of image distortion types, many QR code images may have single or multiple distortions such as blur, noise, distortion, compression, and color shift. Modeling only specific distortion types will lack the ability to comprehensively represent the human visual system and have limited versatility.

[0043] In view of this, the embodiments of the present application provide an image quality evaluation method, device, electronic device, storage medium, and computer program product, which can comprehensively capture the complex relationship between the feature distribution of different scales and regions within the image, making it applicable to various types of distortion, including color distortion and texture structure distortion. Through the image distribution parameters of the image to be evaluated and the reference image, the perceptual distance between the image to be evaluated and the reference image, that is, the difference in image quality, is determined, thereby achieving a more accurate image quality prediction. This method is of great significance to various application fields, including but not limited to image compression, enhancement and restoration, and helps to ensure that images can provide reliable visual effects in different situations.

[0044] It should be noted that the execution subject of the image quality evaluation method provided in the embodiment of the present application can be the image quality evaluation device provided in the implementation of the present application, or an electronic device integrating the image quality evaluation device, and the image quality evaluation device here can be implemented in hardware or software. Among them, the electronic device can be a smart phone, a tablet computer, a personal calculator, a server or an industrial calculator, etc., and the embodiment of the present application does not limit this.

[0045] To facilitate understanding of the technical solutions of the embodiments of the present application, the technical solutions of the present application are described in detail below through specific embodiments. The above related technologies can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application. The embodiments of the present application include at least part of the following contents.

[0046] The present application embodiment provides an image quality evaluation method, such as Figure 2 As shown, the method includes step S210 and step S220.

[0047] S210: Acquire the image to be evaluated and a reference image associated with the image to be evaluated.

[0048] It should be noted that the image to be evaluated mentioned in the embodiments of the present application can also be called a distorted image or a warped image, and the three are equivalent or interchangeable.

[0049] It should be understood that the image to be evaluated is an image that has undergone some form of visual transformation or damage, which may include compression, adding noise, image camouflage, etc. The reference image can be an original, undistorted image, which is generally considered to be of high quality. For example, in the image compression process, the image before compression is the reference image, and the image after compression is the image to be evaluated. For another example, in the image enhancement process, the image before enhancement processing is the image to be evaluated, and the image after enhancement processing is the reference image.

[0050] It should be understood that the image quality evaluation method provided in the embodiment of the present application is a method based on FR-IQA, which compares the image to be evaluated with the reference image and determines the image quality of the image to be evaluated based on the difference between the two.

[0051] In the embodiment of the present application, the image quality evaluation device can obtain the image to be evaluated and the reference image. For example, in the image compression process, the image quality evaluation device can use the image before compression as the reference image and the image after compression as the image to be evaluated. For another example, in the image enhancement process, the image quality evaluation device can use the image before enhancement as the image to be evaluated and the image after enhancement as the reference image.

[0052] S220, inputting the image to be evaluated and the reference image into a trained image quality evaluation model, and outputting a perceptual distance between the image to be evaluated and the reference image through the trained image quality evaluation model, where the perceptual distance is used to characterize the image quality of the image to be evaluated.

[0053] It should be understood that the trained image quality assessment model refers to a pre-trained image quality assessment model. The trained image quality assessment model can be trained by an image quality assessment device, or obtained from other devices providing models (such as a server), and the present application embodiment does not limit this.

[0054] For the convenience of expression, the term “trained model” is used below to refer to a trained image quality assessment model.

[0055] In the embodiment of the present application, the trained model is used to determine the first image distribution parameter of the image to be evaluated and the second image distribution parameter of the reference image; and the perceptual distance between the image to be evaluated and the reference image is determined based on the first image distribution parameter and the second image distribution parameter. Figure 3 As shown, the trained model in the embodiment of the present application can perform the following steps:

[0056] S310, determining a first image distribution parameter of the image to be evaluated and a second image distribution parameter of the reference image;

[0057] S320 , determining a perceptual distance between the image to be evaluated and the reference image based on the first image distribution parameter and the second image distribution parameter.

[0058] It should be noted that the graph distribution parameter characterizes the characteristic distribution of multiple image scales and / or multiple pixel blocks of the image. That is, the first graph distribution parameter can characterize the characteristic distribution of multiple image scales and / or multiple pixel blocks in the image to be evaluated, and the second graph distribution parameter can characterize the characteristic distribution of multiple image scales and / or multiple pixel blocks in the parameter image.

[0059] It should be understood that image scale is an important concept in image processing, which involves the size of objects in the image and the viewing angle. By adjusting the scale of the image, different levels of detail information can be observed. For example, using a larger scale can see broader image features, while a smaller scale can observe more detailed image details.

[0060] It should also be understood that a pixel block can be composed of multiple pixels. In other words, a pixel block can be a small area in an image. It should be noted that in the embodiments of the present application, a pixel block can also be referred to as an image block, an image area, an area, a block, etc., and the above concepts are equivalent or interchangeable.

[0061] It should be noted that the graph distribution parameters may be parameters obtained based on a probabilistic graph model, wherein the probabilistic graph model usually represents the joint probability distribution of variables (e.g., each pixel block in an image at different image scales) through a graph structure. In other words, the graph distribution parameters describe or characterize the complex relationship between feature distributions at different scales and / or regions within an image, and can be applicable to various types of distortion, including color distortion and texture structure distortion.

[0062] In the embodiment of the present application, the first image distribution parameter is used to represent the image to be evaluated, and the second image distribution parameter is used to represent the reference image. The image quality of the image to be evaluated is represented by the perceptual distance between the first image distribution parameter and the second image distribution parameter.

[0063] It should be noted that the perceptual distance can be determined based on the cosine similarity, Euclidean distance, Manhattan distance, Hellinger distance, etc. between the distribution parameters of the first image and the distribution parameters of the second image. The perceptual distance can characterize the difference in image quality between the image to be evaluated and the reference image. The image quality of the image to be evaluated is characterized by the difference between the image qualities. A farther perceptual distance or a perceptual distance with a larger value usually corresponds to a poorer image quality, while a closer perceptual distance or a perceptual distance with a smaller value usually corresponds to a better or superior image quality.

[0064] In some embodiments, the perceived distance may be quantified as a quality score, with lower scores generally corresponding to poorer image quality and higher scores corresponding to better image quality.

[0065] In the embodiment of the present application, the trained model can represent the image to be evaluated and the reference image as graph distribution parameters. The graph distribution parameters capture the complex relationship of feature distributions of different scales and / or regions in the image, and then measure the perceptual distance between the first graph distribution parameter of the image to be evaluated and the second image distribution parameter of the reference image, thereby achieving more accurate image quality prediction.

[0066] The detailed process of determining the graph distribution parameters is described in detail below.

[0067] In some embodiments, the first graph distribution parameter and the second graph distribution parameter may only include graph distribution parameters at one image scale. Specifically, the trained model in S310 determines the graph distribution parameters of the image (including determining the first graph distribution parameters of the image to be evaluated and / or determining the second graph distribution parameters of the reference image), which may be that the trained model constructs the graph structure of the image at a fixed image scale, and encodes the image represented by the graph structure to extract the image features of the image, and then performs graph distribution representation based on the graph structure and the image features, thereby obtaining the graph distribution parameters.

[0068] In some embodiments, the first image distribution parameter and the second image distribution parameter may include image distribution sub-parameters at multiple image scales. In the embodiment of the present application, the number of multiple image scales is recorded as M, and M may be an integer greater than 1. It can be understood that both the first image distribution parameter and the second image distribution parameter may include image distribution sub-parameters at M image scales, or in other words, both the first image distribution parameter and the second image distribution parameter are composed of image distribution sub-parameters at M image scales.

[0069] refer to Figure 4 As shown in the flowchart, the trained model in S310 determines the first image distribution parameter of the image to be evaluated, which can be achieved through the following steps.

[0070] S3101a, establishing a graph structure of the image to be evaluated at the mth image scale; m is a positive integer less than or equal to M, and the graph structure is used to represent the association relationship between multiple pixel blocks in the image;

[0071] S3102a, extracting image features based on the graph structure of the image to be evaluated at the mth image scale;

[0072] S3103a, determining the mth image distribution sub-parameter based on the image structure and image features of the image to be evaluated at the mth image scale;

[0073] S3104a, continue to establish the graph structure of the image to be evaluated at the m+1th image scale to determine the m+1th graph distribution sub-parameter, until the Mth graph distribution sub-parameter is obtained; the first graph distribution parameter includes the above M graph distribution sub-parameters.

[0074] It should be understood that the trained model can determine the first image distribution parameters of the image to be evaluated in M ​​stages (which can be understood as M image scales), and the first image distribution parameters of the image to be evaluated can be composed of the image distribution sub-parameters of the M stages. The image scale processed in each stage is different. In one example, the trained model gradually reduces the resolution of the image to be evaluated in the M-stage processing, that is, the image scale in the M stages continues to decrease, such as downsampling the image to be evaluated in the last step of each stage. In another example, the trained model can gradually increase the resolution of the image to be evaluated in the M-stage processing, that is, the image scale in the M stages continues to increase, such as upsampling the image to be evaluated in the last step of each stage, which can effectively expand the receptive field of the trained model.

[0075] It should also be understood that the processing operations of the trained model in each stage are the same, or in other words, each stage uses the same module (a module can be understood as a processing unit for processing a certain task) for processing. Specifically, in each stage, the trained model can first input the image to be evaluated into the graph construction module, and construct a graph for the image to be evaluated through the graph construction module, that is, establish the graph structure of the image to be evaluated through the graph construction module. Then, the trained model can input the obtained graph structure into the feature extraction module, and extract features from the graph structure through the feature extraction module to obtain image features. Finally, the trained model can input the graph structure output by the graph construction module and the image features output by the feature extraction module into the graph distribution measurement module, and process them through the graph distribution measurement module to obtain the graph distribution sub-parameters of this stage.

[0076] For example, reference Figure 5 The figure shows a schematic diagram of a phased measurement framework, where M is 4. The trained model can process the image to be evaluated in four stages. In each stage, graph construction, feature extraction and graph distribution measurement are performed.

[0077] In the embodiment of the present application, each stage needs to first perform graph construction, that is, establish a graph structure of the image to be evaluated, and convert the image to be evaluated into a graph structure representation.

[0078] It should be noted that the graph structure can be used to represent the association relationship between multiple pixel blocks in an image, or in other words, the graph structure can be used to identify the association relationship between different regions in an image.

[0079] In some embodiments, step S3101a of establishing the graph structure of the image to be evaluated at the mth image scale may be implemented in the following manner:

[0080] The image to be evaluated is cropped according to the mth rule to obtain a plurality of pixel blocks corresponding to the image to be evaluated;

[0081] Based on the association relationship between any two pixel blocks in the multiple pixel blocks, a graph structure of the image to be evaluated at the mth image scale is established.

[0082] It can be understood that at each stage, the trained model can crop the image to be evaluated according to preset rules, and divide the image to be evaluated into multiple pixel blocks, each pixel block representing a small area in the image to be evaluated.

[0083] It should be noted that these pixel blocks are usually a collection of adjacent pixels and can be square or rectangular. The size and shape of the specific blocks can be adjusted according to specific needs and evaluation tasks.

[0084] Furthermore, the trained model can extract the regional features of each pixel block, and calculate the correlation relationship between any two pixel blocks based on the regional features of each pixel block.

[0085] It should be noted that the trained model can use a convolutional neural network (CNN) or a deep residual network to extract the initial feature vector (or regional feature) of each pixel block.

[0086] In some embodiments, the trained model can calculate the correlation between two pixel blocks in the image to be evaluated based on the initial feature vector of each pixel block, for example, calculate the similarity, distance or other correlation information of the two pixel blocks based on the initial feature vectors of the two pixel blocks.

[0087] In one implementation, the graph structure of the image may be an undirected graph model G = {V, E}. V is a vertex set in the undirected graph model G, and E is an edge set in the undirected graph model G. Each pixel block in the image may be represented as a vertex v, and the degree of correlation between blocks may be represented as an edge e. It should be understood that these edges may represent similarity, distance, or other association information between blocks.

[0088] In another implementation, the graph structure of the image can be represented by an adjacency matrix A. A can be an N*N dimensional matrix, where N is the total number of pixel blocks divided in the image. ij It can represent the degree of correlation between pixel block i and pixel block j.

[0089] It should be noted that the undirected graph model G and the adjacency matrix A can be converted into each other.

[0090] It should also be noted that the rules for cropping the image to be processed by the trained model at each stage can be the same or different. In other words, the mth rule and the m+1th rule can be the same or different. This embodiment of the application does not limit this.

[0091] For example, Fig. 6A is the original image, Figure 6B is the graph connection of the first stage, Figure 6C is the graph connection of the third stage. Among them, Figure 6B and Figure 6C Different rules are used to crop the original image into multiple pixel blocks.

[0092] It should be noted that since the image scale is different in each stage, the graph structure established in each stage also changes dynamically.

[0093] In the embodiment of the present application, after obtaining the graph structure of the image to be evaluated at each stage, the image represented by the graph structure may be encoded to extract image features.

[0094] Specifically, step S3102a extracts image features based on the graph structure of the image to be evaluated at the mth image scale, which can be achieved in the following way:

[0095] Based on the graph structure of the image to be evaluated at the m-th image scale, determine the neighboring pixel blocks associated with each pixel block in the image to be evaluated at the m-th image scale;

[0096] Based on the initial feature vector of each pixel block in the image to be evaluated at the mth image scale and the initial feature vectors of neighboring pixel blocks associated with each pixel block, graph convolution processing and feature fusion processing are performed to obtain a target feature vector of each pixel block, wherein the image features include the target feature vector of each pixel block.

[0097] It should be understood that in the mth stage, in order to obtain the global features of the image at the current scale, the trained model can classify and aggregate the pixel blocks, determine the most adjacent pixel blocks of each pixel block, and obtain the neighboring pixel blocks. Graph convolution and feature fusion processing are performed on each pixel block and the neighboring pixel blocks of each pixel block to superimpose the environmental information around the pixel block and establish an overall perceptual connection.

[0098] It should be noted that the above process of extracting image features can also be understood as an "image distribution" processing process. "Image distribution" takes into account that the human visual system receives image signals based on distribution rather than single pixels. Therefore, each pixel block is regarded as a certain distribution of the whole, which can more accurately reflect the overall properties of the pixels in its pixel block.

[0099] In some embodiments, the trained model can use the K-nearest neighbor algorithm (KNN) to calculate the K nearest vertices for each pixel block (i.e., each vertex in the undirected graph model G), establish a connection relationship between them, and comprehensively consider the similarity between them and the position distance in the original image as the degree of correlation between the vertices. The number of neighbors (expressed as K) is a key hyperparameter for shaping the graph representation structure of the image. This hyperparameter affects the ability to exchange information and the size of the constructed group. Too many neighbors may cause serious over-smoothing, and too sparse selection may cause network performance to degrade. Exemplarily, the K value is set to 9 in the embodiment of the present application.

[0100] In the embodiment of the present application, the trained model can use a graph convolutional neural network (GCN) or similar technology to perform graph convolution processing on the image represented by the graph structure. The graph convolution operation in GCN can refer to formula (1).

[0101]

[0102] in, and W represents the feature vector of pixel blocks i and j in the lth layer of the GCN network, that is, the feature vector of pixel blocks i and j. i and W j represents the weight matrix. represents the neighboring pixel blocks of pixel block i, φ (l) () represents an aggregate function.

[0103] In some embodiments, the aggregation function φ (l) () The maximum relative graph convolution method can be used to aggregate neighborhood relative features. For details, please refer to formula (2).

[0104]

[0105] It should be understood that the feature vector of pixel block i in the lth layer is subtracted from the feature vector of each neighboring pixel block of pixel block i, the maximum value in each dimension is obtained, and the feature vector of pixel block i in the l+1th layer is obtained by combining them.

[0106] Then, the feature vector output by the l+1th layer and the lth layer feature vector After connection, the multi-layer perceptron (MLP) is used to further extract features to obtain fused or aggregated features, thereby obtaining the target feature vector of each pixel block. After the above graph convolution and feature fusion processing, the target feature vector of each pixel block reflects the superposition value of the vertex and its surrounding environment.

[0107] It can be understood that all vertices in the undirected graph model G can achieve aggregate update of features through the above method. The updated undirected graph model G can be represented by the following formula (3).

[0108] G (l+1) =F(G (l) ,W (l) ) (3)

[0109] Among them, G (l) is the graph structure of the lth layer, G (l+1) The graph structure of the l+1th layer. (l) Represents G(l) The weight matrix of .

[0110] It should be noted that, in each stage, encoding can be performed based on the graph structure at each image scale, and the feature vectors of each vertex (i.e., pixel block) in the graph structure at each image scale can be updated to obtain the final image features. It can be understood that the final image features include the target feature vectors of each pixel block in the image, or in other words, the image features are composed of the target feature vectors of each pixel block in the image.

[0111] For example, the features extracted by graph convolution are shown in FIG7 , where Fig. 7A is the original image, Figure 7B is the feature obtained after convolution processing (stage 0), Figure 7C is the image feature of the first stage, Fig.7D is the image feature of the second stage, Fig. 7E is the image feature of the third stage, Figure 7F It is the image feature of the fourth stage.

[0112] In the embodiment of the present application, after obtaining the image features of the image to be evaluated at each stage, the image features of the image to be evaluated can be converted into graph distribution parameters to describe the feature distribution of different scales and regions in the image in more detail.

[0113] Specifically, S3103a determines the mth image distribution sub-parameter based on the image structure and image features of the image to be evaluated at the mth image scale, which can be achieved in the following way:

[0114] Based on the graph structure of the image to be evaluated at the mth image scale, multiple clusters are constructed; each of the multiple clusters includes multiple pixel blocks that are related to each other;

[0115] Based on the image features, the potential function product of multiple clusters is calculated to obtain the mth graph distribution sub-parameter.

[0116] It should be understood that in the mth stage (or in any stage), the trained model can convert the image features of the image to be evaluated into graph distribution sub-parameters to describe the feature distribution of different scales and regions in the image in more detail.

[0117] In the embodiment of the present application, considering that adjacent pixels exhibit non-causal interconnections, the present solution determines the graph distribution sub-parameters of the image through a probabilistic graph model based on undirected graph identification.

[0118] It should be understood that the graph distribution sub-parameters (also known as graph distribution representations) are an important step in capturing complex relationships and establishing overall perceptual connections, especially for irregular regions in images. Specifically, the method of first decomposing multiple vertices in the graph structure into clusters and then calculating the product of the potential functions of the clusters to obtain the joint probability distribution of the undirected graph helps to understand the image content more comprehensively.

[0119] It should be noted that the subset of vertices in the undirected graph model G where any two vertices are connected by an edge is called a clique. According to this rule, multiple vertices in the graph structure can be decomposed into multiple cliques.

[0120] Furthermore, after obtaining multiple clusters, the product of the potential functions (also called potential energy functions) of the multiple clusters can be calculated to obtain the joint probability distribution of the image, that is, the graph distribution sub-parameter. Specifically, at the mth stage (or at any stage), the graph distribution parameter of the image to be evaluated can be expressed by the following formula (4).

[0121]

[0122] Wherein, C represents the set of all clusters, and Q represents any cluster in C. Z represents a normalization function. Specifically, Z can refer to the following formula (5).

[0123]

[0124] In addition, E(·) represents a potential energy function. In an embodiment of the present application, the potential energy function of any cluster Q may include a first component and a second component, wherein the first component may be used to determine the perceived distance between the feature vector of a pixel in each pixel block in the cluster Q and the target feature vector of the pixel block, and the second component is used to determine the perceived distance between the target feature vectors of any two pixel blocks in the cluster Q.

[0125] Exemplarily, the potential energy function can be expressed as the following formula (6).

[0126]

[0127] in, is the first component, is the second component.

[0128] It should be noted that in the embodiment of the present application, the target feature vector of the pixel block in the graph structure is represented as X=x1,x2,…,x N For example, in the undirected graph model G, the vertex set V includes N vertices, where N is an integer greater than 1, and the image features corresponding to the vertex set V can be expressed as X=x1,x2,…,x N . In which, each x iCorresponding to a vertex, i.e. a pixel block, it represents the target feature vector of the pixel block.

[0129] It should also be noted that the target feature vector x of each pixel block i It can be a value obtained based on "image distribution" processing. "Image distribution" takes into account that the human visual system receives image signals based on distribution rather than single pixels. Therefore, each pixel block is regarded as a certain distribution of the whole, which can more accurately reflect the overall properties of the pixels in its pixel block.

[0130] It should be understood that the target feature vector x of each pixel block i It can be understood that the characteristic state of the pixel block, such as a certain brightness or texture distribution, can effectively reflect the overall properties of the pixels in the block.

[0131] In the embodiment of the present application, the first component ψ1(x i ) can represent the feature vector of each pixel in the i-th pixel block in the Q-th cluster and the target feature vector x of the i-th pixel block i The difference between them (which can also be understood as perceived cost or perceived distance).

[0132] In some embodiments, ψ1(x i ) can be calculated by the following formula (7).

[0133]

[0134] in, is the pixel set of pixel block i, and d is any pixel in pixel block i. f(d) represents the feature vector of pixel d, which is the internal perception feature extracted by neural network. C(f(d),x i ) is the squared difference cost of the perceptual feature, reflecting the difference between the feature vector f(d) of each pixel and the target feature vector x corresponding to pixel block i. i The difference between.

[0135] For example, C(f(d),x i ) can be specifically That is to say, the above formula (7) can be replaced by: in, is the target feature vector x i The corresponding perceptual feature center can be specifically understood as the mean value of the pixel features in pixel block i.

[0136] In addition, in the potential energy function shown in formula (6), the second component ψ2(x i ,x j ) can be used to measure the target feature vector x of pixel block i in the Qth cluster iand the target feature vector x of pixel block j j The difference between them (which can also be understood as perceived cost or perceived distance).

[0137] In some embodiments, ψ2(x i ,x j ) can be calculated by the following formula (8).

[0138] ψ2(x i ,x j )=ρ(x i ,x j )·s(f i ,f j ) (8)

[0139] Among them, f i represents the initial feature vector of pixel block i, f j represents the initial feature vector of pixel block j. i ,x j ) is a penalty factor used to capture the relationship between different image distributions. Specifically, when pixel blocks i and j with similar attributes have different random variables, a penalty is required. The embodiment of the present application can use the following formula (9) to calculate ρ(x i ,x j ).

[0140] ρ(x i ,x j )=|x i -x j |2 (9)

[0141] It can be understood that the penalty factor can be expressed by the binary norm between the initial eigenvectors.

[0142] In addition, s(f i ,f j ) can represent the variability between the initial feature vectors. The present application embodiment uses the structural similarity index measure (SSIM) to calculate the variability based on the mean, standard deviation and covariance values ​​corresponding to each pixel block. Specifically, s(f i ,f j ) can be calculated using the following formula (10).

[0143]

[0144] Among them, μ i , μ j , δ i , δ j , and δ ijdenote the global mean, standard deviation, and covariance of pixel blocks i and j, respectively.

[0145] Finally, the graph distribution sub-parameters of the mth stage (or any stage) can be expressed as the following formula (11).

[0146]

[0147] In the embodiment of the present application, after obtaining the graph distribution sub-parameters of the image to be evaluated at the mth stage, the image scale of the image to be evaluated can be adjusted (for example, downsampling or upsampling), and the image distribution sub-parameters of the m+1th stage (or at the m+1th image scale) can be obtained at the m+1th stage. The graph distribution sub-parameters of all M stages are obtained to obtain the first graph distribution parameters of the image to be evaluated.

[0148] refer to Figure 4 As shown in the flowchart, the trained model in S310 determines the second image distribution parameters of the reference image, which can be achieved through the following steps.

[0149] S3101b, establishing a graph structure of a reference image at an m-th image scale; m is a positive integer less than or equal to M, and the graph structure is used to represent the association relationship between multiple pixel blocks in the image;

[0150] S3102b, extracting image features based on the graph structure of the reference image at the mth image scale;

[0151] S3103b, determining the mth image distribution sub-parameter based on the image structure and image features of the reference image at the mth image scale;

[0152] S3104b, continue to establish the graph structure of the reference image at the m+1th image scale to determine the m+1th graph distribution sub-parameter, until the Mth graph distribution sub-parameter is obtained; the second graph distribution parameter includes M graph distribution sub-parameters.

[0153] It should be noted that, in one implementation, the processing of the reference image by the trained model can be performed simultaneously with the processing of the image to be evaluated, that is, the trained model can process the image to be evaluated and the reference image in parallel, and simultaneously obtain the graph distribution sub-parameters of the image to be evaluated in each stage, as well as the graph distribution sub-parameters of the reference image. Specifically, steps S3101a to S3104a and steps S3101b to S3104b are performed simultaneously, so that the first graph distribution parameter and the second graph distribution parameter can be quickly obtained, thereby quickly obtaining the image quality of the image to be evaluated.

[0154] In another implementation, the processing of the reference image by the trained model may not be performed simultaneously with the processing of the image to be evaluated. For example, the trained model may first process the image to be evaluated to obtain the first image distribution parameter, and then process the reference image to obtain the second image distribution parameter. Alternatively, the trained model may first process the reference image to obtain the second image distribution parameter, and then process the image to be evaluated to obtain the first image distribution parameter. In this way, the processing of the image to be evaluated and the processing of the reference image may share a set of processing modules, thereby reducing the complexity of the model.

[0155] In some embodiments, step S3101b establishes the graph structure of the reference image at the mth image scale, which can be implemented in the following manner:

[0156] Performing cropping processing on the reference image according to the mth rule to obtain a plurality of pixel blocks corresponding to the reference image;

[0157] Based on the association relationship between any two pixel blocks among the multiple pixel blocks, a graph structure of the reference image at the mth image scale is established.

[0158] It should be noted that the processing of the reference image in the above step S3101b can refer to the relevant description in the above step S3101a, and for the sake of brevity, it will not be repeated here.

[0159] In some embodiments, step S3102b extracts image features based on the graph structure of the reference image at the m-th image scale, which can be implemented in the following manner:

[0160] Determining, based on the graph structure of the reference image at the m-th image scale, neighboring pixel blocks associated with each pixel block in the reference image at the m-th image scale;

[0161] Based on the feature vector of each pixel block in the reference image at the m-th image scale and the feature vector of the neighboring pixel blocks associated with each pixel block, graph convolution processing and feature fusion processing are performed to obtain the image feature.

[0162] It should be noted that the processing of the reference image in the above step S3102b can refer to the relevant description in the above step S3102a, and for the sake of brevity, it will not be repeated here.

[0163] In some embodiments, step S3103b determines the mth image distribution sub-parameter based on the image structure and the image features of the reference image at the mth image scale, which can be implemented in the following manner:

[0164] Based on the graph structure of the reference image at the mth image scale, construct a plurality of clusters; each of the plurality of clusters includes a plurality of mutually related pixel blocks;

[0165] Based on the image features, the potential function product of the multiple clusters is calculated to obtain the mth graph distribution sub-parameter.

[0166] It should be noted that the processing of the reference image in the above step S3103b can refer to the relevant description in the above step S3103a, and for the sake of brevity, it will not be repeated here.

[0167] In the embodiment of the present application, after obtaining the image distribution sub-parameters of the reference image at the mth stage, the image scale of the reference image can be adjusted (for example, downsampling or upsampling), and the m+1th stage is entered to obtain the image distribution sub-parameters of the m+1th stage (or at the m+1th image scale). The image distribution sub-parameters of all M stages are obtained to obtain the second image distribution parameters of the reference image.

[0168] For example, reference Figure 5 Schematic diagram of a staged measurement framework shown in the figure. The trained model can process the image to be evaluated and the reference image separately. In each stage (stage 1 to stage 4), the trained model can perform graph construction, feature extraction and graph distribution measurement on the reference image and the image to be evaluated respectively to obtain the corresponding graph distribution sub-parameters. In addition, in the last module of each stage, in order to reduce the spatial dimension of the image while retaining key visual details, a downsampling operation is used to connect spatially adjacent 2×2 blocks. In the downsampling process, a convolution kernel of size 3×3 and stride 2×2 is used, which can effectively expand the receptive field of the network. This process can create a new adjacency matrix whose vertices represent image blocks of different scales at each stage.

[0169] In an embodiment of the present application, after obtaining the first image distribution parameters of the image to be evaluated and the second image distribution parameters of the reference image, the trained model can execute the above S320, that is, determine the perceptual distance between the image to be evaluated and the reference image based on the first image distribution parameters and the second image distribution parameters.

[0170] It should be noted that, when the first image distribution parameter and the second image distribution parameter may include image distribution sub-parameters at multiple image scales, the trained model can compare the image distribution sub-parameters of the image to be evaluated with the image distribution sub-parameters of the reference image at the same stage or the same image scale, and determine the perceptual sub-distance between the image distribution sub-parameters of the two at each stage or each image scale.

[0171] Exemplarily, as shown in the following formula (12), the perceptual sub-distance between the reference image and the image to be evaluated in the same stage can be the Hellinger distance.

[0172]

[0173] Among them, P x is the graph distribution sub-parameter of the image to be evaluated at the mth stage, P y is the graph distribution sub-parameter of the reference image at the mth stage.

[0174] Furthermore, the trained model combines the perceptual sub-distances of all M stages or M image scales to obtain the final perceptual distance. The perceptual distance can be calculated by the following formula (13).

[0175]

[0176] It should be understood that in the embodiment of the present application, the trained model compares the difference between the image distribution of the image to be evaluated and the reference image at different stages, and the perceptual distance measurement is applied to all stages of the above-mentioned image scale from shallow to deep.

[0177] The following is a detailed introduction to the training process of the trained image quality assessment model. Figure 8 A training flow diagram of a trained image quality assessment model provided in an embodiment of the present application. Figure 8 As shown, the following steps are included:

[0178] S810: Acquire a sample image and a sample reference image associated with the sample image.

[0179] It should be understood that in order to achieve the evaluation of image quality, a model for evaluating image quality can be trained in advance. The embodiment of the present application can perform supervised learning on the model through a large amount of labeled data.

[0180] Specifically, before training the model, sample data needs to be obtained. The sample data includes a sample image, a sample reference image associated with the sample image, and a sample label. The sample image can be a distorted image or a warped image, which is usually an image that has undergone some form of visual transformation or damage, such transformation or damage may include compression, adding noise, image camouflage, etc. The sample reference image can be an original, undistorted image, which is usually considered to be of high quality.

[0181] In addition, the sample label may be a quality score of the sample image, for example, the sample label may be a mean opinion score (MOS) of a sample reference image in the sample image set.

[0182] In the embodiments provided in the present application, sample labels may be manually annotated or obtained by other means, and the embodiments of the present application are not limited to this.

[0183] S820, inputting the sample image and the sample reference image into the image quality assessment model to be trained, and outputting a first output result through the image quality assessment model to be trained; the first output result represents the perceived distance between the sample image and the sample reference image.

[0184] It should be understood that in the embodiments provided in the present application, when conducting training, it is also necessary to adopt deep learning technology to build an image quality assessment model to be trained.

[0185] For the sake of convenience, the term “model to be trained” is used below to refer to the image quality assessment model to be trained.

[0186] It is understandable that the model to be trained is an initial model architecture for evaluating image quality built through deep learning technology. For example, the model to be trained can be a neural network model based on multiple convolutional layers.

[0187] In the embodiment of the present application, the sample image and the sample reference image obtained in step 810 can be input into the model to be trained, and processed by the model to be trained to obtain a first output result. It should be noted that the processing process of the model to be trained refers to the description of the above embodiment, and for the sake of brevity, it will not be repeated here.

[0188] The first output result may be a quality score of the predicted sample image, that is, the network's evaluation of the image quality, and specifically may be a perceptual distance between the sample image and the sample reference image.

[0189] S830: Determine a first difference value between the first output result and the sample label through a target loss function.

[0190] In the embodiment provided in the present application, since the model to be trained is an initial model, the quality score of the sample image in the first output result obtained by the model to be trained is not the actual quality score of the sample image. Instead, it is the quality score of the sample image indicated by the sample label. Therefore, there is a difference between the sample label and the first output result.

[0191] Here, the difference between the sample label and the first output result can be quantified by a target loss function. Exemplarily, the target loss function can be expressed as the following formula (14).

[0192]

[0193] Among them, D(P x' ,P y') is the perceptual distance between the sample image x' and the sample reference image y', Mos(P x' ) is the sample label.

[0194] It is understandable that the target loss function can be a function that estimates the degree of inconsistency between the predicted value of the model to be trained and the true value. Generally, the greater the difference between the sample label and the first output result, the greater the calculated first difference value. The first difference value can characterize the recognition effect of the model to be trained to a certain extent.

[0195] S840: Train the image quality assessment model to be trained based on the first difference value until a training end condition is met, thereby obtaining a trained image quality assessment model.

[0196] In the embodiment provided in the present application, after the first difference value is determined, the model to be trained can be iteratively trained according to the first difference value, and finally a trained model is obtained through training.

[0197] Specifically, the parameters in the model to be trained can be adjusted according to the first difference value; that is, when the recognition model to be trained is trained using the target loss function, the model parameters of the model to be trained are adjusted by back propagating through the first difference value. Further, the sample image is processed based on the adjusted model to be trained to obtain a second output result; the second output result can be a predicted quality score of the sample image, representing the perceived distance between the sample image and the sample reference image. Here, the adjusted model to be trained is better at predicting the quality score of the sample image than the unadjusted model to be trained, so that the quality score in the second output result is more accurate than the quality score in the first output result.

[0198] Further, continue to determine the second difference value between the sample label and the second output result based on the target loss function; and, based on the second difference value, train the adjusted model to be trained until the training end condition is met, and use the trained adjusted model to be trained as the final trained model.

[0199] The training process of the trained image quality assessment model is obtained through iterative training; the iterative process is to continuously adjust the model parameters of the model through the first difference value, the second difference value, etc., until the adjusted model to be trained converges, or the obtained Nth difference value is within the preset threshold range, and it is determined that the training end condition is met; here, N is an integer greater than or equal to 1. The adjusted model to be trained obtained at this time is the final trained model.

[0200] In simple terms, the model parameters of the model to be trained can be initialized, and then the sample image and the sample reference image are used as the input of the model to be trained, and an output result is obtained in the output layer of the model to be trained; the purpose of training the model to be trained is to make the output result of the model infinitely close to the sample label corresponding to the sample image. During the initial training, since the model parameters are initialized, the output result is quite different from the label data. Therefore, after each output result is obtained, the output result and the sample label can be substituted into the pre-set target loss function to calculate a difference value, and then the difference value is used to update the model parameters. After repeatedly iterating the above process using a large amount of sample data, a set of model parameters that can make the output result of the model very close to the sample label will be obtained.

[0201] In summary, the image quality evaluation method provided in the embodiment of the present application can be understood as a full-reference image quality evaluation method based on the graph distribution similarity index. The main advantage of this method is that it can fully capture the complex relationship between the feature distributions of different scales and regions within the image, making it applicable to various types of distortion, including color distortion and texture structure distortion. By using a unified measurement method, it can accurately measure the perceived distance, that is, the difference in image quality, thereby achieving more accurate image quality prediction. This method is of great significance to a variety of application fields, including but not limited to image compression, enhancement, and restoration, and helps ensure that images can provide reliable visual effects in different scenarios.

[0202] The method provided in the embodiment of the present application is described in detail below in combination with actual application scenarios.

[0203] The method provided in the embodiment of the present application designs a hierarchical image quality evaluation model. The image quality evaluation can obtain the characteristic distribution of the input image content at multiple scales, where the image is presented in the form of a graphical representation. The visual graph is automatically generated based on the image content, and can establish an overall perceptual connection for irregular areas, which is more general and effective for visual perception purposes. Furthermore, the model can decompose the graph representation of the image into clusters, and then calculate the product of the potential functions of the clusters to obtain the joint probability distribution of the undirected graph, which helps to understand the image content more comprehensively. Finally, the model can compare the distribution distance between the graph distributions of distorted images and reference images at different stages to obtain the image quality evaluation results, which has strong innovation and robustness, and improves the accuracy of image quality assessment.

[0204] The overall process of this solution is as follows: It should be noted that the following steps S1 to S5 are the model training process, and S6 is the model reasoning process.

[0205] S1: Obtain a distorted image and its reference image.

[0206] It should be noted that a distorted image refers to an image that has undergone some form of visual transformation or damage, which may include compression, adding noise, image camouflage, etc. The reference image is the original, undistorted image, which is usually considered to be of high quality. For example, in the image compression process, the image before compression is the reference image, and the image after compression is the distorted image.

[0207] S2: Construct the graph structure of the distorted image and the graph structure of the reference image.

[0208] In this step, refer to Fig. 9 The model architecture diagram shown in the figure shows that the model to be trained can transform the distorted image and the reference image into a graph structure for further image quality assessment. This process mainly includes the following key steps:

[0209] S21: Pixel block division.

[0210] It should be noted that before pixel block division, the acquired distorted image and reference image can be converted into an RGB color model, and based on the RGB color model, the pixel values ​​of the three channels R, G, and B of the distorted image and the reference image can be acquired. Subsequent image processing (such as feature extraction) can be performed based on the RGB color model.

[0211] In an embodiment of the present application, the distorted image and the reference image are divided into a plurality of pixel blocks (also referred to as blocks for short), each of which represents a small area in the image. These blocks are usually a collection of adjacent pixels, which can be square or rectangular, and the size and shape of the specific blocks can be adjusted according to specific needs and evaluation tasks. In addition, this scheme applies a CNN to crop the reference image and the distorted image of size H×W into multiple blocks according to the same scheme, and extract the regional features of each block.

[0212] S22: Graph structure establishment.

[0213] It should be understood that in the embodiment of the present application, corresponding graph structures need to be established for the reference image and the distorted image respectively.

[0214] In some embodiments, the graph structure of the image can be represented by an undirected graph model G={V, E}, where V is a vertex set in the undirected graph model G, and E is an edge set in the undirected graph model G.

[0215] It should be noted that each block in the reference image and the distorted image can be represented as a vertex v, and the correlation between blocks can be represented as an edge e. It should be understood that these edges can represent the similarity, distance or other correlation information between blocks.

[0216] In some embodiments, the graph structure of the image can be represented by an adjacency matrix A. A can be an N*N dimensional matrix, where N is the total number of blocks divided in the image. ij It can represent the degree of correlation between pixel block i and pixel block j.

[0217] It should be noted that the undirected graph model G and the adjacency matrix A can be converted to each other. The adjacency matrix A is a variable automatically learned from the graph structure.

[0218] S3: Encode the image represented by the graph structure to extract relevant features of the image.

[0219] In the examples of this application, refer to Fig. 9 As shown in Figure 1, after the graph structure is obtained through graph construction, the GCN encoder can be used to extract the features of the image represented by the graph structure. The convolution operation in GCN is implemented through aggregation.

[0220] Specifically, the graph convolution operation in GCN can refer to formula (1).

[0221]

[0222] in, and W represents the feature vector of pixel blocks i and j in the lth layer of the GCN network, that is, the feature vector of pixel blocks i and j. i and W j represents the weight matrix. represents the neighboring pixel blocks of pixel block i, φ (l) () represents an aggregate function.

[0223] In some embodiments, the aggregation function φ (l) () The maximum relative graph convolution method can be used to aggregate neighborhood relative features. For details, please refer to formula (2).

[0224]

[0225] It should be understood that the feature vector of pixel block i in the lth layer is subtracted from the feature vector of each neighboring pixel block of pixel block i, the maximum value in each dimension is obtained, and the feature vector of pixel block i in the l+1th layer is obtained by combining them.

[0226] It should be noted that the reference Fig. 9 As shown in Figure 1, after the graph convolution operation, the graph representation of the image is converted back to the grid matrix representation to achieve graph deconstruction. Then, the conversion is successfully achieved by applying MLP to further extract features.

[0227] Specifically, the feature vector output by the l+1th layer is and the lth layer feature vector After the connection, the graph structure is passed through and features are further extracted through MLP to obtain fused or aggregated features, thereby obtaining the target feature vector of each pixel block. After the above graph convolution and feature fusion processing, the target feature vector of each pixel block reflects the superposition value of the vertex and its surrounding environment.

[0228] It can be understood that all vertices in the undirected graph model G can achieve aggregate update of features through the above method. The updated undirected graph model G can be represented by the following formula (3).

[0229] G (l+1) =F(G (l) ,W (l) ) (3)

[0230] Among them, G (l) is the graph structure of the lth layer, G (l+1) The graph structure of the l+1th layer. (l) Represents G (l) The weight matrix of .

[0231] S4: Graph distribution measurements.

[0232] In this step, the perceptual distance between the distorted image and the reference image is evaluated based on the difference between the graph distribution parameters of the distorted image and the reference image to further predict the image quality. Specifically, this step includes the following key processes:

[0233] S41: Representation of graph distribution parameters.

[0234] In this step, the embodiment of the present application converts the graph feature representation of the distorted image and the reference image into graph distribution parameters. This representation form can provide a more detailed understanding of the feature distribution of different scales and regions within the image.

[0235] Given that adjacent image blocks exhibit non-causal interconnections, this application constructs their correlations through a probabilistic graph model based on an undirected graph representation. It is understandable that the model to be trained can calculate the graph distribution parameters based on the probabilistic graph model algorithm of G = {V, E}. It should be noted that the target feature vectors X = x1, x2, ..., x N As a random variable associated with the vertex set V. It can be understood that each x i Corresponding to a vertex, that is, a pixel block.

[0236] It should be noted that the x associated with each vertex i, is the value obtained based on the “image distribution” processing. “Image distribution” takes into account that the human visual system receives image signals based on distribution rather than single pixels. Therefore, each image block is regarded as a certain distribution of the whole, which can more accurately reflect its characteristic attributes. In a specific example, the random variable x i The value of refers to the characteristic state assigned to the block, such as a certain brightness or texture distribution, which enables the random variable to effectively reflect the overall properties of the pixels within the block.

[0237] In the examples of this application, refer to Fig. 9 As shown, the model to be trained can first decompose the graph representation of the image into clusters, and then calculate the product of the potential functions of the clusters to obtain the joint probability distribution of the undirected graph, which helps to understand the image content more comprehensively.

[0238] The graph distribution parameter of the image can be expressed by the following formula (4).

[0239]

[0240] Wherein, C represents the set of all clusters, and Q represents any cluster in C. Z represents a normalization function. Specifically, Z can refer to the following formula (5).

[0241]

[0242] In addition, E(·) represents a potential energy function. In an embodiment of the present application, the potential energy function of any cluster Q may include a first component and a second component, wherein the first component may be used to determine the perceived distance between the feature vector of a pixel in each pixel block in the cluster Q and the target feature vector of the pixel block, and the second component is used to determine the perceived distance between the target feature vectors of any two pixel blocks in the cluster Q.

[0243] Exemplarily, the potential energy function can be expressed as the following formula (6).

[0244]

[0245] in, is the first component, is the second component.

[0246] In the first component, ψ1(x i ) can represent the feature vector of each pixel in the i-th pixel block in the Q-th cluster and the target feature vector x of the i-th pixel block i The difference between them (which can also be understood as perceived cost or perceived distance).

[0247] In some embodiments, ψ1(x i ) can be calculated by the following formula (7).

[0248]

[0249] in, is the pixel set of pixel block i, and d is any pixel in pixel block i. f(d) represents the feature vector of pixel d, which is the internal perception feature extracted by neural network. C(f(d),x i ) is the squared difference cost of the perceptual feature, reflecting the difference between the feature vector f(d) of each pixel and the target feature vector x corresponding to pixel block i. i The difference between.

[0250] For example, C(f(d),x i ) can be specifically That is to say, the above formula (7) can be replaced by: in, is the target feature vector x i The corresponding perceptual feature center can be specifically understood as the mean value of the pixel features in pixel block i.

[0251] In addition, in the potential energy function shown in formula (6), the second component ψ2(x i ,x j ) can be used to measure the target feature vector x of pixel block i in the Qth cluster i and the target feature vector x of pixel block j j The difference between them (which can also be understood as perceived cost or perceived distance).

[0252] In some embodiments, ψ2(x i ,x j ) can be calculated by the following formula (8).

[0253] ψ2(x i ,x j )=ρ(x i ,x j )·s(f i ,f j ) (8)

[0254] Among them, f i represents the initial feature vector of pixel block i, f j represents the initial feature vector of pixel block j. i ,x j ) is a penalty factor used to capture the relationship between different image distributions. Specifically, when pixel blocks i and j with similar attributes have different random variables, a penalty is required. The embodiment of the present application can use the following formula (9) to calculate ρ(x i ,x j ).

[0255] ρ(x i ,x j )=|x i -x j |2 (9)

[0256] It can be understood that the penalty factor can be expressed by the binary norm between the initial eigenvectors.

[0257] In addition, s(f i ,f j ) can represent the variability between the initial feature vectors. The present application embodiment uses the structural similarity index measure (SSIM) to calculate the variability based on the mean, standard deviation and covariance values ​​corresponding to each pixel block. Specifically, s(f i ,f j ) can be calculated using the following formula (10).

[0258]

[0259] Among them, μ i , μ j , δ i , δ j , and δ ij denote the global mean, standard deviation, and covariance of pixel blocks i and j, respectively.

[0260] Finally, the graph distribution parameters of the image can be expressed as the following formula (11).

[0261]

[0262] It should be noted that the above-mentioned method of graph distribution representation is a general framework, and the above-mentioned formal formula is just a special case. This special case realizes the distribution representation of the graph by decomposing the structure of the graph and calculating the product of the relevant potential functions. It is worth noting that the choice of this special case depends on the specific application field and problem, and the general framework allows the use of other suitable measurement methods to capture the complexity of the distribution of image features.

[0263] S42: Perceptual distance measurement.

[0264] In this step, the trained model compares the differences between the graph distribution parameters of the distorted image and the reference image, and uses a unified metric to determine the perceptual distance between them, that is, the difference in image quality.

[0265] The perceptual distance between the reference image and the image to be evaluated may be the Hellinger distance, with specific reference to the following formula (12).

[0266]

[0267] Among them, P x is the image distribution parameter of the image to be evaluated, P y is the distribution parameter of the reference image.

[0268] This metric is an optional choice for this solution, but it is worth noting that there are other metrics that can be used to measure different aspects of image quality, depending on the specific application requirements and problem context. This step is a critical step in evaluating image quality, as it helps understand the degree of distortion and provides a reliable metric for image quality assessment.

[0269] It should be noted that this application can use a phased approach to evaluate image quality. Fig. 9 Only one of the stages is shown. Specifically, the model to be trained can gradually reduce the initial high-resolution image in a staged manner. Figure 5 The figure shows a schematic diagram of a phased measurement framework, where each phase consists of a series of modules.

[0270] Specifically, in each stage, the model to be trained can first input the image into the graph construction module, and then construct the graph of the image through the graph construction module, that is, the graph structure of the image to be evaluated is established through the graph construction module. Then, the trained model can input the obtained graph structure into the feature extraction module, and then extract the features of the graph structure through the feature extraction module to obtain the image features. Finally, the model can input the graph structure output by the graph construction module and the image features output by the feature extraction module into the graph distribution measurement module, and then process them through the graph distribution measurement module to obtain the graph distribution parameters of this stage.

[0271] It should be noted that before each graph convolution operation, the graph representation of the image is reconstructed to capture more fine-grained information. Subsequently, after each graph convolution operation, the graph representation of the image is converted back to a grid matrix representation. Fig. 9 As shown in the figure, in the last block of each stage, in order to reduce the spatial dimension of the image while retaining key visual details, a downsampling operation is used to connect spatially adjacent 2×2 blocks. In the downsampling process, a convolution kernel of size 3×3 and stride 2×2 is used, which effectively expands the receptive field of the network. This process can create a new adjacency matrix whose vertices represent pixel blocks of different scales at each stage.

[0272] In the embodiment of the present application, the perceptual distance measurement can be applied to all stages of the model from shallow to deep. A hierarchical network is used to perform graph encoding and distribution representation in stages. In the embodiment of the present application, referring to the following formula (13), the model can aggregate the distribution measurement results of all M stages to obtain the predicted perceptual quality:

[0273]

[0274] S5: Use the training data to train the neural network model of this solution.

[0275] In this step, the proposed scheme uses a large amount of labeled data to supervise the network. Specifically, the reference image and the corresponding distorted image are taken as input and fed into the network together with MOS as a label. At the output layer of the network, a predicted quality score is obtained, which is the network's assessment of the image quality.

[0276] Specifically, this scheme minimizes the error between the prediction and the actual situation according to the goal of quality assessment; therefore, the loss function of this scheme is defined as follows:

[0277]

[0278] Among them, D(P x' ,P y' ) is the perceptual distance between the sample image x' and the sample reference image y', Mos(P x' ) is a subjective quality rating label.

[0279] S6: Apply the trained image quality assessment model to implement a full-reference image quality assessment method based on graph distribution similarity index.

[0280] First, the distorted image to be evaluated and its corresponding reference image are provided to the trained neural network model. These two images constitute the input of the image quality assessment task. Then, the model extracts features from the two input images through the forward propagation process of the convolutional neural network. After feature extraction, the model uses the graph distribution similarity index method to calculate the graph distribution similarity between the distorted image and the reference image at different stages. This process involves comparing the feature distribution of the image at various stages to measure the difference between them. Based on the results of the graph distribution similarity calculation, the model calculates the perceptual distance between the distorted image and the reference image through a unified metric method. Finally, the model converts the perceptual distance into a quality score that can represent the image quality of the distorted image. Lower scores usually correspond to poorer image quality, while higher scores correspond to better image quality.

[0281] The above specific implementation only expresses one implementation form of the present invention, and its description is relatively specific and detailed, but this implementation does not represent a limitation on the scope of the invention patent. It should be particularly noted that other technicians in this field can still make several modifications and improvements without departing from the concept of the present invention. Such currently unpredictable alternatives or modifications to existing solutions are all within the scope of protection of the present invention.

[0282] The embodiment of the present application proposes a full-reference image quality evaluation method based on the graph distribution similarity index. The main advantage of this method is that it can fully capture the complex relationship between the feature distributions of different scales and regions within the image, making it applicable to various types of distortion, including color distortion and texture structure distortion. By using a unified measurement method, it can accurately measure the perceived distance, that is, the difference in image quality, thereby achieving more accurate image quality prediction. This method is of great significance to a variety of application fields, including but not limited to image compression, enhancement and restoration, and helps ensure that images can provide reliable visual effects in different scenarios.

[0283] It should be noted that the above specific implementation only expresses one implementation form of the present invention, and its description is relatively specific and detailed, but the implementation does not represent a limitation on the scope of the invention patent. It should be particularly noted that for other technicians in this field, several modifications and improvements can still be made without departing from the concept of the present invention. Such currently unpredictable alternatives or modifications to existing solutions are all within the scope of protection of the present invention.

[0284] An embodiment of the present application also provides an image quality evaluation device, referring to Fig.10 As shown, the image quality evaluation device comprises:

[0285] An acquisition unit 1010 is configured to acquire an image to be evaluated and a reference image associated with the image to be evaluated;

[0286] The processing unit 1020 is configured to input the image to be evaluated and the reference image into a trained image quality assessment model, and output a perceptual distance between the image to be evaluated and the reference image through the trained image quality assessment model, wherein the perceptual distance is used to characterize the image quality of the image to be evaluated;

[0287] Among them, the trained image quality assessment model is used to determine the first image distribution parameter of the image to be evaluated, and to determine the second image distribution parameter of the reference image; and based on the first image distribution parameter and the second image distribution parameter, the perceptual distance between the image to be evaluated and the reference image is determined; the image distribution parameter characterizes the characteristic distribution of multiple image scales and / or multiple pixel blocks of the image.

[0288] In some embodiments, the trained image quality assessment model includes a graph construction module, a feature extraction module, and a graph distribution measurement module; wherein the graph construction module is configured to establish a graph structure of the image to be evaluated at the mth image scale; m is a positive integer less than or equal to M, M is the total number of multiple image scales; M is an integer greater than 1, and the graph structure is used to characterize the association relationship between multiple pixel blocks in the image;

[0289] The feature extraction module is configured to extract image features based on the graph structure of the image to be evaluated at the mth image scale;

[0290] A graph distribution measurement module, configured to determine an i-th graph distribution sub-parameter based on the graph structure of the image to be evaluated at the m-th image scale and the image features;

[0291] The graph construction module is further configured to continue to establish the graph structure of the image to be evaluated at the m+1th image scale. The feature extraction module is further configured to extract image features based on the graph structure of the image to be evaluated at the m+1th image scale. The graph distribution measurement module is further configured to determine the m+1th graph distribution sub-parameter until the Mth graph distribution sub-parameter is obtained; the first graph distribution parameter includes M graph distribution sub-parameters.

[0292] In some embodiments, the graph construction module is further configured to crop the image to be evaluated according to the mth rule to obtain a plurality of pixel blocks corresponding to the image to be evaluated; and establish a graph structure of the image to be evaluated at the mth image scale based on the association relationship between any two pixel blocks among the plurality of pixel blocks.

[0293] In some embodiments, the feature extraction module is further configured to determine the neighboring pixel blocks associated with each pixel block in the image to be evaluated at the mth image scale based on the graph structure of the image to be evaluated at the mth image scale; perform graph convolution processing and feature fusion processing based on the initial feature vector of each pixel block in the image to be evaluated at the mth image scale, and the initial feature vector of the neighboring pixel blocks associated with each pixel block to obtain the target feature vector of each pixel block, and the image features of the image to be evaluated include the target feature vector of each pixel block in the image to be evaluated.

[0294] In some embodiments, the graph distribution measurement module is further configured to construct a plurality of clusters based on the graph structure of the image to be evaluated at the mth image scale; each of the plurality of clusters includes a plurality of mutually related pixel blocks; and based on the image features, the potential function product of the plurality of clusters is calculated to obtain the mth graph distribution sub-parameter.

[0295] In some embodiments, the graph construction module is further configured to establish a graph structure of the reference image at the mth image scale; m is a positive integer less than or equal to M, M is the total number of multiple image scales; M is an integer greater than 1, and the graph structure is used to represent the association relationship between multiple pixel blocks in the image;

[0296] The feature extraction module is further configured to extract image features based on the graph structure of the reference image at the mth image scale;

[0297] The image distribution measurement module is further configured to determine an mth image distribution sub-parameter based on the image structure and the image features of the reference image at the mth image scale;

[0298] The graph construction module is further configured to continue to establish the graph structure of the reference image at the m+1th image scale; the feature extraction module is further configured to extract image features based on the graph structure of the reference image at the m+1th image scale; the graph distribution measurement module is further configured to determine the m+1th graph distribution sub-parameter based on the graph structure and the image features of the reference image at the m+1th image scale until the Mth graph distribution sub-parameter is obtained; the second graph distribution parameter includes M graph distribution sub-parameters.

[0299] In some embodiments, the graph construction module is further configured to crop the reference image according to the mth rule to obtain multiple pixel blocks corresponding to the reference image; and establish the graph structure of the reference image at the mth image scale based on the association relationship between any two pixel blocks among the multiple pixel blocks.

[0300] In some embodiments, the feature extraction module is further configured to determine the neighboring pixel blocks associated with each pixel block in the reference image at the mth image scale based on the graph structure of the reference image at the mth image scale; perform graph convolution processing and feature fusion processing based on the feature vector of each pixel block in the reference image at the mth image scale, and the feature vector of the neighboring pixel blocks associated with each pixel block to obtain the image feature.

[0301] In some embodiments, the graph distribution measurement module is further configured to construct a plurality of clusters based on the graph structure of the reference image at the mth image scale; each of the plurality of clusters includes a plurality of mutually related pixel blocks; and based on the image features, the potential function product of the plurality of clusters is calculated to obtain the mth graph distribution sub-parameter.

[0302] In some embodiments, the potential function of each cluster in the multiple clusters is composed of a first component and a second component; wherein the first component is used to determine the difference between the feature vector of the pixels in each pixel block in the cluster and the target feature vector of the pixel block; and the second component is used to determine the difference between any two pixel blocks in the cluster.

[0303] In some embodiments, the graph structure is an undirected graph model, and / or an adjacency matrix.

[0304] In some embodiments, the image quality assessment device also includes a training unit, which is configured to: obtain a sample image and a sample reference image associated with the sample image; input the sample image and the sample reference image into an image quality assessment model to be trained, and output a first output result through the image quality assessment model to be trained; the first output result represents the perceptual distance between the sample image and the sample reference image; determine a first difference value between the first output result and the sample label through a target loss function; and train the image quality assessment model to be trained based on the first difference value until a training end condition is met, thereby obtaining the trained image quality assessment model.

[0305] Those skilled in the art should understand that the relevant description of the above-mentioned image quality assessment device in the embodiment of the present application can be understood by referring to the relevant description of the image quality assessment method in the embodiment of the present application.

[0306] Fig.11 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. Fig.11 The electronic device 1100 shown includes a processor 1510, and the processor 1110 can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0307] Alternatively, if Fig.11 As shown, the electronic device 1100 may further include a memory 1120. The processor 1110 may call and run a computer program from the memory 1120 to implement the method in the embodiment of the present application.

[0308] The memory 1120 may be a separate device independent of the processor 1110 , or may be integrated into the processor 1110 .

[0309] Alternatively, if Fig.11 As shown, the electronic device 1100 may further include a transceiver 1130, and the processor 1110 may control the transceiver 1130 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices.

[0310] The transceiver 1130 may include a transmitter and a receiver. The transceiver 1130 may further include an antenna, and the number of the antennas may be one or more.

[0311] The present application also provides a computer storage medium, specifically a computer-readable storage medium, on which computer instructions are stored. When the computer storage medium is located in a communication device, the computer instructions are executed by a processor to implement any step of the product recommendation method in the present application.

[0312] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0313] The computer program product can be applied to the electronic device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0314] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0315] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0316] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or at least two units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0317] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0318] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0319] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0320] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for evaluating image quality, characterized in that: The method comprises: Acquire an image to be evaluated and a reference image associated with the image to be evaluated; Inputting the image to be evaluated and the reference image into a trained image quality assessment model, and outputting a perceptual distance between the image to be evaluated and the reference image through the trained image quality assessment model, wherein the perceptual distance is used to characterize the image quality of the image to be evaluated; Among them, the trained image quality assessment model is used to determine the first image distribution parameter of the image to be evaluated, and to determine the second image distribution parameter of the reference image; and based on the first image distribution parameter and the second image distribution parameter, the perceptual distance between the image to be evaluated and the reference image is determined; the image distribution parameter characterizes the characteristic distribution of multiple image scales and / or multiple pixel blocks of the image.

2. The method according to claim 1, characterized in that The determining of the first image distribution parameter of the image to be evaluated comprises: Establishing a graph structure of the image to be evaluated at the mth image scale; m is a positive integer less than or equal to M, M is the total number of multiple image scales; M is an integer greater than 1, and the graph structure is used to represent the association relationship between multiple pixel blocks in the image; Extracting image features based on the graph structure of the image to be evaluated at the m-th image scale; Determining an i-th image distribution sub-parameter based on the image structure of the image to be evaluated at the m-th image scale and the image features; Continue to establish the graph structure of the image to be evaluated at the m+1th image scale to determine the m+1th graph distribution sub-parameter, until the Mth graph distribution sub-parameter is obtained; the first graph distribution parameter includes M graph distribution sub-parameters.

3. The method according to claim 2, characterized in that The step of establishing the graph structure of the image to be evaluated at the mth image scale includes: Performing cropping processing on the image to be evaluated according to the mth rule to obtain a plurality of pixel blocks corresponding to the image to be evaluated; Based on the association relationship between any two pixel blocks in the multiple pixel blocks, a graph structure of the image to be evaluated at the mth image scale is established.

4. The method according to claim 2, characterized in that: The step of extracting image features of the image to be evaluated based on the graph structure of the image to be evaluated at the m-th image scale includes: Determining neighboring pixel blocks associated with each pixel block in the image to be evaluated at the m-th image scale based on the graph structure of the image to be evaluated at the m-th image scale; Based on the initial feature vector of each pixel block in the image to be evaluated at the m-th image scale and the initial feature vectors of neighboring pixel blocks associated with each pixel block, graph convolution processing and feature fusion processing are performed to obtain a target feature vector of each pixel block, and the image features of the image to be evaluated include the target feature vector of each pixel block in the image to be evaluated.

5. The method according to claim 2, characterized in that: The determining the mth image distribution sub-parameter based on the image structure of the image to be evaluated at the mth image scale and the image feature includes: Based on the graph structure of the image to be evaluated at the mth image scale, construct a plurality of clusters; each of the plurality of clusters includes a plurality of mutually related pixel blocks; Based on the image features, the potential function product of the multiple clusters is calculated to obtain the mth graph distribution sub-parameter.

6. The method according to claim 2, characterized in that The determining of the second image distribution parameter of the reference image comprises: Establishing a graph structure of the reference image at the mth image scale; m is a positive integer less than or equal to M, M is the total number of multiple image scales; M is an integer greater than 1, and the graph structure is used to represent the association relationship between multiple pixel blocks in the image; Extracting image features based on the graph structure of the reference image at the m-th image scale; Determining an mth image distribution sub-parameter based on the image structure of the reference image at the mth image scale and the image feature; Continue to establish the graph structure of the reference image at the m+1th image scale to determine the m+1th graph distribution sub-parameter until the Mth graph distribution sub-parameter is obtained; the second graph distribution parameter includes M graph distribution sub-parameters.

7. The method according to claim 6, characterized in that The step of establishing the graph structure of the reference image at the mth image scale includes: Performing cropping processing on the reference image according to the mth rule to obtain a plurality of pixel blocks corresponding to the reference image; Based on the association relationship between any two pixel blocks among the multiple pixel blocks, a graph structure of the reference image at the m-th image scale is established.

8. The method according to claim 6, characterized in that The extracting image features based on the graph structure of the reference image at the m-th image scale includes: Determining, based on the graph structure of the reference image at the m-th image scale, neighboring pixel blocks associated with each pixel block in the reference image at the m-th image scale; Based on the feature vector of each pixel block in the reference image at the m-th image scale and the feature vector of the neighboring pixel blocks associated with each pixel block, graph convolution processing and feature fusion processing are performed to obtain the image feature.

9. The method according to claim 6, characterized in that The determining the mth image distribution sub-parameter based on the image structure of the reference image at the mth image scale and the image feature comprises: Based on the graph structure of the reference image at the mth image scale, construct a plurality of clusters; each of the plurality of clusters includes a plurality of mutually related pixel blocks; Based on the image features, the potential function product of the multiple clusters is calculated to obtain the mth graph distribution sub-parameter.

10. The method according to claim 5 or 9, characterized in that: The potential function of each of the multiple clusters consists of a first component and a second component; wherein the first component is used to determine the difference between the feature vector of the pixels in each pixel block in the cluster and the target feature vector of the pixel block; and the second component is used to determine the difference between any two pixel blocks in the cluster.

11. The method according to any one of claims 2 to 9, characterized in that: The graph structure is an undirected graph model and / or an adjacency matrix.

12. The method according to any one of claims 1 to 9, characterized in that: The training process of the trained image quality assessment model is as follows: Acquire a sample image and a sample reference image associated with the sample image; Inputting the sample image and the sample reference image into an image quality assessment model to be trained, and outputting a first output result through the image quality assessment model to be trained; The first output result represents the perceived distance between the sample image and the sample reference image; Determining a first difference value between the first output result and the sample label by using a target loss function; The image quality assessment model to be trained is trained based on the first difference value until a training end condition is met, thereby obtaining the trained image quality assessment model.

13. An image quality evaluation device, characterized in that: The device comprises: an acquisition unit, configured to acquire the image to be evaluated and a reference image associated with the image to be evaluated; a processing unit configured to input the image to be evaluated and the reference image into a trained image quality evaluation model, and output a perceptual distance between the image to be evaluated and the reference image through the trained image quality evaluation model, wherein the perceptual distance is used to characterize the image quality of the image to be evaluated; Among them, the trained image quality assessment model is used to determine the first image distribution parameter of the image to be evaluated, and to determine the second image distribution parameter of the reference image; and based on the first image distribution parameter and the second image distribution parameter, the perceptual distance between the image to be evaluated and the reference image is determined; the image distribution parameter characterizes the characteristic distribution of multiple image scales and / or multiple pixel blocks of the image.

14. A server, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program causing a computer to execute the method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 12.

Images