Cross multi-scale based no-reference image quality assessment method

Abstract

The application discloses a cross multi-scale based no-reference image quality evaluation method, comprising the following steps: step 1, obtaining a training sample set B and a test sample set C; step 2, constructing a network model S based on cross multi-scale no-reference image quality evaluation; step 3, iteratively training the network model S based on cross multi-scale no-reference image quality evaluation; and step 4, obtaining a no-reference quality evaluation result Q of an image: taking the test sample set C as an input of the no-reference image quality evaluation network model S based on the cross multi-scale network and performing forward reasoning to obtain a quality prediction score of each test sample, so as to verify the model effect. The application utilizes the local and non-local semantic information of an image and the ability of image pyramid to extract different level information, and solves the problem that a single network model in the prior art cannot simultaneously consider global distortion and local distortion when facing a complex distorted image.

Description

Technical Field

[0001] This invention belongs to the field of image quality assessment technology, specifically relating to a cross-multi-scale no-reference image quality assessment method. Background Technology

[0002] Digital images often experience varying degrees of distortion during acquisition, compression, transmission, and retrieval. This distortion reduces image quality, causing discrepancies between the information conveyed and the actual information, leading to misinterpretations when humans perceive the true meaning of the digital image. Therefore, accurately evaluating the quality of digital images is crucial for deriving this true information. Digital image quality evaluation has numerous applications, such as evaluating the quality of acquired digital images to select suitable hardware; evaluating the quality of encoded and compressed digital images to assess encoding and compression algorithms; and evaluating the quality of denoised digital images to determine the effectiveness of denoising algorithms.

[0003] The quality of a digital image is characterized by its degree of distortion; therefore, the quality of a digital image is obtained by evaluating its degree of distortion. This study uses a quality score to represent the degree of distortion; a lower quality score indicates lower image quality, and a higher quality score indicates higher image quality. Digital image quality assessment is divided into subjective and objective quality assessment. Subjective quality assessment provides a quality score based on human observation. This method is highly reliable, but suffers from high cost and poor stability. Objective quality assessment, on the other hand, automatically provides image quality scores through designed algorithms, offering advantages such as low cost and high stability, thus becoming a key focus in digital image quality assessment. Objective digital image quality assessment is further divided into full-reference, partial-reference, and no-reference methods. Full-reference digital image quality assessment obtains a quality score by comparing the original image with its corresponding distorted image; partial-reference digital image quality assessment obtains a quality score by comparing extracted information from the original image with information from its corresponding distorted image; and no-reference digital image quality assessment requires no information from a reference image, predicting the quality score solely based on the information provided by the distorted image itself. In real-world scenarios, obtaining original image information is difficult; therefore, full-reference and partial-reference digital image quality assessments are challenging to apply in practice. In contrast, referenceless digital image quality assessment only requires algorithm design and can provide image quality scores without requiring information from the original image. Consequently, referenceless digital image quality assessment has become a hot topic in image quality assessment and has significant practical application value.

[0004] The patent application with publication number CN 115205658 A, entitled "A Referenceless Comprehensive Perception Method for Evaluating the Quality of Panoramic Images," provides a more accurate evaluation of panoramic images. However, its drawback lies in the fact that a single network model cannot simultaneously address both global and local distortions in complex distorted images. As the network depth increases, the receptive field of the extracted image features becomes larger and the semantic depth becomes richer, but the ability to represent local distortions weakens, leading to a decrease in the accuracy of fine-grained image evaluation.

[0005] The patent application, CN 115272203 A, entitled "A Deep Learning-Based Method for Evaluating the Quality of No-Reference Images," not only solves the problem of traditional methods failing to effectively evaluate image quality scores when faced with multiple types of distortion, but also addresses the issues of unclear physical meaning of hierarchical features and underutilization of hierarchical feature information that sometimes arise with deep learning-based methods. However, this method has a drawback: it does not consider the influence of deeper information on shallower information during the fusion process, and it does not adequately conform to the characteristic of the human visual perception system that it perceives content before quality, leading to inaccurate evaluation results.

[0006] Existing image quality assessment methods use only a single network model and obtain quality scores using features at a single level. When faced with complex distorted images, they cannot simultaneously take into account both global and local distortion information, and therefore cannot accurately evaluate the quality of the images. Summary of the Invention

[0007] To overcome the shortcomings of the existing technologies, the present invention aims to provide a cross-multi-scale no-reference image quality assessment method. This method utilizes the local and non-local semantic information of the image and the ability of the image pyramid to extract information at different levels, thereby solving the problem that existing methods cannot simultaneously address global and local distortion when faced with complex distorted images using a single network model.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A cross-scale, no-reference image quality assessment method includes the following steps;

[0010] Step 1: Obtain the training sample set B and the test sample set C;

[0011] Step 2: Construct a network model S based on cross-scale no-reference image quality assessment;

[0012] Step 3: Iteratively train the network model S based on cross-scale no-reference image quality assessment;

[0013] Step 4, obtain the no-reference quality assessment result of the image Q:

[0014] The test sample set C is used as input to the trained no-reference image quality assessment network model S based on a cross-multi-scale network for forward inference to obtain the quality prediction score for each test sample, thereby verifying the model's effectiveness.

[0015] Step 1 specifically involves:

[0016] Step 1.1: In the three datasets LIVE, TID2013, and CSIQ, the reference image is randomly divided into two parts, one part is used for training, and the remaining part is used for testing; the distorted image corresponding to the reference image is assigned to the training sample set B1 and the test sample set C1, and all data is randomly divided into two subsets, one part is assigned to the training sample set B2, and the remaining part is assigned to the test sample set C2.

[0017] Step 1.2: Combine the images in set B1 and their corresponding quality score labels with the images in set B2 and their corresponding quality score labels to form training sample set B, and combine the images in set C1 and their corresponding quality score labels with the images in set C2 and their corresponding quality score labels to form training sample set C.

[0018] Step 2 specifically involves:

[0019] Step 2.1: Construct a vertical multi-scale network V based on cross-scale no-reference image quality assessment. The vertical multi-scale network V is used to extract features from the same image at different scales, ensuring that the obtained features simultaneously contain complex semantics and high resolution. It can express both global spatial information and local features. V includes a ResNet-50 backbone network R, a top-down upsampling fusion network TD, a bottom-up downsampling fusion network BU, and a vertical multi-scale feature output module F. The ResNet-50 backbone network R outputs feature sequences f from four different layers. i (i = 1, 2, 3, 4), the top-down upsampling fusion network TD includes four fused feature sequences f. i,a (i = 1, 2, 3, 4), the bottom-up downsampling fusion network BU includes four fused feature sequences f. i,b (i = 1, 2, 3, 4), the vertical multi-scale feature output module F includes four processed feature sequences f. i (i = 1, 2, 3, 4);

[0020] Step 2.2: Construct a horizontal multi-scale network H based on cross-scale no-reference image quality evaluation to obtain the features with optimal local representation. From the perspective of feature local information processing and statistics, pooling is performed on the sequence features output by the vertical multi-scale network using pooling groups of different sizes to achieve optimal representation of local image features.

[0021] Step 2.3: Construct a quality regression network R based on cross-scale no-reference image quality assessment;

[0022] In step 2, the input image passes through the vertical multi-scale network V and the horizontal multi-scale network H to obtain the required features. These features are then input into the quality regression network R to obtain the final quality regression score.

[0023] The network model S based on cross-scale no-reference image quality assessment consists of a vertical multi-scale network V, a horizontal multi-scale network H, and a quality regression network R.

[0024] Step 2.1 specifically involves:

[0025] Step 2.1.1: The ResNet-50 backbone network R extracts semantic features at four levels. The feature resolution decreases progressively from bottom to top. In the ResNet-50 backbone network R, the features extracted at four scales from bottom to top are f1, f2, f3, and f4.

[0026] Step 2.1.2, the fusion process of the top-down upsampling fusion network TD is as follows: perform a 1×1 convolution on the top-level feature f4 to obtain feature f 4,a f 4,a Perform a 2x upsampling and fuse it with the second-highest level feature f3 to obtain the fused feature f. 3,a ; Fusion feature f 3,a Perform a 2x upsampling and fuse it with the next lower-level feature f2 to obtain the fused feature f. 2,a ; Fusion feature f 2,a Perform a 2x upsampling and fuse it with the underlying feature f1 to obtain the fused feature f. 1,a All feature fusion operations are performed element-wise additions. At this point, the top-down pre-fusion is complete, yielding the feature sequence f. i,a (i = 1, 2, 3, 4); the above process is represented by the following formula:

[0027] f 4,a =Conv1(f4)

[0028]

[0029] Conv1 represents a 1×1 convolution operation, and 2×Up represents a 2x upsampling. This indicates feature fusion based on element-wise addition;

[0030] Step 2.1.3, the fusion process of the bottom-up downsampling fusion network BU is as follows: for feature f 1,a Convolution yields f 1,b ; Feature f 2,a After doubling upsampling, and with feature f 1,b By adding and fusing, we obtain the feature f. 2,b ; Feature f 3,a After doubling upsampling, and with feature f 2,b By adding and fusing, we obtain the feature f. 3,b ; Feature f 4,a After doubling upsampling, and with feature f 3,b By adding and fusing, we obtain the feature f. 4,b At this point, the bottom-up feature fusion and enhancement is complete, and the resulting sequence feature f i,b (i = 1, 2, 3, 4), this feature combines the advantages of high resolution of shallow features and rich semantics of deep features. The above feature fusion enhancement process is represented by the following formula:

[0031] f 1,b =Conv1(f 1,a )

[0032]

[0033] Conv1 represents a 1×1 convolution operation, and 2×Up represents a 2x upsampling. This indicates feature fusion based on element-wise addition;

[0034] Step 2.1.4: The vertical multi-scale feature output module F outputs the processed features respectively, and the features from bottom to top are f1', f2', f3', and f4'.

[0035] Step 2.2 specifically involves:

[0036] Step 2.2.1, predefine the pooling kernel sequence: pool x (x = 1, 2, 4, 6), layer features f b,i via pool x Feature sequences of different local sizes were statistically analyzed:

[0037] f x,i =pool x (f i ')

[0038] This operation follows the vertical network V and is the first step in the horizontal network H. It is the output f of the vertical network V. i ', to perform pooling operations.

[0039] Step 2.2.2, feature f x,i After 1×1 convolution, redundancy is removed through rectification, and the channel dimension is compressed:

[0040] f c x,i =Conv1(f x,i )

[0041] Step 2.2.3, for the feature sequence f c xi Upsampling is performed to align the features, and they are then concatenated according to the channel dimension to form a multi-scale fused feature F. i :

[0042] F i =Comcat(f c x,1 ,f c x,2 ,f c x,3 ,f c x,4 )

[0043] Among them, pool x The first indicates a pooling operation, the second indicates a convolution operation, and the third indicates a concatenation operation.

[0044] The horizontal multi-scale network H takes four feature vectors as input and outputs four feature vectors after passing through it. F i This is the output result.

[0045] Step 2.3 specifically involves:

[0046] Step 2.3.1, fuse the multi-scale features F i (i = 1, 2, 3, 4) are summed pixel by pixel to obtain the final distortion feature.

[0047] Step 2.3.2: Flatten the distorted feature F to obtain the feature in vector form. Where m is the feature dimension, the input is a regression module consisting of three fully connected layers, and the weight matrix of the fully connected layer FC1 is W1∈R. 628×m The bias is b1∈R 628 The weight matrix of the fully connected layer FC2 is W2∈R 128×628 The bias is b2∈R 128 The weight matrix of the fully connected layer Fc3 is W3∈R 1×128 The bias is b3∈R 1 The output of the fully connected layer is:

[0048] y i =ReLU(W i y i-1 +b i )

[0049] ReLU is the activation function. The output y3 of the fully connected layer FC2 is the final quality score Q of the distorted image, which is the final output of the quality regression network R. For the input features F, after passing through three fully connected layers, the output is the final quality score.

[0050] Step 3 specifically involves:

[0051] Step 3.1: Initialize the number of iterations to t, the maximum number of iterations to T, and the current no-reference image quality assessment network model based on cross-multiscale networks to S. t And let t = 1, S t =S;

[0052] Step 3.2: Randomly select b training samples without replacement from the training sample set B obtained in Step 1 as the no-reference image quality assessment network model S. t The input is: the vertical multi-scale network V extracts features at different levels for each training sample, the horizontal multi-scale network H fuses features at different scales at the same level, the quality regression network R concatenates the output multi-level features and features at different scales at the same level to obtain cross-multi-scale features, and then uses three fully connected layers to regress the cross-multi-scale features to obtain the quality prediction score corresponding to each training sample.

[0053] Step 3.3: First, calculate the mean squared error between the predicted quality score and the corresponding quality score label for each training sample as the model S. t The loss value is then used to optimize the network model S using the Adam optimization algorithm. t The weight parameters of each layer are updated;

[0054] The formula for calculating the mean squared error between the predicted quality score and the quality score label for each training sample is as follows:

[0055]

[0056] The Adam optimization algorithm is used, with all parameters except the learning rate set to default values. To ensure the model's convergence speed and learning accuracy, the learning rate is updated according to the following formula.

[0057]

[0058]

[0059] b represents the number of training samples randomly selected without replacement from the training sample set B during iterative training of the no-reference image quality assessment network model S based on a cross-multi-scale network, and q represents the number of training samples selected without replacement from the training sample set B. g This represents the quality score label corresponding to the g-th training sample out of the b training samples. This represents the quality prediction score corresponding to the g-th training sample among the b training samples;

[0060] T represents the current iteration number, L represents the step size for updating the learning rate, that is, the learning rate will be updated once every L iterations, d represents the decay coefficient of the learning rate, a regularization term is added to the loss function, and weight decay is added during the training process.

[0061] b=16, L=1, d=0.5, weight_decay=5×10 -4 ;

[0062] Step 3.4: Determine whether the training sample set B has been traversed. If yes, proceed to step 3.5; otherwise, proceed to steps 3.2 and 3.3.

[0063] Step 3.5: Determine if t = T holds true. If so, obtain the trained no-reference image quality assessment network model S based on a cross-multi-scale network. * =S t Otherwise, let t = t + 1, and execute steps 3.3 and 3.4.

[0064] The beneficial effects of this invention are:

[0065] This invention utilizes a cross-scale network to extract features from semantically rich image patches. By analyzing the different information contained in image patches at different levels, it achieves referenceless quality evaluation and analysis of images. By fully utilizing the coarse-grained features under a large receptive field and the fine-grained features under a small receptive field in the image's feature map, this invention can effectively evaluate the quality of an image that has both global and local distortions. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the process of the present invention.

[0067] Figure 2 This is a diagram of the cross-scale network model of the present invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings.

[0069] like Figure 1 , Figure 2 As shown:

[0070] Step 1: Obtain the training sample set B and the test sample set C.

[0071] Step 1.1: In the LIVE, TID2013, and CSIQ datasets, the reference images are randomly divided into two parts, with 70% used for training and the remaining 30% for testing. The distorted images corresponding to the reference images are assigned to the training sample set B1 and the test sample set C1, ensuring that there are no identical images between the training and test sets. For the LIVEC dataset, since the image data has no reference images and all are truly distorted, all data are randomly divided into two subsets, with 70% assigned to the training sample set B2 and the remaining 30% assigned to the test sample set C2.

[0072] Step 1.2: Combine the images in set B1 and their corresponding quality score labels with the images in set B2 and their corresponding quality score labels to form training sample set B, and combine the images in set C1 and their corresponding quality score labels with the images in set C2 and their corresponding quality score labels to form training sample set C.

[0073] Step 2: Construct a network model S based on cross-scale no-reference image quality evaluation.

[0074] Step 2.1: Construct a vertical multi-scale network V based on cross-scale no-reference image quality assessment. This network includes a ResNet-50 backbone network R, a top-down upsampling fusion network TD, a bottom-up downsampling fusion network BU, and a vertical multi-scale feature output module F. The ResNet-50 backbone network R outputs feature sequences f from four different layers. i (i = 1, 2, 3, 4), the top-down upsampling fusion network TD includes four fused feature sequences f. i,a (i = 1, 2, 3, 4), the bottom-up downsampling fusion network BU includes four fused feature sequences f. i,b (i = 1, 2, 3, 4), the vertical multi-scale feature output module F includes four processed feature sequences f. i (i = 1, 2, 3, 4).

[0075] Step 2.1.1: The ResNet-50 backbone network R extracts semantic features at four levels, with the feature resolution decreasing progressively from bottom to top. In the ResNet-50 backbone network R, the features extracted at four scales from bottom to top are f1, f2, f3, and f4.

[0076] Step 2.1.2, the fusion process of the top-down upsampling fusion network TD is as follows: perform a 1×1 convolution on the top-level feature f4 to obtain feature f 4,a f 4,aPerform a 2x upsampling and fuse it with the second-highest level feature f3 to obtain the fused feature f. 3,a ; Fusion feature f 3,a Perform a 2x upsampling and fuse it with the next lower-level feature f2 to obtain the fused feature f. 2,a ; Fusion feature f 2,a Perform a 2x upsampling and fuse it with the underlying feature f1 to obtain the fused feature f. 1,a All feature fusion operations are performed element-wise additions. At this point, the top-down pre-fusion is complete, yielding the feature sequence f. i,a (i = 1, 2, 3, 4). The above process can be expressed as follows:

[0077] f 4,a =Conv1(f4)

[0078]

[0079] In this embodiment, Conv1 represents a 1×1 convolution operation, and 2×Up represents a 2x upsampling. This indicates feature fusion based on element-wise addition.

[0080] Step 2.1.3, the fusion process of the bottom-up downsampling fusion network BU is as follows: for feature f 1,a Convolution yields f 1,b ; Feature f 2,a After doubling upsampling, and with feature f 1,b By adding and fusing, we obtain the feature f. 2,b ; Feature f 3,a After doubling upsampling, and with feature f 2,b By adding and fusing, we obtain the feature f. 3,b ; Feature f 4,a After doubling upsampling, and with feature f 3,b By adding and fusing, we obtain the feature f. 4,b At this point, the bottom-up feature fusion and enhancement is complete, and the resulting sequence feature f is... i,b (i = 1, 2, 3, 4), this feature combines the advantages of high resolution of shallow features and rich semantics of deep features. The above feature fusion enhancement process can be expressed by the following formula:

[0081] f 1,b =Conv1(f 1,a )

[0082]

[0083] In this embodiment, Conv1 represents a 1×1 convolution operation, and 2×Up represents a 2x upsampling. This indicates feature fusion based on element-wise addition.

[0084] Step 2.1.4: The vertical multi-scale feature output module F outputs the processed features respectively, and the features from bottom to top are f1', f2', f3', and f4'.

[0085] Step 2.2: Construct a horizontal multi-scale network H based on cross-multi-scale no-reference image quality evaluation.

[0086] Step 2.2.1, predefine the pooling kernel sequence: pool x (x = 1, 2, 4, 6), layer features f b,i via pool x Feature sequences of different local sizes were statistically analyzed:

[0087] f x,i =pool x (f i ')

[0088] Step 2.2.2, feature f x,i After 1×1 convolution, redundancy is removed through rectification, and the channel dimension is compressed:

[0089] f c x,i =Conv1(f x,i )

[0090] Step 2.2.3, for the feature sequence f c xi Upsampling is performed to align the features, and they are then concatenated according to the channel dimension to form a multi-scale fused feature F. i :

[0091] F i =Comcat(f c x,1 ,f c x,2 ,f c x,3 ,f c x,4 )

[0092] In this embodiment, pool x The first character represents pooling, the second represents convolution, and the third represents concatenation.

[0093] Step 2.3: Construct a quality regression network R based on cross-scale no-reference image quality assessment.

[0094] Step 2.3.1, fuse the multi-scale features F i (i = 1, 2, 3, 4) are summed pixel by pixel to obtain the final distortion feature.

[0095] Step 2.3.2: Flatten the distorted feature F to obtain the feature in vector form. Where m is the feature dimension. The input is a regression module consisting of three fully connected layers, and the weight matrix of the fully connected layer FC1 is W1∈R. 628×m The bias is b1∈R 628 The weight matrix of the fully connected layer FC2 is W2∈R 128×628 The bias is b2∈R 128 The weight matrix of the fully connected layer FC3 is W3∈R 1×128 The bias is b3∈R 1 The output of the fully connected layer is:

[0096] y i =ReLU(W i y i-1 +b i )

[0097] In this embodiment, ReLU is the activation function. The output y3 of the fully connected layer FC2 is the final quality score Q of the distorted image.

[0098] Step 3: Iteratively train the network model S based on cross-scale no-reference image quality assessment.

[0099] Step 3.1: Initialize the number of iterations to t, the maximum number of iterations to T, and the current no-reference image quality assessment network model based on cross-multiscale networks to S. t And let t = 1, S t =S;

[0100] In this embodiment, Y = 60;

[0101] Step 3.2: Randomly select b training samples without replacement from the training sample set B as the no-reference image quality assessment network model S. t The input is: the vertical multi-scale network V extracts features at different levels for each training sample, the horizontal multi-scale network H fuses features at different scales at the same level, the quality regression network R concatenates the output multi-level features and features at different scales at the same level to obtain cross-multi-scale features, and then uses three fully connected layers to regress the cross-multi-scale features to obtain the quality prediction score corresponding to each training sample.

[0102] In this embodiment, b = 16;

[0103] Step 3.3: First, calculate the mean squared error between the predicted quality score and the corresponding quality score label for each training sample as the model S. t The loss value is then used to optimize the network model S using the Adam optimization algorithm. tThe weight parameters of each layer are updated;

[0104] In this embodiment, the formula for calculating the mean squared error between the quality prediction score and the quality score label corresponding to each training sample is as follows:

[0105]

[0106] In this embodiment, the Adam optimization algorithm is used, and all parameters except the learning rate use default values. To ensure the model's convergence speed and learning accuracy, the learning rate is updated according to the following formula.

[0107]

[0108]

[0109] b represents the number of training samples randomly selected without replacement from the training sample set B during iterative training of the no-reference image quality assessment network model S based on a cross-multi-scale network, and q represents the number of training samples selected without replacement from the training sample set B. g This represents the quality score label corresponding to the g-th training sample out of the b training samples. This represents the quality prediction score corresponding to the g-th training sample among the b training samples;

[0110] T represents the current iteration number, L represents the step size for updating the learning rate, that is, the learning rate will be updated once every L iterations, and d represents the decay coefficient of the learning rate. At the same time, in order to prevent the model from overfitting during training, a regularization term is added to the loss function, and weight decay is added during training.

[0111] In this embodiment, b = 16, L = 1, d = 0.5, and weight_decay = 5 × 10⁻⁶. -4 ;

[0112] Step 3.4: Determine whether the training sample set B has been traversed. If yes, proceed to step 3.5; otherwise, proceed to steps 3.2 and 3.3.

[0113] Step 3.5: Determine if t = T holds true. If so, obtain the trained no-reference image quality assessment network model S based on a cross-multi-scale network. * =S t Otherwise, let t = t + 1, and execute steps 3.3 and 3.4.

[0114] Step 4, obtain the no-reference quality assessment result of the image Q:

[0115] The test sample set C is used as input to the trained no-reference image quality assessment network model S based on a cross-multi-scale network for forward inference to obtain the quality prediction score for each test sample, thereby verifying the model's effectiveness.

Claims

1. A no-reference image quality assessment method based on cross-scale multi-scale methods, characterized in that, Includes the following steps; Step 1: Obtain the training sample set B and the test sample set C; Step 2: Construct a network model S based on cross-scale no-reference image quality assessment; Step 3: Iteratively train the network model S based on cross-scale no-reference image quality assessment; Step 4, obtain the no-reference quality assessment result of the image Q: The test sample set C is used as input to the trained no-reference image quality assessment network model S based on a cross-multi-scale network for forward inference to obtain the quality prediction score of each test sample, so as to verify the model effect. Step 2 specifically involves: Step 2.1: Construct a vertical multi-scale network V based on cross-scale no-reference image quality assessment to extract features from the same image at different scales. This network includes a ResNet-50 backbone network R, a top-down upsampling fusion network TD, a bottom-up downsampling fusion network BU, and a vertical multi-scale feature output module F. The ResNet-50 backbone network R outputs feature sequences f from four different layers. i (i = 1, 2, 3, 4), the top-down upsampling fusion network TD includes four fused feature sequences f. i,a (i = 1, 2, 3, 4), the bottom-up downsampling fusion network BU includes four fused feature sequences f. i,b (i = 1, 2, 3, 4), the vertical multi-scale feature output module F includes four processed feature sequences f. i (i = 1, 2, 3, 4); Step 2.2: Construct a horizontal multi-scale network H based on cross-multi-scale no-reference image quality evaluation to obtain features with optimal local representation; Step 2.3: Construct a quality regression network R based on cross-scale no-reference image quality assessment.

2. The method for evaluating the quality of a no-reference image based on cross-scale multi-scale as described in claim 1, characterized in that, Step 1 specifically involves: Step 1.1: In the three datasets LIVE, TID2013, and CSIQ, the reference image is randomly divided into two parts, one part is used for training, and the remaining part is used for testing; the distorted image corresponding to the reference image is assigned to the training sample set B1 and the test sample set C1, and all data is randomly divided into two subsets, one part is assigned to the training sample set B2, and the remaining part is assigned to the test sample set C2. Step 1.2: Combine the images in set B1 and their corresponding quality score labels with the images in set B2 and their corresponding quality score labels to form training sample set B, and combine the images in set C1 and their corresponding quality score labels with the images in set C2 and their corresponding quality score labels to form training sample set C.

3. The no-reference image quality assessment method based on cross-multi-scale as described in claim 1, characterized in that, Step 2.1 specifically involves: Step 2.1.1: The ResNet-50 backbone network R extracts semantic features at four levels, with the feature resolution decreasing progressively from bottom to top. In the ResNet-50 backbone network R, the features extracted at four scales from bottom to top are f1, f2, f3, and f4. Step 2.1.2, the fusion process of the top-down upsampling fusion network TD is as follows: perform a 1×1 convolution on the top-level feature f4 to obtain feature f 4,a f 4,a Perform a 2x upsampling and fuse it with the second-highest level feature f3 to obtain the fused feature f. 3,a ; Fusion feature f 3,a Perform a 2x upsampling and fuse it with the next lower-level feature f2 to obtain the fused feature f. 2,a ; Fusion feature f 2,a Perform a 2x upsampling and fuse it with the underlying feature f1 to obtain the fused feature f. 1,a All feature fusion operations are performed element-wise additions. At this point, the top-down pre-fusion is complete, yielding the feature sequence f. i,a (i = 1, 2, 3, 4); the above process is represented by the following formula: f 4,a =Conv1(f4) Conv1 represents a 1×1 convolution operation, and 2×Up represents a 2x upsampling. This indicates feature fusion based on element-wise addition; Step 2.1.3, the fusion process of the bottom-up downsampling fusion network BU is as follows: for feature f 1,a Convolution yields f 1,b ; Feature f 2,2 After doubling upsampling, and with feature f 1,b By adding and fusing, we obtain the feature f. 2,b ; Feature f 3,a After doubling upsampling, and with feature f 2,b By adding and fusing, we obtain the feature f. 3,b ; Feature f 4,a After doubling upsampling, and with feature f 3,b By adding and fusing, we obtain the feature f. 4,b At this point, the bottom-up feature fusion and enhancement is complete, and the resulting sequence feature f i,b (i = 1, 2, 3, 4), the above feature fusion enhancement process is represented by the following formula: f 1,b =Conv1(f 1,a ) Conv1 represents a 1×1 convolution operation, and 2×Up represents a 2x upsampling. This indicates feature fusion based on element-wise addition; Step 2.1.4: The vertical multi-scale feature output module F outputs the processed features respectively, and the features from bottom to top are f1', f2', f3', and f4'.

4. The method for evaluating the quality of a no-reference image based on cross-scale multi-scale as described in claim 1, characterized in that, Step 2.2 specifically involves: Step 2.2.1, predefine the pooling kernel sequence: pool x (x = 1, 2, 4, 6), layer features f b,i via pool x Feature sequences of different local sizes were statistically analyzed: f x,i =pool x (f i ’) Step 2.2.2, feature f x,i After 1×1 convolution, redundancy is removed through rectification, and the channel dimension is compressed: f c x,i =Conv1(f x,i ) Step 2.2.3, for the feature sequence f c xi Upsampling is performed to align the features, and they are then concatenated according to the channel dimension to form a multi-scale fused feature F. i : F i =Concat(f c x,1 ,f c x,2 ,f c x,3 ,f c x,4 ) Among them, pool x The first character represents pooling, the second represents convolution, and the third represents concatenation.

5. The method for evaluating the quality of a no-reference image based on cross-scale multi-scale as described in claim 1, characterized in that, Step 2.3 specifically involves: Step 2.3.1, fuse the multi-scale features F i (i = 1, 2, 3, 4) are summed pixel by pixel to obtain the final distortion feature. Step 2.3.2: Flatten the distorted feature F to obtain the feature in vector form. Where m is the feature dimension, the input is a regression module consisting of three fully connected layers, and the weight matrix of the fully connected layer FC1 is W1∈R. 628×m The bias is b1∈R 628 The weight matrix of the fully connected layer FC2 is W2∈R 128×628 The bias is b2∈R 128 The weight matrix of the fully connected layer FC3 is W3∈R 1×128 The bias is b3∈R 1 The output of the fully connected layer is: y i =ReLU(W i y i-1 +b i ) ReLU is the activation function, and the output y3 of the fully connected layer FC2 is the final quality score Q of the distorted image.

6. The method for evaluating the quality of a no-reference image based on cross-scale multi-scale as described in claim 1, characterized in that, Step 3 specifically involves: Step 3.1: Initialize the number of iterations to t, the maximum number of iterations to T, and the current no-reference image quality assessment network model based on cross-multiscale networks to S. t And let t = 1, S t =S; Step 3.2: Randomly select b training samples without replacement from the training sample set B obtained in Step 1 as the no-reference image quality assessment network model S. t The input is: the vertical multi-scale network V extracts features at different levels for each training sample, the horizontal multi-scale network H fuses features at different scales at the same level, the quality regression network R concatenates the output multi-level features and features at different scales at the same level to obtain cross-multi-scale features, and then uses three fully connected layers to regress the cross-multi-scale features to obtain the quality prediction score corresponding to each training sample. Step 3.3: First, calculate the mean squared error between the predicted quality score and the corresponding quality score label for each training sample as the model S. t The loss value is then used to optimize the network model S using the Adam optimization algorithm. t The weight parameters of each layer are updated; The formula for calculating the mean squared error between the predicted quality score and the corresponding quality score label for each training sample is as follows: The Adam optimization algorithm is used, with all parameters except the learning rate set to default values. To ensure the model's convergence speed and learning accuracy, the learning rate is updated according to the following formula. b represents the number of training samples randomly selected without replacement from the training sample set B during iterative training of the no-reference image quality assessment network model S based on a cross-multi-scale network, and q represents the number of training samples selected without replacement from the training sample set B. g This represents the quality score label corresponding to the g-th training sample out of the b training samples. This represents the quality prediction score corresponding to the g-th training sample among the b training samples; T represents the current iteration number, L represents the step size for updating the learning rate, that is, the learning rate will be updated once every L iterations, d represents the decay coefficient of the learning rate, a regularization term is added to the loss function, and weight decay is added during the training process. Step 3.4: Determine whether the training sample set B has been traversed. If yes, proceed to step 3.5; otherwise, proceed to steps 3.2 and 3.

3. Step 3.5: Determine if t = T holds true. If so, obtain the trained no-reference image quality assessment network model S based on a cross-multi-scale network. * =S t Otherwise, let t = t + 1, and execute steps 3.3 and 3.4.

Images