Image quality evaluation model based on multidirectional perception integration and difference capture mechanism

Abstract

The invention provides an image quality evaluation model based on multidirectional perception integration and a difference capture mechanism. The method is mainly used in an underwater image enhancement process, and comprises the following steps of: firstly, generating a series of progressive images between an original image and an enhanced image through a proportional image synthesis technology aiming at image deviation generated by an underwater image enhancement algorithm, especially deviation such as excessive enhancement and insufficient enhancement; the method comprises the following steps: accurately simulating possible deviation in an image enhancement process, introducing a multivariate grouping strategy, applying multidirectional perception integration to an evolution process from an original image to a reference image, and then adopting a difference capture mechanism, the method comprises the following steps of: performing interactive capture of an image deviation region on an image sequence from an original image to a reference image and then to excessive enhancement or insufficient enhancement, and finally, comprehensively analyzing score features and weight features from multiple dimensions by utilizing a parallel processing architecture to realize accurate prediction of a final quality score of the image.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image quality assessment, and in particular to an image quality assessment model based on multi-directional perception integration and difference capture mechanism. Background Art

[0002] High-quality images are essential in marine industries and maritime activities, but the underwater environment often causes image quality impairment, mainly due to factors such as light attenuation, scattering, noise caused by suspended particles, color distortion, and interference from water currents and organisms. These interferences make it difficult to extract valuable information from images.

[0003] In order to improve the quality of underwater images, researchers have developed a variety of enhancement techniques, including traditional image processing methods and deep learning methods. Although the enhancement algorithms in recent years have significantly improved the quality of underwater images, they are often prone to over-enhancement and under-enhancement. Over-enhancement can make the image appear unnatural, resulting in loss of details and color distortion, while under-enhancement cannot effectively improve the original image, resulting in problems such as blur, noise, and color distortion that cannot be solved. Therefore, evaluating the quality of images produced by these algorithms is a complex and challenging task. Although subjective evaluations are considered reliable, they are time-consuming and cumbersome and are not suitable for large-scale applications. Therefore, it is urgent to develop objective indicators that can be used to evaluate the quality of underwater images. Such indicators can both reduce human intervention and be closely consistent with human perception. They are an effective way to evaluate the performance of algorithms in complex underwater environments. Summary of the invention

[0004] According to the technical problems raised above, an image quality assessment model based on multi-directional perception integration and difference capture mechanism is provided. The present invention mainly uses the image deviation generated by the underwater image enhancement algorithm in the process of underwater image enhancement, especially the deviations such as over-enhancement and under-enhancement. First, a series of progressive images are generated between the original image and the enhanced image through proportional image synthesis technology to accurately simulate the deviations that may occur in the image enhancement process. Then, a multivariate grouping strategy is introduced to apply multi-directional perception integration to the evolution process from the original image to the reference image. Subsequently, a difference capture mechanism is adopted to interactively capture the image deviation area of ​​the image sequence from the original image to the reference image and then to the over-enhancement or under-enhancement. Finally, a parallel processing architecture is used to comprehensively analyze the score features and weight features from multiple dimensions to achieve an accurate prediction of the final image quality score.

[0005] The technical means adopted by the present invention are as follows:

[0006] The image quality assessment method based on multi-directional perception integration and difference capture mechanism includes the following steps:

[0007] Step 1: Generate a series of images between the original image and the enhanced image through proportional image synthesis technology to simulate the deviation in the image enhancement process;

[0008] Step 2: Introduce a multi-level grouping strategy to perform multi-level grouping processing on the dataset images obtained after step 1, that is, divide the normally enhanced image dataset into group G1, and at the same time, name the dataset obtained after step 1 including the normally enhanced, over-enhanced, and under-enhanced images into group G2;

[0009] Step 3: Apply multi-directional perception integration technology to the evolution process from the original image to the reference image, that is, construct a full range of image feature information through layer-by-layer feature extraction and fusion;

[0010] Step 4: Use a difference capture mechanism to capture the deviation characteristics of the image sequence from the original image to the reference image, and then to the over-enhanced or under-enhanced image sequence;

[0011] Step 5: Through the symmetric transformation of the parallel processing structure, the score features and weight features obtained in steps 3 and 4 are comprehensively processed from different angles to finally obtain the predicted score of image quality.

[0012] Furthermore, it is characterized in that the step 1 comprises the following steps:

[0013] Step 11: Process the original input image through image enhancement technology to generate a sequence from the original image to the enhanced image, and then to the over-enhanced image and the under-enhanced image;

[0014] Step 12: Mix the original input image with the enhanced image to generate a moderately enhanced data set, and mix the enhanced image with the over-enhanced image and the under-enhanced image to generate an over-enhanced data set and an under-enhanced data set, respectively. The mixing formula is:

[0015] M1=K·Orig+(1-K)·GT;

[0016] M2=K·GT+(1-K)·Over / Under Enh;

[0017] Among them, M1 represents the normal enhancement process, M2 represents the over-enhancement or under-enhancement process, K represents the mixing ratio, Orig represents the original input image, GT represents the enhanced image, and Over / Under Enh represents the over-enhancement or under-enhancement image.

[0018] Furthermore, the step 3 comprises the following steps:

[0019] Step 31: Use the visual transformer ViT to extract features and obtain feature representation;

[0020] Step 32: Extract edge features from the feature representation using an edge detection method; first, calculate the gradient features of the G1 group image in the horizontal and vertical directions, and then combine the gradient features from the two directions to generate edge features:

[0021] E x =conv_x(F i );

[0022] E y =conv_y(F i );

[0023] F E =σ(E)·F i ;

[0024] Among them, E x and E y Represent the gradient features in the horizontal and vertical directions respectively, σ represents the Sigmoid function, F i represents the feature map extracted by the visual transformer ViT, F E Represents edge features;

[0025] Step 33: Introduce a multidimensional feature extraction module; for a given feature map F i , that is, the feature map extracted by ViT is transformed through the convolution kernel to generate features of multiple branches, and then the results of multiple branches are fused by element-level summation to obtain the fused feature map F f , and then use global average pooling to calculate the statistics of each channel:

[0026] F1=f 3×3 (F i );

[0027] F2=f 5×5 (F i );

[0028] F f =F1+F2;

[0029]

[0030] Among them, F1 and F2 represent the feature maps processed by 3×3 and 5×5 convolution kernels respectively, and s i represents the statistics of channel i, H and W represent the feature map F f The height and width of

[0031] Step 34: The model uses a fully connected layer and activation function to generate selection weights, uses a softmax operation to adaptively select between features of different convolution kernel sizes, and adjusts the feature map through the attention mechanism;

[0032] First, the global statistics s of the feature are calculated, and then transformed through the fully connected layer and batch normalization layer to obtain the compact feature z. Then, the selection weights α and β are calculated. Through the calculated selection weights, it is possible to adaptively select between features of different convolution kernel sizes, thereby forming the final multi-dimensional feature representation F H :

[0033] z = δ(B(Ws));

[0034] α=softmax(Az);

[0035] β=softmax(Bz);

[0036] F H =α·F1+β·F2;

[0037] Among them, δ represents the ReLU function, B represents the batch normalization layer, W represents the weight matrix of the fully connected layer, and A and B represent the weight matrices of different channels respectively;

[0038] Step 35: Add the output features of the multi-dimensional feature extraction module and the edge detection module to achieve multi-directional perceptual integration of global and local features:

[0039] F M =F H +F E ;

[0040] Among them, F M Represents the feature representation obtained in this step.

[0041] Furthermore, step 4 includes the following steps:

[0042] Step 41: Divide the G2 group image into blocks, and input each of the divided blocks into multiple difference capture extractors; each difference capture extractor includes an adaptive average pooling layer and a 1×1 convolution layer; each difference capture extractor first reduces the input feature map to a predetermined size through adaptive average pooling, and then extracts features through a 1×1 convolution layer;

[0043] Step 42: A bottleneck layer is added after each difference capture extractor, and the bottleneck layer compresses the feature dimension, i.e., the dimension of the feature extracted in step 41, from 2C to C through a 1×1 convolutional layer. out , to reduce the number of channels of the feature map; where 2C means that each input channel corresponds to two output channels;

[0044] Step 43: Introduce a weight network consisting of a 1×1 convolutional layer into the model to calculate the weights between the feature maps of each scale, i.e., the feature maps extracted in step 41;

[0045] Step 44: extract multi-dimensional features from the input feature map X, i.e., the one obtained after steps 41 and 42;

[0046] Step 45: Calculate each scale feature map, that is, the difference between the feature map extracted in step 41 and the original feature map:

[0047] D i =S i -X;

[0048] Among them, S i represents the feature map of the i-th scale, D i Represents the difference feature map of the i-th scale;

[0049] Step 46: Input each difference feature map into the weight network to generate a weight feature map:

[0050] W i =σ(WeightNet(D i ));

[0051] Among them, W i represents the weight feature map of the i-th scale, σ represents the Sigmoid activation function;

[0052] Step 47: Based on the weight feature map generated in step 46, weighted sum is performed on each scale feature map to obtain a difference feature map:

[0053]

[0054] Among them, X context represents the weighted difference feature map, ⊙ represents element-by-element multiplication;

[0055] Step 48: Concatenate the difference captured feature map with the original feature map in the channel dimension, perform feature compression through the bottleneck layer, and finally output the feature value through the ReLU activation function.

[0056] Furthermore, the step 5 comprises the following steps:

[0057] Step 51: F M and F D All layers are normalized to obtain normalized features, and then the normalized features are added together, and the addition results are processed by MLP to obtain the fused feature representation F:

[0058] F = MLP (LayerNorm (F M +F D )); where F Mrepresents the features obtained through multi-directional perception integration, F D It represents the features obtained by the difference capture mechanism, LayerNorm is used for layer normalization, MLP is used for fusion features, and the final F is the final representation after integrating different features;

[0059] Among them, F M is the feature representation obtained in step 3, F D is the feature representation obtained in step 4;

[0060] Step 52: The score features and weight features are processed comprehensively from different angles through the symmetric transformation of the parallel processing structure, and finally the predicted score of the image is obtained:

[0061]

[0062] Among them, α i Represents ω i The normalized weight of .

[0063] Compared with the prior art, the present invention has the following advantages:

[0064] 1. In terms of data set processing, the present invention uses proportional image synthesis to process images generated by multiple enhancement algorithms, and distinguishes original images, real reference images, and over-enhanced images by layering to simulate the dynamic changes of scores during image enhancement. This method can effectively reduce the interference of subjective factors and provide more objective and accurate scoring results.

[0065] 2. The present invention proposes a differentiated processing strategy based on multi-element grouping. Through multi-directional perception integration, the image detail perception capability is enhanced. In order to deal with over-enhancement or enhancement deviation areas, a difference capture module is introduced, which effectively solves the problem of insufficient accuracy of image deviation features.

[0066] 3. The present invention adopts a parallel processing structure, and comprehensively processes score features and weight features from different angles through symmetric transformation, balances the difference between the over-enhanced or under-enhanced areas in the image blocks and the true reference areas, and finally weightedly calculates the activation score and weight of each image block to obtain a comprehensive underwater image quality assessment score.

[0067] 4. The present invention was trained and tested on multiple underwater image datasets, and cross-dataset testing was performed to verify the effectiveness and reliability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0069] Figure 1 It is a model flow chart of the present invention. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0071] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0072] like Figure 1 As shown, the image quality assessment method based on multi-directional perception integration and difference capture mechanism includes the following steps:

[0073] Step 1: Generate a series of images between the original image and the enhanced image by using a proportional image synthesis technique to simulate the deviation in the image enhancement process; Step 1 includes the following steps:

[0074] Step 11: Process the original input image through image enhancement technology to generate a sequence from the original image to the enhanced image, and then to the over-enhanced image and the under-enhanced image;

[0075] Step 12: Mix the original input image with the enhanced image to generate a moderately enhanced data set, and mix the enhanced image with the over-enhanced image and the under-enhanced image to generate an over-enhanced data set and an under-enhanced data set, respectively. The mixing formula is:

[0076] M1=K·Orig+(1-K)·GT;

[0077] M2=K·GT+(1-K)·Over / Under Enh;

[0078] Among them, M1 represents the normal enhancement process, M2 represents the over-enhancement or under-enhancement process, K represents the mixing ratio, Orig represents the original input image, GT represents the enhanced image, and Over / Under Enh represents the over-enhancement or under-enhancement image.

[0079] Furthermore, step 2: introduce a multi-grouping strategy to perform multi-level grouping processing on the data set images obtained after step 1, that is, the normally enhanced image data set is divided into group G1, and the data set obtained after step 1 including the normally enhanced, over-enhanced, and under-enhanced images is named group G2; (group G1 contains the normally enhanced image data set, group G2 contains the entire data set including the normally enhanced, over-enhanced, and under-enhanced images, and the contents of groups G1 and G2 have overlapping parts).

[0080] As a preferred implementation, step 3: applying multi-directional perception integration technology to the evolution process from the original image to the reference image, that is, constructing all-round image feature information through layer-by-layer feature extraction and fusion; step 3 includes the following steps:

[0081] Step 31: Use the visual transformer ViT to extract features and obtain feature representation;

[0082] Step 32: Extract edge features from the feature representation using an edge detection method; first, calculate the gradient features of the G1 group image in the horizontal and vertical directions, and then combine the gradient features from the two directions to generate edge features:

[0083] E x =conv_x(F i );

[0084] E y =conv_y(F i );

[0085] F E =σ(E)·F i ;

[0086] Among them, E xand E y Represent the gradient features in the horizontal and vertical directions respectively, σ represents the Sigmoid function, F i represents the feature map extracted by the visual transformer ViT, F E Represents edge features;

[0087] Step 33: Introduce a multidimensional feature extraction module; for a given feature map F i , that is, the feature map extracted by ViT is transformed through the convolution kernel to generate features of multiple branches, and then the results of multiple branches are fused by element-level summation to obtain the fused feature map F f , and then use global average pooling to calculate the statistics of each channel:

[0088] F1=f 3×3 (F i );

[0089] F2=f 5×5 (F i );

[0090] F f =F1+F2;

[0091]

[0092] Among them, F1 and F2 represent the feature maps processed by 3×3 and 5×5 convolution kernels respectively, and s i represents the statistics of channel i, H and W represent the feature map F f The height and width of

[0093] Step 34: The model uses a fully connected layer and activation function to generate selection weights, uses a softmax operation to adaptively select between features of different convolution kernel sizes, and adjusts the feature map through the attention mechanism;

[0094] First, the global statistics s of the feature are calculated, and then transformed through the fully connected layer and batch normalization layer to obtain the compact feature z. Then, the selection weights α and β are calculated. Through the calculated selection weights, it is possible to adaptively select between features of different convolution kernel sizes, thereby forming the final multi-dimensional feature representation F H :

[0095] z = δ(B(Ws));

[0096] α=softmax(Az);

[0097] β=softmax(Bz);

[0098] F H =α·F1+β·F2;

[0099] Among them, δ represents the ReLU function, B represents the batch normalization layer, W represents the weight matrix of the fully connected layer, and A and B represent the weight matrices of different channels respectively;

[0100] Step 35: Add the output features of the multi-dimensional feature extraction module and the edge detection module to achieve multi-directional perceptual integration of global and local features:

[0101] F M =F H +F E ;

[0102] Among them, F M Represents the feature representation obtained in this step.

[0103] Step 4: adopting a difference capture mechanism to capture deviation features from the original image to the reference image and then to the over-enhanced or under-enhanced image sequence; the step 4 comprises the following steps:

[0104] Step 41: Divide the G2 group image into blocks, and input each of the divided blocks into multiple difference capture extractors; each difference capture extractor includes an adaptive average pooling layer and a 1×1 convolution layer; each difference capture extractor first reduces the input feature map to a predetermined size through adaptive average pooling, and then extracts features through a 1×1 convolution layer;

[0105] Step 42: A bottleneck layer is added after each difference capture extractor, and the bottleneck layer compresses the feature dimension, i.e., the dimension of the feature extracted in step 41, from 2C to C through a 1×1 convolutional layer. out , to reduce the number of channels of the feature map; where 2C means that each input channel corresponds to two output channels;

[0106] Step 43: Introduce a weight network consisting of a 1×1 convolutional layer into the model to calculate the weights between the feature maps of each scale, i.e., the feature maps extracted in step 41;

[0107] Step 44: extract multi-dimensional features from the input feature map X, i.e., the one obtained after steps 41 and 42;

[0108] Step 45: Calculate each scale feature map, that is, the difference between the feature map extracted in step 41 and the original feature map:

[0109] D i =S i -X;

[0110] Among them, S i represents the feature map of the i-th scale, D i Represents the difference feature map of the i-th scale;

[0111] Step 46: Input each difference feature map into the weight network to generate a weight feature map:

[0112] W i =σ(WeightNet(D i ));

[0113] Among them, W i represents the weight feature map of the i-th scale, σ represents the Sigmoid activation function;

[0114] Step 47: Based on the weight feature map generated in step 46, weighted sum is performed on each scale feature map to obtain a difference feature map:

[0115]

[0116] Among them, X context represents the weighted difference feature map, ⊙ represents element-by-element multiplication;

[0117] Step 48: Concatenate the difference captured feature map with the original feature map in the channel dimension, perform feature compression through the bottleneck layer, and finally output the feature value through the ReLU activation function.

[0118] Step 5: Through the symmetric transformation of the parallel processing structure, the score features and weight features obtained in step 3 and step 4 are comprehensively processed from different angles to finally obtain the predicted score of the image quality. Step 5 includes the following steps:

[0119] Step 51: F M and F D All layers are normalized to obtain normalized features, and then the normalized features are added together, and the addition results are processed by MLP to obtain the fused feature representation F:

[0120] F = MLP (LayerNorm (F M +F D )); where F M represents the features obtained through multi-directional perception integration, F D It represents the features obtained by the difference capture mechanism, LayerNorm is used for layer normalization, MLP is used for fusion features, and the final F is the final representation after integrating different features;

[0121] Among them, F M is the feature representation obtained in step 3, F D is the feature representation obtained in step 4;

[0122] Step 52: The score features and weight features are processed comprehensively from different angles through the symmetric transformation of the parallel processing structure, and finally the predicted score of the image is obtained:

[0123]

[0124] Among them, α i Represents ω i The normalized weight of .

[0125] Embodiment 1:

[0126] In order to verify the generalization of the present invention for the evaluation of different underwater scenes, multiple underwater image datasets were selected as test sets, and the experimental results of the proposed method were compared with those of CCF (An imaging-inspired no-reference underwater color image quality assessment-ment metric) algorithm, Twice Mixing (Twice mixing: a rank learningbased quality assessment approach for underwater image enhancement) algorithm, FDUM (A reference-free underwater image quality assessment metric in frequency domain) algorithm, ATUIQP (Underwater image quality assessment: Benchmark database and objective method) algorithm, NMFC (Underwater image quality assessment method based on color space multi-feature fusion) algorithm, and URanker (Underwater ranker: Learn which is better and how to be better) algorithm from both qualitative and quantitative aspects.

[0127] This embodiment compares the experimental results of different algorithms from four objective indicators: SROCC, KROCC, PLCC, and RMSE; when SROCC is 1, it indicates that the algorithm performance is very good, and -1 indicates that it is very poor. The closer the value is to 1, the better the performance of the IQA algorithm. The larger the value of KROCC, the better the correlation between the two data and the better the performance of the IQA algorithm; the smaller the value of KROCC, the worse the correlation. In contrast, SROCC focuses on calculating the correlation between two vectors, while KROCC tends to evaluate the dependence strength of two vectors. PLCC describes the linear correlation between two sets of data, and its value range is -1 to 1. When the value of PLCC is zero, it means that the two sets of data are independent. The closer the RMSE root mean square error is to 0, the better the performance of the IQA algorithm.

[0128] From the data in Table 1, it can be seen that the present invention has significant improvements in SROCC, KROCC, PLCC and RMSE, and is superior to other underwater image quality evaluation algorithms.

[0129] Table 1 Comparison of the processing results of the proposed model and other advanced algorithms on the composite data set generated by SAUD and proportional image synthesis technology

[0130]

[0131] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image quality assessment method based on multi-directional perception integration and difference capture mechanism, characterized in that: The following steps are involved: Step 1: Generate a series of images between the original image and the enhanced image through proportional image synthesis technology to simulate the deviation in the image enhancement process; Step 2: Introduce a multi-level grouping strategy to perform multi-level grouping processing on the dataset images obtained after step 1, that is, divide the normally enhanced image dataset into group G1, and at the same time, name the dataset obtained after step 1 including the normally enhanced, over-enhanced, and under-enhanced images into group G2; Step 3: Apply multi-directional perception integration technology to the evolution process from the original image to the reference image, that is, construct a full range of image feature information through layer-by-layer feature extraction and fusion; Step 4: Use a difference capture mechanism to capture the deviation characteristics of the image sequence from the original image to the reference image, and then to the over-enhanced or under-enhanced image sequence; Step 5: Through the symmetric transformation of the parallel processing structure, the score features and weight features obtained in steps 3 and 4 are comprehensively processed from different angles to finally obtain the predicted score of image quality.

2. The image quality assessment method based on multi-directional perception integration and difference capture mechanism according to claim 1 is characterized in that: The step 1 comprises the following steps: Step 11: Process the original input image through image enhancement technology to generate a sequence from the original image to the enhanced image, and then to the over-enhanced image and the under-enhanced image; Step 12: Mix the original input image with the enhanced image to generate a moderately enhanced data set, and mix the enhanced image with the over-enhanced image and the under-enhanced image to generate an over-enhanced data set and an under-enhanced data set, respectively. The mixing formula is: M1=K·Orig+(1-K)·GT; M2=K·GT+(1-K)·Over / Under Enh; Among them, M1 represents the normal enhancement process, M2 represents the over-enhancement or under-enhancement process, K represents the mixing ratio, Orig represents the original input image, GT represents the enhanced image, and Over / Under Enh represents the over-enhancement or under-enhancement image.

3. The image quality assessment method based on multi-directional perception integration and difference capture mechanism according to claim 1 is characterized in that: The step 3 comprises the following steps: Step 31: Use the visual transformer ViT to extract features and obtain feature representation; Step 32: Extract edge features from the feature representation using an edge detection method; first, calculate the gradient features of the G1 group image in the horizontal and vertical directions, and then combine the gradient features from the two directions to generate edge features: E x =conv_x(F i ); E y =conv_y(F i ); F E =σ(E)·F i ; Among them, E x and E y Represent the gradient features in the horizontal and vertical directions respectively, σ represents the Sigmoid function, F i represents the feature map extracted by the visual transformer ViT, F E Represents edge features; Step 33: Introduce a multidimensional feature extraction module; for a given feature map F i , that is, the feature map extracted by ViT is transformed through the convolution kernel to generate features of multiple branches, and then the results of multiple branches are fused by element-level summation to obtain the fused feature map F f , and then use global average pooling to calculate the statistics of each channel: F1=f 3×3 (F i ); F2=f 5×5 (F i ); F f =F1+F2; Among them, F1 and F2 represent the feature maps processed by 3×3 and 5×5 convolution kernels respectively, and s i represents the statistics of channel i, H and W represent the feature map F f The height and width of Step 34: The model uses a fully connected layer and activation function to generate selection weights, uses a softmax operation to adaptively select between features of different convolution kernel sizes, and adjusts the feature map through the attention mechanism; First, the global statistics s of the feature are calculated, and then transformed through the fully connected layer and batch normalization layer to obtain the compact feature z. Then, the selection weights α and β are calculated. Through the calculated selection weights, it is possible to adaptively select between features of different convolution kernel sizes, thereby forming the final multi-dimensional feature representation F H : z = δ(B(Ws)); α=softmax(Az); β=softmax(Bz); F H =α·F1+β·F2; Among them, δ represents the ReLU function, B represents the batch normalization layer, W represents the weight matrix of the fully connected layer, and A and B represent the weight matrices of different channels respectively; Step 35: Add the output features of the multi-dimensional feature extraction module and the edge detection module to achieve multi-directional perceptual integration of global and local features: F M =F H +F E ; Among them, F M Represents the feature representation obtained in this step.

4. The image quality assessment method based on multi-directional perception integration and difference capture mechanism according to claim 1, characterized in that: The step 4 comprises the following steps: Step 41: Divide the G2 group image into blocks, and input each of the divided blocks into multiple difference capture extractors; each difference capture extractor includes an adaptive average pooling layer and a 1×1 convolution layer; each difference capture extractor first reduces the input feature map to a predetermined size through adaptive average pooling, and then extracts features through a 1×1 convolution layer; Step 42: A bottleneck layer is added after each difference capture extractor, and the bottleneck layer compresses the feature dimension, i.e., the dimension of the feature extracted in step 41, from 2C to C through a 1×1 convolutional layer. out , to reduce the number of channels of the feature map; where 2C means that each input channel corresponds to two output channels; Step 43: Introduce a weight network consisting of a 1×1 convolutional layer into the model to calculate the weights between the feature maps of each scale, i.e., the feature maps extracted in step 41; Step 44: extract multi-dimensional features from the input feature map X, i.e., the one obtained after steps 41 and 42; Step 45: Calculate each scale feature map, that is, the difference between the feature map extracted in step 41 and the original feature map: D i =S i -X; Among them, S i represents the feature map of the i-th scale, D i Represents the difference feature map of the i-th scale; Step 46: Input each difference feature map into the weight network to generate a weight feature map: W i =σ(WeightNet(D i )); Among them, W i represents the weight feature map of the i-th scale, σ represents the Sigmoid activation function; Step 47: Based on the weight feature map generated in step 46, weighted sum is performed on each scale feature map to obtain a difference feature map: Among them, X context represents the weighted difference feature map, ⊙ represents element-by-element multiplication; Step 48: Concatenate the difference captured feature map with the original feature map in the channel dimension, perform feature compression through the bottleneck layer, and finally output the feature value through the ReLU activation function.

5. The image quality assessment method based on multi-directional perception integration and difference capture mechanism according to claim 1 is characterized in that: The step 5 comprises the following steps: Step 51: F M and F D All layers are normalized to obtain normalized features, and then the normalized features are added together, and the addition results are processed by MLP to obtain the fused feature representation F: F = MLP (LayerNorm (F M +F D )); where F M represents the features obtained through multi-directional perception integration, F D Represents the features obtained through the difference capture mechanism, LayerNorm is used for layer normalization, MLP is used to fuse features, and the final F is the final representation after integrating different features; Among them, F M is the feature representation obtained in step 3, F D is the feature representation obtained in step 4; Step 52: The score features and weight features are processed comprehensively from different angles through the symmetric transformation of the parallel processing structure, and finally the predicted score of the image is obtained: Among them, α i Represents ω i The normalized weight of .

Images