A quaternion convolutional neural network-based no-reference image quality assessment method and system
By using quaternion convolutional neural networks to process color images, the correlation and integrity of color channels are preserved, solving the problem of color feature loss in existing technologies and achieving higher evaluation accuracy and robustness.
Patent Information
- Application Number
- CN202410025201.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-08
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2044-01-08
AI Technical Summary
Existing convolutional neural networks lose the color feature correlation of color images in no-reference image quality assessment, resulting in insufficient assessment accuracy and robustness.
A quaternion convolutional neural network is used to process the three color channels of a color image as a whole and to extract features using quaternion algebra operations. This is combined with multi-scale feature fusion and low-dimensional feature extraction to maintain the correlation and integrity of the color channels.
It improves the accuracy and robustness of no-reference image quality assessment and enhances the ability to represent true image distortion.
Smart Images
Figure CN117876316B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of image analysis, and particularly relates to a no-reference image quality evaluation method and system based on a quaternion convolutional neural network. BACKGROUND
[0002] Methods for image quality evaluation mainly include subjective quality evaluation and objective quality evaluation. The subjective quality evaluation refers to judging the visual quality of an image according to human subjective will, and the objective quality evaluation refers to automatically calculating the visual quality of an image through an objective algorithm. According to whether the information of a reference image is needed when calculating the visual quality of an image, the objective quality evaluation can be divided into three categories: full-reference quality evaluation (FR-IQA), reduced-reference quality evaluation (RR-IQA) and no-reference quality evaluation (NR-IQA). The FR-IQA algorithm and the RR-IQA algorithm mainly analyze the visual features of an image, quantify the difference between a reference image and a distorted image, and calculate the visual quality of the distorted image. Compared with the FR-IQA algorithm and the RR-IQA algorithm, the NR-IQA algorithm does not need any information of a reference image when calculating the visual quality of a distorted image, and has a wider application prospect in actual application systems.
[0003] In recent years, with the development of deep learning technology, among which, the convolutional neural network is one of the most widely used deep learning models in the image field, and the convolutional neural network (CNN) has been widely used in the field of image quality evaluation. The powerful feature extraction and learning ability of the CNN have achieved good results in the past, but the real number convolutional neural network destroys the correlation between the color features of the real environment in the process of convolution calculation, ignores the spatial dependence between the color channels, and the loss of this color space correlation is irreversible. Studies have shown that the correlation information between the color channels of a color image is very important in the image processing process. SUMMARY
[0004] The application provides a no-reference image quality evaluation method and system based on a quaternion convolutional neural network, which is used to solve the technical problem of color feature loss of a color image caused by using a convolutional neural network to process an image in the existing general no-reference image analysis.
[0005] In a first aspect, the present application provides a quaternion convolutional neural network-based no-reference image quality assessment method and system, comprising: obtaining at least one original image, converting the at least one original image into quaternion form to obtain at least one pure quaternion matrix; performing feature extraction on the at least one pure quaternion matrix based on a preset improved ResNet network to obtain first, second, third and fourth scale features stage1, stage2, stage3 and stage4; performing first feature fusion on the first and second scale features stage1 and stage2, and performing second feature fusion on the second target scale feature stage2' obtained by the fusion and the third scale feature stage3 to obtain a third target scale feature stage3'; performing channel dimension reduction processing on the first scale feature stage1, the second target scale feature stage2' and the third target scale feature stage3' respectively; extracting feature vectors in the processed first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3' and the fourth scale feature stage4 based on a preset feature vector ratio and a preset feature vector size, and aggregating the extracted feature vectors to obtain a target feature vector; inputting the target feature vector into a preset fully connected neural network, wherein the fully connected neural network contains a Dropout layer and at least one fully connected layer.
[0006] In a second aspect, the present application provides a quaternion convolutional neural network-based no-reference image analysis system, comprising: an acquisition module configured to acquire at least one original image, convert the at least one original image into a quaternion form, and obtain at least one pure quaternion matrix; an extraction module configured to perform feature extraction on the at least one pure quaternion matrix based on a preset improved ResNet network, and obtain a first scale feature stage1, a second scale feature stage2, a third scale feature stage3, and a fourth scale feature stage4; a fusion module configured to perform first feature fusion on the first scale feature stage1 and the second scale feature stage2, and perform second feature fusion on a second target scale feature stage2' obtained by the fusion and the third scale feature stage3, to obtain a third target scale feature stage3'; a processing module configured to perform channel dimension reduction processing on the first scale feature stage1, the second target scale feature stage2', and the third target scale feature stage3', respectively; an aggregation module configured to extract feature vectors in the first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3', and the fourth scale feature stage4 based on a preset feature vector ratio and a preset feature vector size, and aggregate the extracted feature vectors to obtain a target feature vector; and an output module configured to input the target feature vector into a preset fully connected neural network, wherein the fully connected neural network contains a Dropout layer and at least one fully connected layer, and the preset fully connected neural network maps and outputs a quality score corresponding to the target feature vector.
[0007] In a third aspect, an electronic device is provided, comprising: at least one processor, and a memory connected to the at least one processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of any one of the embodiments of the present application.
[0008] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the program instructions are executed by a processor to enable the processor to perform the steps of any one of the embodiments of the present application.
[0009] The quaternion convolutional neural network-based no-reference image quality evaluation method and system provided by the application uses pure quaternions to represent a color image, processes the three color channels of the color image as a whole, introduces a quaternion convolutional neural network to perform feature extraction, and uses quaternion algebra operations to perform feature extraction on the image, so that the pixels of the image can be in the form of pure quaternions in the entire convolution process, the correlation and integrity of the three color channels are maintained, and layer-by-layer multi-scale feature fusion and low-dimensional feature extraction are adopted to improve the accuracy and robustness of the no-reference image real distortion. BRIEF DESCRIPTION OF DRAWINGS
[0010] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.
[0011] Figure 1 A flowchart of a quaternion convolutional neural network-based no-reference image quality evaluation method and system provided by an embodiment of the application is provided.
[0012] Figure 2 A structure block diagram of a down-sampling residual block and an identity residual block of an improved ResNet network of a specific embodiment provided by an embodiment of the application is provided.
[0013] Figure 3 A structure block diagram of a quaternion convolutional neural network-based no-reference image analysis system provided by an embodiment of the application is provided.
[0014] Figure 4 A structure diagram of an electronic device provided by an embodiment of the application is provided. DETAILED DESCRIPTION
[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the application more clear, the following will combine the drawings in the embodiments of the application to clearly and completely describe the technical solutions in the embodiments of the application. Obviously, the described embodiments are some embodiments of the application, not all embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.
[0016] Please refer to Figure 1 which shows a flowchart of a quaternion convolutional neural network-based no-reference image quality evaluation method and system provided by the application.
[0017] As Figure 1As shown, the quaternion convolutional neural network-based no-reference image quality assessment method and system specifically comprises the following steps:
[0018] In step S101, at least one original image is obtained, and the at least one original image is converted into a quaternion form to obtain at least one pure quaternion matrix.
[0019] In this step, the pixels in the at least one original image are normalized to between 0 and 1; the RGB channels of each normalized pixel are taken as the three imaginary parts of a quaternion respectively, and the real part is 0, to form at least one pure quaternion matrix corresponding to the at least one original image, wherein the expression of the pure quaternion matrix is: Q = xi + yj + zk, wherein Q is a quaternion, x, y, and z are image R channel data, G channel data, and B channel data respectively, and i, j, and k are quaternion unit bases.
[0020] In step S102, the at least one pure quaternion matrix is subjected to feature extraction based on a preset improved ResNet network to obtain first scale features stage1, second scale features stage2, third scale features stage3, and fourth scale features stage4.
[0021] In this step, the improved ResNet network contains a quaternion convolution kernel for extracting features of an input image, a quaternion pooling layer for halving the resolution of a feature map, an identity residual block, and a plurality of continuously stacked down-sampling residual blocks.
[0022] It should be noted that the at least one pure quaternion matrix is subjected to feature extraction by the conv2_10 convolution layer, the conv3_12 convolution layer, the conv4_18 convolution layer, and the conv5_9 convolution layer in the improved ResNet network to obtain the first scale features stage1, the second scale features stage2, the third scale features stage3, and the fourth scale features stage4.
[0023] Specifically, the network architecture of the improved ResNet network is referenced to the classic image recognition network ResNet, then all real number convolutional neural network parts are replaced with quaternion convolutional neural network parts, the input data is a quaternion matrix, the convolution kernel is expanded to a quaternion convolution kernel, the convolution layer and the pooling layer are replaced with a quaternion convolution layer and a quaternion pooling layer, and the entire process maintains quaternion algebraic operations, wherein the structure block diagram of the down-sampling residual block and the identity residual block of the improved ResNet network is as shown in Figure 2
[0024] In one specific embodiment, the selected image quality assessment dataset is two real distortion datasets LIVE and CSIQ and real distortion dataset LIVEC, for the LIVEC dataset, 35 image blocks with a size of 224x224 pixels are randomly cropped from each image and horizontally flipped. The LIVE and CSIQ datasets are randomly cropped 55 and 50 image blocks respectively, and all image blocks inherit the MOS or DMOS of the original image. Then 80% of the data in the dataset is randomly selected as the training set, 10% of the data is randomly selected as the test set, and the remaining 10% of the data is the verification set. The training set is input into the improved ResNet network, and the improved ResNet network is iteratively trained, then the test set tests the improved ResNet network, and finally the improved ResNet network is verified based on the verification set.
[0025] In step S103, the first scale feature stage1 and the second scale feature stage2 are first fused, and the fused second target scale feature stage2' is second fused with the third scale feature stage3 to obtain a third target scale feature stage3'.
[0026] In this step, the first scale feature stage1 is first processed with 256 convolution kernels with a size of 256x3x3 and a step of 2; the second scale feature stage2 is second processed with 256 convolution kernels with a size of 512x1x1; the first scale feature stage1 after the first processing and the second scale feature stage2 after the second processing are preliminarily fused to obtain a first fused feature, and the number of channels of the first fused feature is reduced by half according to 128 convolution kernels with a size of 256x1x1, then the number of channels of the first fused feature is expanded by 2 times using 256 convolution kernels with a size of 128x1x1, and the expanded first fused feature is channel weighted to obtain a second target scale feature stage2'; the second target scale feature stage2' is third processed with 256 convolution kernels with a size of 256x3x3 and a step of 2; the third scale feature stage3 is fourth processed with 256 convolution kernels with a size of 512x1x1; the second target scale feature stage2' after the third processing and the third scale feature stage3 after the second processing are preliminarily fused to obtain a second fused feature, and the number of channels of the first fused feature second fused feature is reduced by half according to 128 convolution kernels with a size of 256x1x1, then the number of channels of the second fused feature is expanded by 2 times using 256 convolution kernels with a size of 128x1x1, and the expanded second fused feature is channel weighted to obtain a third target scale feature stage3'.
[0027] In practical applications, the low-level features in adjacent scale features are down-sampled and fused with high-level features. By layer-by-layer fusing multi-scale features to construct a layer-by-layer transmission of detail information flow, the fused features can better represent the local distortion between the receptive fields of adjacent scale features, and improve the representation of image quality by high-level features. The specific process is as follows: in order to keep the resolution and channel number of the to-be-fused features consistent for subsequent addition fusion, 256 convolution kernels with a size of 256x3x3 are used for down-sampling of the first scale feature stage1 with a step of 2, and 256 convolution kernels with a size of 512x1x1 are used for channel dimension reduction of the second scale feature stage2. The two preprocessed input features are added for fusion. Then input the FFM, first reduce the channel number of the preliminary fused feature by half by using 128 convolution kernels with a size of 256x1x1, then further fuse the feature by using 128 convolution kernels with a size of 128x3x3 and a step of 1, and then expand the channel number of the fused feature by 2 times by using 256 convolution kernels with a size of 128x1x1. This part of processing uses the Bottleneck structure, the purpose of which is to reduce the network parameter quantity. Finally, the channel attention SE Block module is used to weight the fused feature in the channel dimension to obtain the fused feature, which is called the second target scale feature stage2', so that the network selectively enhances the features that can better represent the image quality, fully activates these features and suppresses other features in the channel dimension. The fusion method of the second target scale feature stage2' and the third scale feature stage3 is also the same as the fusion process of the first scale feature stage1 and the second scale feature stage2, so as to obtain the third target scale feature stage3'. In this way, the second target scale feature stage2' adds adjacent low-level features, and more detailed information is added to the original semantic information, which can accurately describe the local distortion of the image. The third target scale feature stage3' adds the fused adjacent resolution features, which can effectively integrate the information of adjacent layers and obtain the information representing the image quality on multiple receptive fields.
[0028] In step S104, the first scale feature stage1, the second target scale feature stage2', and the third target scale feature stage3' are respectively subjected to channel dimension reduction processing.
[0029] In this step, for the first scale feature stage1, the second target scale feature stage2' and the third target scale feature stage3', first, the number of channels is reduced to 128, reducing the model parameter quantity; then the channel attention SE Block module is used for channel weighting, highlighting the feature map that can better distinguish the image distortion; then the channel dimension is reduced again, and finally, global average pooling and average pooling operation with step size 7 and size 7x7 are performed on the reduced features, respectively, to obtain the global low-dimensional quality perception feature and the local low-dimensional quality perception feature of the image at this resolution.
[0030] In step S105, the feature vectors in the processed first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3' and the fourth scale feature stage4 are extracted based on the preset feature vector proportion and the preset feature vector size, respectively, and the extracted feature vectors are aggregated to obtain a target feature vector.
[0031] In this step, the feature vectors in the processed first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3' and the fourth scale feature stage4 are extracted based on the feature vector proportion of 1:1:1:4 and the feature vector size of 56, 28, 28, 28, respectively.
[0032] In step S106, the target feature vector is input into a preset fully connected neural network, and the quality score corresponding to the target feature vector is mapped and output by the fully connected neural network, wherein the fully connected neural network comprises a Dropout layer and at least one fully connected layer.
[0033] In this step, in order to balance the network parameter quantity and the network fitting ability, a fully connected neural network composed of three fully connected layers is used for regression, wherein the first fully connected layer has a Dropout layer in front of it with a node retention probability of 0.5 to avoid overfitting; ReLU is used as the activation function for the first two fully connected layers. The fully connected layers FC1, FC2 and FC3 with output dimensions of 56, 14 and 1, respectively, are obtained, and then the predicted quality score is finally obtained. The performance of the model is calculated, and the model evaluation indicators are the Pearson linear correlation coefficient (PLCC) and the Spearman rank correlation coefficient (SRCC).
[0034] In summary, the method of the present application uses pure quaternions to represent color images, processes the three color channels of the color image as a whole, and introduces a quaternion convolutional neural network for feature extraction. During the entire convolution process, the pixels of the image are in the form of pure quaternions, maintaining the correlation and integrity of the three color channels. The use of layer-by-layer multi-scale feature fusion and low-dimensional feature extraction further improves the accuracy and robustness of the no-reference image distortion.
[0035] Referring to Figure 3 , a structure block diagram of a no-reference image analysis system based on a quaternion convolutional neural network is shown.
[0036] As Figure 3 shown, the no-reference image analysis system 200 includes an acquisition module 210, an extraction module 220, a fusion module 230, a processing module 240, an aggregation module 250, and an output module 260.
[0037] The acquisition module 210 is configured to acquire at least one original image, convert the at least one original image into a quaternion form, and obtain at least one pure quaternion matrix.
[0038] The extraction module 220 is configured to perform feature extraction on the at least one pure quaternion matrix based on a pre-set improved ResNet network, and obtain a first scale feature stage1, a second scale feature stage2, a third scale feature stage3, and a fourth scale feature stage4.
[0039] The fusion module 230 is configured to perform first feature fusion on the first scale feature stage1 and the second scale feature stage2, and perform second feature fusion on the second target scale feature stage2' obtained by the fusion and the third scale feature stage3, to obtain a third target scale feature stage3'.
[0040] The processing module 240 is configured to perform channel dimension reduction processing on the first scale feature stage1, the second target scale feature stage2', and the third target scale feature stage3', respectively.
[0041] The aggregation module 250 is configured to extract feature vectors in the processed first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3', and the fourth scale feature stage4 based on a pre-set feature vector ratio and a pre-set feature vector size, and aggregate the extracted feature vectors to obtain a target feature vector.
[0042] The output module 260 is configured to input the target feature vector into a preset fully connected neural network, the preset fully connected neural network mapping an output corresponding to a quality score of the target feature vector, wherein the fully connected neural network comprises a Dropout layer and at least one fully connected layer.
[0043] It should be understood that Figure 3 The modules described in the above detailed description and illustrated in the drawings are presented as examples. As such, the person skilled in the art will understand that changes can be made to the modules described without departing from the scope of the present application. Figure 1 The modules described in the above detailed description and illustrated in the drawings are presented as examples. As such, the person skilled in the art will understand that changes can be made to the modules described without departing from the scope of the present application. Figure 3 The modules described in the above detailed description and illustrated in the drawings are presented as examples. As such, the person skilled in the art will understand that changes can be made to the modules described without departing from the scope of the present application.
[0044] In some other embodiments, the present application also provides a computer readable storage medium having stored thereon a computer program, the program instructing a processor to execute the above-described any method embodiment of the quaternion convolutional neural network based no-reference image quality assessment method and system when the program is executed by the processor.
[0045] As an implementation, the computer readable storage medium of the present application stores computer executable instructions, the computer executable instructions being configured to:
[0046] obtaining at least one original image, converting the at least one original image into quaternion form to obtain at least one pure quaternion matrix;
[0047] extracting features from the at least one pure quaternion matrix based on a preset improved ResNet network to obtain a first scale feature stage1, a second scale feature stage2, a third scale feature stage3 and a fourth scale feature stage4;
[0048] performing first feature fusion on the first scale feature stage1 and the second scale feature stage2, and performing second feature fusion on the second target scale feature stage2' obtained by the fusion and the third scale feature stage3 to obtain a third target scale feature stage3';
[0049] respectively performing channel dimension reduction processing on the first scale feature stage1, the second target scale feature stage2' and the third target scale feature stage3';
[0050] Based on the preset feature vector ratio and preset feature vector size, feature vectors are extracted from the processed first scale feature stage1, second target scale feature stage2', third target scale feature stage3', and fourth scale feature stage4, and the extracted feature vectors are aggregated to obtain the target feature vector;
[0051] The target feature vector is input into a preset fully connected neural network, and the preset fully connected neural network maps and outputs a quality score corresponding to the target feature vector. The fully connected neural network includes a Dropout layer and at least one fully connected layer.
[0052] Computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the quaternion-based convolutional neural network-based no-reference image analysis system, etc. Furthermore, the computer-readable storage medium may include high-speed random access memory, and may also include memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer-readable storage medium may optionally include memory remotely located relative to the processor, which can be connected to the quaternion-based convolutional neural network-based no-reference image analysis system via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0053] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 4 As shown, the device includes a processor 310 and a memory 320. The electronic device may also include an input device 330 and an output device 340. The processor 310, memory 320, input device 330, and output device 340 can be connected via a bus or other means. Figure 4 Taking a bus connection as an example, memory 320 is the computer-readable storage medium described above. Processor 310 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in memory 320, thereby implementing the above-described method embodiment of a no-reference image quality assessment method and system based on a quaternion convolutional neural network. Input device 330 can receive input digital or character information and generate key signal inputs related to user settings and function control of the no-reference image analysis system based on a quaternion convolutional neural network. Output device 340 may include a display screen or other display device.
[0054] The electronic device can execute the method provided by the embodiment of the application, has the function modules and beneficial effects corresponding to the execution method. Technical details not described in detail in the embodiment can be referred to the method provided by the embodiment of the application.
[0055] As an implementation form, the electronic device is applied to a quaternion convolutional neural network-based no-reference image analysis system, and is used for a client, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:
[0056] obtain at least one original image, convert the at least one original image into a quaternion form to obtain at least one pure quaternion matrix;
[0057] perform feature extraction on the at least one pure quaternion matrix based on a preset improved ResNet network to obtain a first scale feature stage1, a second scale feature stage2, a third scale feature stage3, and a fourth scale feature stage4;
[0058] perform first feature fusion on the first scale feature stage1 and the second scale feature stage2, and perform second feature fusion on the second target scale feature stage2' obtained by the fusion and the third scale feature stage3 to obtain a third target scale feature stage3';
[0059] respectively perform channel dimension reduction processing on the first scale feature stage1, the second target scale feature stage2', and the third target scale feature stage3';
[0060] extract feature vectors in the processed first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3', and the fourth scale feature stage4 based on a preset feature vector ratio and a preset feature vector size, and aggregate the extracted feature vectors to obtain a target feature vector;
[0061] input the target feature vector into a preset fully connected neural network, wherein the preset fully connected neural network maps and outputs a quality score corresponding to the target feature vector, and the fully connected neural network comprises a Dropout layer and at least one fully connected layer.
[0062] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of the various embodiments or some parts of the embodiments.
[0063] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A quaternion convolutional neural network based no-reference image quality assessment method, characterized in that, The method comprises: obtaining at least one original image, converting the at least one original image into quaternion form to obtain at least one pure quaternion matrix; performing feature extraction on the at least one pure quaternion matrix based on a preset improved ResNet network to obtain first scale features stage1, second scale features stage2, third scale features stage3, and fourth scale features stage4; performing first feature fusion on the first scale features stage1 and the second scale features stage2, and performing second feature fusion on the second target scale features stage2' obtained by the fusion and the third scale features stage3 to obtain third target scale features stage3'; respectively performing channel dimension reduction processing on the first scale features stage1, the second target scale features stage2', and the third target scale features stage3'; extracting feature vectors in the processed first scale features stage1, the second target scale features stage2', the third target scale features stage3', and the fourth scale features stage4 based on a preset feature vector ratio and a preset feature vector size, and aggregating the extracted feature vectors to obtain target feature vectors; inputting the target feature vectors into a preset fully connected neural network, wherein the fully connected neural network maps and outputs a quality score corresponding to the target feature vectors, and the fully connected neural network comprises a Dropout layer and at least one fully connected layer.
2. The method of claim 1, wherein the method is based on a quaternion convolutional neural network. The method comprises: normalizing pixels in the at least one original image to between 0 and 1; taking the RGB channels of each normalized pixel as three imaginary parts of a quaternion, with the real part being 0, to form at least one pure quaternion matrix corresponding to the at least one original image, wherein the expression of the pure quaternion matrix is: ， In the formula, is a quaternion, x, y, z are respectively image R channel data, G channel data, B channel data, i, j, k are all quaternion unit bases.
3. The method of claim 1, wherein the method is based on a quaternion convolutional neural network. The improved ResNet network comprises a quaternion convolution kernel for extracting features of an input image, a quaternion pooling layer for halving the resolution of a feature map, an identity residual block, and a plurality of down-sampling residual blocks stacked in series; The method comprises: performing feature extraction on the at least one pure quaternion matrix through a conv2_10 convolution layer, a conv3_12 convolution layer, a conv4_18 convolution layer, and a conv5_9 convolution layer in the improved ResNet network to obtain the first scale features stage1, the second scale features stage2, the third scale features stage3, and the fourth scale features stage4.
4. The method of claim 1, wherein, Before the at least one pure quaternion matrix is subjected to feature extraction based on a preset improved ResNet network to obtain a first scale feature stage1, a second scale feature stage2, a third scale feature stage3 and a fourth scale feature stage4, the method further comprises: constructing a real distortion dataset and a synthetic distortion dataset; inputting each first image in the real distortion dataset and each second image in the synthetic distortion dataset into an improved ResNet network, and iteratively training the improved ResNet network.
5. The method of claim 1, wherein, The first feature fusion of the first scale feature stage1 and the second scale feature stage2, and the second feature fusion of the second target scale feature stage2' obtained by the fusion and the third scale feature stage3, to obtain a third target scale feature stage3' comprises: first processing of the first scale feature stage1 using 256 convolution kernels with a size of 256x3x3 and a step of 2; second processing of the second scale feature stage2 using 256 convolution kernels with a size of 512x1x1; preliminary fusion of the first scale feature stage1 after the first processing and the second scale feature stage2 after the second processing to obtain a first fusion feature, reduction of the channel number of the first fusion feature by half according to 128 convolution kernels with a size of 256x1x1, expansion of the channel number of the first fusion feature by two times using 256 convolution kernels with a size of 128x1x1, and channel weighting of the expanded first fusion feature to obtain a second target scale feature stage2'; third processing of the second target scale feature stage2' using 256 convolution kernels with a size of 256x3x3 and a step of 2; fourth processing of the third scale feature stage3 using 256 convolution kernels with a size of 512x1x1; preliminary fusion of the second target scale feature stage2' after the third processing and the third scale feature stage3 after the second processing to obtain a second fusion feature, reduction of the channel number of the first fusion feature second fusion feature by half according to 128 convolution kernels with a size of 256x1x1, expansion of the channel number of the second fusion feature by two times using 256 convolution kernels with a size of 128x1x1, and channel weighting of the expanded second fusion feature to obtain a third target scale feature stage3'.
6. The method of claim 1, wherein, After the channel dimension reduction processing of the first scale feature stage1, the second target scale feature stage2' and the third target scale feature stage3' respectively, the method further comprises: and performing global average pooling and average pooling operation with preset step and preset size on the first scale feature stage1, the second target scale feature stage2' and the third target scale feature stage3' after the channel dimension reduction processing respectively.
7. The method of claim 1, wherein, The feature vectors in the first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3', and the fourth scale feature stage4 after the preset feature vector proportion and the preset feature vector size are extracted include: The feature vectors in the first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3', and the fourth scale feature stage4 after the preset feature vector proportion and the preset feature vector size are extracted include:
8. A quaternion convolutional neural network based no-reference image quality assessment system, characterized in that, The method comprises: The acquisition module is configured to acquire at least one original image, convert the at least one original image into a quaternion form, and obtain at least one pure quaternion matrix; The extraction module is configured to perform feature extraction on the at least one pure quaternion matrix based on a preset improved ResNet network, and obtain a first scale feature stage1, a second scale feature stage2, a third scale feature stage3, and a fourth scale feature stage4; The fusion module is configured to perform first feature fusion on the first scale feature stage1 and the second scale feature stage2, and perform second feature fusion on the second target scale feature stage2' obtained by the fusion and the third scale feature stage3, to obtain a third target scale feature stage3'; The processing module is configured to perform channel dimension reduction processing on the first scale feature stage1, the second target scale feature stage2', and the third target scale feature stage3', respectively; The aggregation module is configured to extract feature vectors in the first scale feature stage1, the second target scale feature stage2', the third target scale feature stage3', and the fourth scale feature stage4 after processing based on a preset feature vector proportion and a preset feature vector size, and aggregate the extracted feature vectors to obtain a target feature vector; The output module is configured to input the target feature vector into a preset fully connected neural network, and the preset fully connected neural network maps and outputs a quality score corresponding to the target feature vector, wherein the fully connected neural network comprises a Dropout layer and at least one fully connected layer.
9. An electronic device, comprising: The method comprises: The program is executed by the processor to implement the method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1 to 7.