Image color aesthetics evaluation method based on multi-scale feature fusion

Abstract

The invention relates to the technical field of image color aesthetics evaluation, in particular to an image color aesthetics evaluation method based on multi-scale feature fusion, and the method comprises the following steps: S1, converting the image data of an input image into the sequence data of the input image; s2, performing n times of feature extraction operation on the sequence data of the input image to obtain n multi-scale features, and performing vector addition to obtain an aggregation feature F; s3, performing feature weighted fusion operation on the aggregated feature F to obtain a calibration feature # imgabs0 #; and S4, performing image color aesthetic evaluation on the input image by using the calibration feature # imgabs1 #. In conclusion, color information in the image can be captured more comprehensively. Different color details can obtain different degrees of attention according to learnable weight distribution, and key color information is ensured to be preferentially considered. Therefore, the model can understand the color information more comprehensively in the whole image, so that the overall perception capability of the model is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image color aesthetics evaluation, and in particular to an image color aesthetics evaluation method based on multi-scale feature fusion. Background Art

[0002] With the advancement of digital photography equipment and image processing technology, image aesthetics, especially color rendering, has gradually become an important topic in image processing and evaluation. Existing photography equipment and image signal processors (ISPs) can improve the visual effect of images through algorithms to compensate for problems such as insufficient lighting. However, these adjustments often bring deviations in color reproduction, making the same scene show significant color differences on different devices. Therefore, the color rendering effect not only affects the image quality, but is also directly related to the beauty of the image, which has also promoted the development of the emerging field of image color aesthetic assessment (ICAA). ICAA specifically evaluates the color quality or color preference of images, which involves evaluating key factors such as color harmony, color combinations, and their overall impact on color perception. In the ICAA task, an important challenge is to be able to establish a global perception of complex color combinations in the image, so as to extract a more comprehensive color aesthetic representation to guide the model to make accurate evaluations that conform to the human subjective perspective.

[0003] Traditional color aesthetic evaluation methods usually rely on simple pixel statistics or color histograms to evaluate color quality, but these methods have some limitations. Although pixel statistics can calculate the distribution characteristics of the colors of each pixel in the image, it cannot capture the overall aesthetic effect of the color combination. Similarly, although the color histogram reflects the color characteristics of the image, it cannot express the subtle relationship and matching effect between different colors. These traditional methods are difficult to effectively capture human color aesthetic preferences, because color aesthetics often involves multi-dimensional perceptual experience, which includes not only the presentation of the color itself, but also the matching and global combination between colors. This multi-level perceptual experience requires more advanced models to achieve.

[0004] For example, a Chinese patent document with publication number CN117315438A discloses an image color aesthetics evaluation method, device and equipment based on interest points, including: dividing an original image color aesthetics evaluation data set into a training set and a test set; performing data preprocessing on the training set to obtain a target training set, and performing data preprocessing on the test set to obtain a target test set; constructing an image color aesthetics evaluation model based on interest point allocation; performing model training on the image color aesthetics evaluation model based on interest point allocation according to the target training set to obtain a primary evaluation model, and performing parameter adjustment processing on the primary evaluation model according to the target test set to obtain a target evaluation model; obtaining image color aesthetics data to be processed; inputting the image color aesthetics data to be processed into the target evaluation model to obtain an image color aesthetics evaluation result.

[0005] Among the existing technical methods, there is a Transformer-based image color aesthetics assessment method. This method uses a delegate transformer to adaptively assign interest points and learns human segmentation behavior of color space through a dedicated module to improve the accuracy of color aesthetics assessment. However, this method mainly focuses on the key color information in the image, resulting in some color information being ignored, which affects the comprehensiveness and accuracy of the assessment.

[0006] In order to overcome the limitations of traditional methods, it is necessary to optimize and improve image color aesthetic assessment (ICAA). Summary of the invention

[0007] In order to overcome the above-mentioned shortcomings, the present invention aims to provide a technical solution that can solve the above-mentioned problems.

[0008] The present invention provides an image color aesthetics evaluation method based on multi-scale feature fusion, comprising the following steps: S1, converting image data of an input image into sequence data of the input image; S2, perform n feature extraction operations on the sequence data of the input image to obtain n multi-scale features, and perform vector addition to obtain the aggregated feature F; S3, perform feature weighted fusion operation on the aggregated feature F to obtain the calibration feature ; S4, using calibration features Perform image color aesthetic evaluation on the input image.

[0009] As a further solution of the present invention: in step S3, a feature weighted fusion operation is performed on the aggregated feature F, which specifically includes the following steps: S3.1, perform feature weighting operation on the aggregated feature F to generate the attention weight W; S3.2, use the attention weight W to calibrate the multi-scale features; S3.3, perform fusion operation on the calibrated multi-scale features to obtain the calibrated features .

[0010] As a further solution of the present invention: in step S3.1, a feature weighting operation is performed on the aggregated feature F, which specifically includes the following steps: the aggregated feature F is calculated and processed using global average pooling to generate a global feature descriptor; the global feature descriptor is then mapped through a multi-layer perceptron, and the mapped data is then converted through a Softmax function to generate an attention weight W.

[0011] As a further solution of the present invention: in step S3.2, a calibration operation is performed on the multi-scale feature, specifically comprising the following steps: calibrating the nth multi-scale feature F n , using the corresponding nth attention weight W n Perform scalar multiplication to obtain the nth calibration feature .

[0012] As a further solution of the present invention: in step S3.3, the calibrated multi-scale features are fused, specifically comprising the following steps: Perform vector addition to obtain calibration features .

[0013] As a further solution of the present invention: in step S4, using the calibration feature Perform image color aesthetic evaluation on the input image, which includes the following steps: Use global average pooling and multi-layer perceptron to learn calibration features The mapping relationship between the aesthetic score and the regression Convert feature mappings into final aesthetic scores.

[0014] As a further solution of the present invention: in step S2, a feature extraction operation is performed on the sequence data of the input image, which specifically includes the following steps: using VMamba blocks to perform continuous data processing operations on the sequence data of the input image, and after multiple data processing operations, performing a feature extraction operation.

[0015] As a further solution of the present invention: in step S2, a feature extraction operation is performed on the sequence data of the input image, which specifically includes the following steps: the sequence data of the input image is subjected to a feature extraction operation once every time it undergoes data processing operations of four VMamba blocks, and a multi-scale feature is generated. After n feature extraction operations, n multi-scale features (F1, F2, ..., F n ).

[0016] As a further solution of the present invention: in step S2, the data processing operation performed on the sequence data of the input image using the VMamba block specifically includes the following steps: S2.1, preprocess the sequence data using normalized data preprocessing; S2.2, using linear projection and activation function to process the preprocessed sequence data to obtain first data; S2.3, sequentially processing the preprocessed sequence data using linear projection, depthwise separable convolution, and state-space model to obtain second data; S2.4, perform element-by-element multiplication and merging of the first data and the second data, perform linear projection, and then perform residual connection and merging with the sequence data that has not been preprocessed.

[0017] As a further solution of the present invention: in steps S2.3 and S2.4, the following steps are also included: using linear projection, forward convolution and forward state space model to process the preprocessed sequence data to obtain third data; using linear projection, reverse convolution and backward state space model to process the preprocessed sequence data to obtain fourth data; using the first data to perform element-by-element multiplication and merging with the third data and the fourth data respectively, and performing vector addition processing on the results of the two, and then performing linear projection, and then performing residual connection and merging with the sequence data that has not been preprocessed.

[0018] Compared with the prior art, the present invention has the following beneficial effects: This paper proposes an image color aesthetic assessment method based on multi-scale feature fusion, aiming to accurately and efficiently provide a better aesthetic perception of color information in an image. The method consists of three key components. The first component is a feature extraction network based on Vision Mamba (VMamba), which serves as the core module for extracting multi-scale color features. The second component is a weighted feature fusion module (WFFM), which dynamically assigns learnable weights to multi-scale features to provide a more accurate representation for aesthetic evaluation. The third component is a regressor that uses the recalibrated features to predict aesthetics and perform aesthetic evaluation on image colors.

[0019] The image color aesthetics assessment method of the present invention can capture the color information in the image more comprehensively. Different color details can receive different degrees of attention according to the learnable weight distribution, ensuring that key color information is given priority. This method enables the model to have a more comprehensive understanding of color information in the entire image, thereby enhancing its overall perception ability.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 It is a schematic block diagram of the overall process of the present invention; Figure 2 It is a schematic flow chart of the feature weighted fusion operation of the present invention; Figure 3 It is a schematic flow chart of the feature extraction operation of the present invention; Figure 4 is another schematic flow chart of the feature extraction operation of the present invention; Figure 5 It is a schematic diagram of the model framework of the present invention. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figures 1 to 5 In an embodiment of the present invention, a method for image color aesthetics evaluation based on multi-scale feature fusion includes the following steps: S1, converting image data of an input image into sequence data of the input image; S2, perform n feature extraction operations on the sequence data of the input image to obtain n multi-scale features, and perform vector addition to obtain the aggregated feature F; S3, perform feature weighted fusion operation on the aggregated feature F to obtain the calibration feature ; S4, using calibration features Perform image color aesthetic evaluation on the input image.

[0025] Specifically, in order to overcome the limitations of traditional image color aesthetic assessment methods, the present invention explores the emerging basic model - state space model (Mamba) in the context of image color aesthetic assessment (ICAA). The Mamba model has a powerful ability to process complex sequence data, has a global receptive field, and has linear computational complexity when processing long sequence data, making it particularly suitable for capturing multi-level and global color information in images.

[0026] Therefore, the present invention proposes an image color aesthetics evaluation method based on the Mamba model and multi-scale feature fusion, aiming to accurately and efficiently perform better aesthetic perception of color information in images. Figure 5 The model architecture consists of three key components. The first component is a feature extraction network based on Vision Mamba (VMamba), which is the core module for extracting multi-scale color features. The second component is a weighted feature fusion module (WFFM), which dynamically assigns learnable weights to multi-scale features to provide more accurate feature representation for aesthetic evaluation. The third component is a regressor ,It utilizes the recalibrated features to predict the aesthetic ,evaluation of image colors.

[0027] In addition, it should be noted that the model mentioned in the present invention refers to an image data processing model produced according to the image color aesthetics evaluation method provided by the present invention.

[0028] like Figure 1 As shown, preferably, in step S1, the image data of the input image is converted into sequence data of the input image, which specifically includes the following steps: the input image is divided into multiple image blocks of fixed size, each image block is subjected to feature extraction through a convolution layer, thereby being converted into a feature vector of fixed dimension, the extracted feature vector is linearly projected to obtain a vector representation of the image block, and the vector representations of multiple image blocks are combined to form the sequence data of the input image.

[0029] Specifically, you can use ‌Patch Embedding‌ to convert the image data of the input image to obtain the sequence data of the input image. ‌Patch Embedding‌ is the process of dividing an image into multiple small patches (patches) and converting each small patch into a fixed-length vector representation. Its basic principles and processes include: ‌Split the image‌: First, split the input image into multiple patches of a fixed size. For example, if the image size is 224x224 pixels, you can split the image into patches of 16x16 pixels.

[0030] ‌Feature Extraction‌: Each patch is passed through a convolutional layer for feature extraction, usually with the same kernel size and stride as the patch size. In this way, each patch is converted into a feature vector of fixed dimension.

[0031] ‌Linear Projection‌: The extracted feature vector is linearly projected to obtain the final embedding representation. This step usually involves a fully connected layer to map the dimension of the feature vector to a fixed dimension.

[0032] Secondly, Patch Embedding can convert image data into sequence data, so that the model made according to the method provided by the present invention can directly process sequence data, thereby using its powerful context understanding ability to process image data. In addition, Patch Embedding can also improve the generalization ability of the model, enabling it to better process images of different input sizes and resolutions.

[0033] In another embodiment, preferably, in step S1, the position information of each image block in the input image is encoded to form a position vector, and the position vector and the vector representation of the image block are added to form sequence data with position information.

[0034] Specifically, in the model, although each element in the sequence data can be processed in parallel, the relative position information between elements cannot be directly perceived. Therefore, position embedding can be introduced to encode the position information into a vector and add the vector to the embedding vector of each element, thereby retaining the order information of each element and allowing the model to learn the relative importance of elements at different positions. Position Embedding is an existing technology used in the model to represent the position information of each element or token in the sequence.

[0035] like Figure 1 and Figure 3 As shown, preferably, in step S2, a feature extraction operation is performed on the sequence data of the input image, specifically comprising the following steps: using VMamba blocks to perform continuous data processing operations on the sequence data of the input image, and after multiple data processing operations, a feature extraction operation is performed. For example, the following scheme can be adopted: each time the sequence data of the input image undergoes data processing operations of four consecutive VMamba blocks, a feature extraction operation is performed, and a multi-scale feature is generated, and the above steps are continuously performed until after n feature extraction operations, n multi-scale features (F1, F2, ..., F n ).

[0036] Specifically, the present invention uses a bidirectional Vision Mamba (VMamba) block as the core of a visual encoder for feature extraction, and can perform feature extraction at different scales after multiple data processing, thereby forming n multi-scale features. Vision Mamba is a deep learning model for visual tasks, and is commonly used for tasks such as object detection and image classification in the field of computer vision. Its design concept is to improve processing speed and accuracy while reducing the demand for computing resources.

[0037] Secondly, multi-scale features refer to extracting image features at different scales to capture structural and texture information of different sizes, and obtain information at different levels by changing the scale of the image. Features at different scales can provide different detailed information about the image, thereby describing the image more comprehensively and accurately. Multi-scale features can capture texture and structural features at different scales and improve the accuracy of image data.

[0038] like Figure 3 As shown, preferably, in step S2, the data processing operation performed on the sequence data of the input image using the VMamba block specifically includes the following steps: S2.1, preprocess the sequence data using normalized data preprocessing; S2.2, using linear projection and activation function to process the preprocessed sequence data to obtain first data; S2.3, sequentially processing the preprocessed sequence data using linear projection, depthwise separable convolution, and state-space model to obtain second data; S2.4, perform element-by-element multiplication and merging of the first data and the second data, perform linear projection, and then perform residual connection and merging with the sequence data that has not been preprocessed.

[0039] Specifically, for each image block marker in the sequence data of the input image, normalization data preprocessing is first performed, and then split into two branches. The first branch processes the data through a linear layer and an activation function (SiLU); the second branch processes the data using a linear layer, a depthwise separable convolution, and a state-space model. The data results processed by the two branches are then merged by element-by-element multiplication, and then merged through a linear layer and a residual connection. This design enables the VMamba block to effectively model the global visual context and enhance the ability to perceive spatial information.

[0040] Secondly, normalization is a data preprocessing technology that aims to scale the numerical range of data to a specific scale, usually to eliminate the dimensional differences between different features, making the data more suitable for subsequent analysis and processing. It can significantly improve the training efficiency and performance of the model, and provide strong support for the optimization of deep learning models and the improvement of generalization capabilities.

[0041] Again, the activation function (SiLU) combines the advantages of ReLU and Sigmoid functions, has smooth and non-monotonic characteristics, and can provide better numerical stability and gradient propagation. The State Space Model (SSM) is a statistical model used to describe time series data, mainly used to deal with dynamic systems and time-varying processes. It is a model for sequence prediction tasks that can solve the problem that the traditional Transformer model is insensitive to discrete deep potential features, thereby establishing a stronger logic for time series data.

[0042] In addition, depthwise separable convolution refers to ‌Convld, which refers to ‌Linear Deformable Convolution (LDConv)‌, which is an improved convolution operation designed to address the limitations of standard convolution and deformable convolution. Although deformable convolution solves the problem of fixed sampling, its number of parameters grows quadratically with the increase of the convolution kernel size, and the computational efficiency is low. LDConv overcomes these problems by giving the convolution kernel an arbitrary number of parameters and arbitrary sampling shapes, making the number of parameters grow linearly‌. LDConv can replace traditional convolution operations and improve network performance‌.

[0043] like Figure 4 As shown, preferably, in steps S2.3 and S2.4, the following steps are also included: using linear projection, forward convolution and forward state space model to process the preprocessed sequence data to obtain third data; using linear projection, reverse convolution and backward state space model to process the preprocessed sequence data to obtain fourth data; using the first data to perform element-by-element multiplication and merging with the third data and the fourth data respectively, and performing vector addition processing on the results of the two, and then performing linear projection, and then performing residual connection and merging with the sequence data that has not been preprocessed.

[0044] Specifically, forward convolution and backward convolution can also be introduced in the depthwise separable convolution process. Forward convolution refers to the process in which input data (such as sequence data of input images) is convolved with convolution kernels (filters) through convolution layers in forward propagation to generate feature maps. Thus, the features of the input data are extracted through forward convolution so that they can be processed by subsequent layers of the network. Backward convolution refers to the process used in the backward propagation stage to calculate the gradient of the loss function and update the model parameters (such as convolution kernels). The calculation of backward convolution can be regarded as the "reverse operation" of the forward convolution process, and its purpose is to transfer the gradient from the output feature map back to the input feature map and convolution kernel. Thus, the self-optimized gradient is obtained through backward convolution, which is used to update the parameters and improve the learning of the model.

[0045] Secondly, you can also introduce Forward SSM and Backward SSM in the state space model processing. The forward state space model is mainly used to infer the current state or output based on historical data (previous state) starting from the current moment. It usually merges new observations into previous states for processing in a recursive way. The backward state space model infers the state from the future to the past when the complete observation sequence is known. It is usually used to improve existing estimates.

[0046] In addition, the state-space model can also include activation functions (SiLU) and 2D-Selective-Scan (SS2D) modules. The SS2D module selectively scans specific areas of the input image instead of processing the entire image uniformly. It can more effectively utilize computing resources and focus on areas that are more likely to contain targets or important features, thereby improving the processing efficiency and accuracy of areas of interest in the image. For example, the SS2D module can expand the image patch in four different directions: from the upper left corner to the lower right corner, from the lower right corner to the upper left corner, from the upper right corner to the lower left corner, and from the lower left corner to the upper right corner, creating four different sequences.

[0047] like Figure 2 As shown, preferably, in step S3, a feature weighted fusion operation is performed on the aggregated feature F, specifically comprising the following steps: S3.1, perform feature weighting operation on the aggregated feature F to generate the attention weight W; S3.2, use the attention weight W to calibrate the multi-scale features; S3.3, perform fusion operation on the calibrated multi-scale features to obtain the calibrated features .

[0048] Specifically, the present invention does not rely on a set of fully connected layers after the last layer of features when performing quality prediction, but instead uses a set of rich multi-scale features to construct a global representation of color information. In order to integrate these multi-scale features, the present invention carefully designs a WFFM module to assign learnable weights to color features of different scales (such as Figure 5 In this way, the model can fully perceive and integrate the advantages of color features across multiple scales.

[0049] Second, for example, given Multi-scale features of , calculate the element-by-element sum of the features to get an aggregate feature , its calculation formula #(1) is defined as: ,in, It is a multi-scale feature.

[0050] In another embodiment, in step S3.1, a feature weighting operation is performed on the aggregate feature F, specifically comprising the following steps: using global average pooling (GAP) to calculate channel statistics of the aggregate feature F across spatial dimensions, thereby generating a global feature descriptor; then mapping the global feature descriptor through a multi-layer perceptron (MLP), and then transforming the mapped data through a Softmax function, thereby generating an attention weight W. In step S3.2, a calibration operation is performed on the multi-scale feature, specifically comprising the following steps: performing a calibration operation on the nth multi-scale feature F n , using the corresponding nth attention weight W n Perform scalar multiplication to obtain the nth calibration feature In step S3.3, the calibrated multi-scale features are fused, which specifically includes the following steps: Perform vector addition to obtain calibration features .

[0051] Specifically, aggregate features A global feature descriptor is generated by combining information from multi-scale streams. A global feature descriptor is a vector used to describe the entire image or feature map, which can capture the overall characteristics of the input data. For example, the number of output channels of a multi-layer perceptron (MLP) can be defined as n×c, where n is the number of features aggregated in the aggregated feature F, and c is the number of channels. The attention of each feature on the channel can be learned through the Softmax function, thereby calibrating the features.

[0052] Second, for example, given Multi-scale features of , the calculation formula of attention weight W is The multi-scale feature maps can then be adaptively recalibrated using the dynamically changing weights W.

[0053] Again, the overall process of feature aggregation and recalibration is defined by equations #(2) and #(3) as: , ,in, is a multi-scale feature, is the attention weight.

[0054] Overall, the WFFM module forms a comprehensive perception of color information in an image by aggregating and dynamically selecting key global features.

[0055] In addition, (GAP) and (MLP) in the feature weighted fusion module (WFFM) are used to learn the weights of feature calibration. Global average pooling (GAP) is a pooling operation that is applied after the last convolution layer of the convolutional neural network to take the average value of each feature map as the overall representation of the feature. For example, for each feature map, GAP calculates the average value of all elements to generate a single scalar value. Therefore, for a convolution layer with multiple feature maps, GAP outputs a vector with the same dimension as the number of feature maps, which can then be used for classification or other tasks.

[0056] Multilayer Perceptron (MLP) is a feedforward artificial neural network model that maps multiple input data sets to a single output data set. It consists of multiple layers (input layer, one or more hidden layers, and output layer), each layer contains multiple neurons (nodes), which are connected to each other through weights. It can handle a variety of tasks such as classification and regression. Its structure is relatively simple. Through reasonable design and training, MLP can achieve good results in many tasks.

[0057] ‌Softmax function‌ is a mathematical function that is mainly used to convert a set of arbitrary real numbers into real numbers representing probability distribution. The Softmax function is a commonly used activation function, and its main function is to convert a set of arbitrary real values ​​(such as the output of a neural network) into a probability distribution. The Softmax function maps the results to (0, 1) through normalization, and the sum of all outputs is 1, so it is suitable for representing the probabilities of multiple categories. It is mainly used to convert the output of a neural network into an interpretable probability distribution. It enables the model to better distinguish between categories by emphasizing larger values ​​and suppressing smaller values.

[0058] In another embodiment, in step S4, the calibration feature is used Perform image color aesthetic evaluation on the input image, which includes the following steps: Use global average pooling (GAP) and multi-layer perceptron (MLP) to learn calibration features The mapping relationship between the aesthetic score and the regression Convert feature mappings into final aesthetic scores.

[0059] Specifically, the recalibrated features are then passed through a regressor To predict the color aesthetic score, it is defined as Here, (GAP) and (MLP) are the mapping relationship between the learned calibrated features and the aesthetic scores. The regressor (Regressor) Q is the last component of the model architecture, and its main function is to convert the extracted and fused feature maps into the final aesthetic scores.

[0060] The present invention firstly adopts the emerging basic model, namely the state space model (SSM) Mamba, as the backbone for extracting global features of images. This model has linear complexity when processing long sequence data and plays a significant role in the ICAA field. Through the application of this model, global information can be effectively extracted from the input image, providing strong support for subsequent color aesthetic evaluation.

[0061] Subsequently, a WFFM module is proposed, which can assign learnable weights to multi-scale color features. The introduction of this mechanism enhances the model's ability to prioritize relevant features, so that color details can be more effectively represented in image aesthetic evaluation. In this way, the method of the present invention can more accurately capture key color information in the image and provide more accurate and comprehensive results for aesthetic evaluation.

[0062] The present invention provides an image color aesthetics evaluation method based on multi-scale feature fusion, which can achieve the following technical effects: (1) Global perception and fine-grained attention: Compared with the Transformer-based method, this paper combines the Mamba model and multi-scale feature fusion technology to achieve global perception of complex color combinations. Through dynamic weight allocation, it ensures that key color information is finely focused, thereby improving the comprehensiveness and accuracy of color aesthetic evaluation.

[0063] (2) Spatial information perception and performance optimization: By using the bidirectional Vision Mamba block to model global visual context, the present invention effectively enhances the model's ability to perceive spatial information and improves the overall performance in the field of image color aesthetics evaluation. This comprehensive design enables the present invention to have significant advantages over existing technologies in dealing with complex color combinations and improving overall aesthetic perception.

[0064] In summary, the feature extraction capability of the present invention in image color aesthetic evaluation is more comprehensive and accurate. Compared with the prior art, it can more effectively deal with complex color combinations and significantly improve the overall aesthetic perception of image color.

[0065] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

Claims

1. A method for image color aesthetics assessment based on multi-scale feature fusion, characterized in that: The steps include: S1, converting image data of an input image into sequence data of the input image; S2, perform n feature extraction operations on the sequence data of the input image to obtain n multi-scale features, and perform vector addition to obtain the aggregated feature F; S3, perform feature weighted fusion operation on the aggregated feature F to obtain the calibration feature ; S4, using calibration features Perform image color aesthetic evaluation on the input image.

2. According to claim 1, the image color aesthetics evaluation method based on multi-scale feature fusion is characterized in that: In step S3, a feature weighted fusion operation is performed on the aggregated feature F, which specifically includes the following steps: S3.1, perform feature weighting operation on the aggregated feature F to generate the attention weight W; S3.2, use the attention weight W to calibrate the multi-scale features; S3.3, perform fusion operation on the calibrated multi-scale features to obtain the calibrated features .

3. The image color aesthetics evaluation method based on multi-scale feature fusion according to claim 2 is characterized in that: In step S3.1, a feature weighting operation is performed on the aggregated feature F, which specifically includes the following steps: the aggregated feature F is calculated and processed using global average pooling to generate a global feature descriptor; the global feature descriptor is then mapped through a multi-layer perceptron, and the mapped data is converted through a Softmax function to generate an attention weight W.

4. The image color aesthetics evaluation method based on multi-scale feature fusion according to claim 2 is characterized in that: In step S3.2, the multi-scale feature is calibrated, which specifically includes the following steps: n , using the corresponding nth attention weight W n Perform scalar multiplication to obtain the nth calibration feature .

5. The image color aesthetics evaluation method based on multi-scale feature fusion according to claim 2 is characterized in that: In step S3.3, the calibrated multi-scale features are fused, which specifically includes the following steps: Perform vector addition to obtain calibration features .

6. The image color aesthetics evaluation method based on multi-scale feature fusion according to claim 1, characterized in that: In step S4, the calibration feature Perform image color aesthetic evaluation on the input image, which includes the following steps: Use global average pooling and multi-layer perceptron to learn calibration features The mapping relationship between the aesthetic score and the regression Convert feature mappings into final aesthetic scores.

7. The image color aesthetics evaluation method based on multi-scale feature fusion according to claim 1, characterized in that: In step S2, a feature extraction operation is performed on the sequence data of the input image, which specifically includes the following steps: using the VMamba block to perform continuous data processing operations on the sequence data of the input image, and after multiple data processing operations, performing a feature extraction operation.

8. The method for image color aesthetics assessment based on multi-scale feature fusion according to claim 7, characterized in that: In step S2, a feature extraction operation is performed on the sequence data of the input image, which specifically includes the following steps: the sequence data of the input image is subjected to a feature extraction operation once every time it undergoes data processing operations of four VMamba blocks, and a multi-scale feature is generated. After n feature extraction operations, n multi-scale features (F1, F2, ..., F n ).

9. The image color aesthetics evaluation method based on multi-scale feature fusion according to claim 7, characterized in that: In step S2, the data processing operation performed on the sequence data of the input image using the VMamba block specifically includes the following steps: S2.1, preprocess the sequence data using normalized data preprocessing; S2.2, using linear projection and activation function to process the preprocessed sequence data to obtain first data; S2.3, sequentially processing the preprocessed sequence data using linear projection, depthwise separable convolution, and state-space model to obtain second data; S2.4, perform element-by-element multiplication and merging of the first data and the second data, perform linear projection, and then perform residual connection and merging with the sequence data that has not been preprocessed.

10. The image color aesthetics evaluation method based on multi-scale feature fusion according to claim 9, characterized in that: In steps S2.3 and S2.4, the following steps are also included: using linear projection, forward convolution and forward state space model to process the preprocessed sequence data to obtain third data; using linear projection, reverse convolution and backward state space model to process the preprocessed sequence data to obtain fourth data; using the first data to perform element-by-element multiplication and merging with the third data and the fourth data respectively, and performing vector addition processing on the results of the two, and then performing linear projection, and then performing residual connection and merging with the sequence data that has not been preprocessed.

Images