Image aesthetics evaluation method and device, electronic equipment and medium

Abstract

The invention provides an image aesthetics evaluation method and device, electronic equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the coding of a plurality of attribute features, a plurality of main body neighborhood relation features and overall aesthetics features extracted from an input image, obtaining a plurality of attribute feature codes, a plurality of main body neighborhood relation feature codes and an overall aesthetic feature code of the input image; performing feature fusion on the plurality of attribute feature codes, the plurality of main body neighborhood relation feature codes and the overall aesthetic feature code to obtain an aesthetic feature code of the input image; and performing aesthetic evaluation on the aesthetic feature codes, and determining an aesthetic evaluation result of the input image. By combining a plurality of attribute features, a plurality of main body neighborhood relation features and overall aesthetic features in an input image in an image aesthetic evaluation process, multi-attribute aesthetic structure features are enhanced, and the accuracy of an aesthetic evaluation result is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an image aesthetics evaluation method, device, electronic device and medium. Background Art

[0002] Image aesthetic evaluation is a very important basic task in image analysis. It aims to reflect people's overall aesthetic appreciation of images by analyzing image aesthetic structure, color distribution, image brightness, etc.

[0003] At present, the image aesthetic evaluation methods in related technologies mainly rely on two methods: region of interest (ROI)-based methods and multimodal-based methods. The region of interest-based method extracts the features of specific regions of the image through detection algorithms or attention mechanisms, and inputs them into the aesthetic evaluation neural network in combination with the overall image features to obtain the aesthetic evaluation results. The multimodal-based method integrates multiple aspects of information such as text semantic information and image theme information to perform image aesthetic evaluation. However, most of the image aesthetic evaluation methods in related technologies focus on the visual characteristics of the image itself, and the aesthetic structural characteristics of the image (such as layout, composition and other elements) are relatively insufficient. Summary of the invention

[0004] The present disclosure provides an image aesthetic evaluation method, device, electronic device and medium to solve the problems in the related art. By combining multiple attribute features and multiple subject neighborhood relationship features and overall aesthetic features in an input image during the image aesthetic evaluation process, the aesthetic structural characteristics of multiple attributes are strengthened, thereby improving the accuracy of the aesthetic evaluation results.

[0005] A first aspect embodiment of the present disclosure proposes an image aesthetic assessment method, the method comprising: encoding multiple attribute features, multiple subject neighborhood relationship features, and overall aesthetic features extracted from an input image to obtain multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes of the input image; performing feature fusion on the multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes to obtain aesthetic feature codes of the input image; and performing aesthetic assessment on the aesthetic feature codes to determine an aesthetic assessment result of the input image.

[0006] In some embodiments of the present disclosure, multiple attribute features, multiple subject neighborhood relationship features, and overall aesthetic features extracted from an input image are enhanced and encoded to obtain multiple attribute feature encodings, multiple subject neighborhood relationship feature encodings, and overall aesthetic feature encodings of the input image, including: using a pre-trained image multi-attribute feature encoding module to extract multiple attribute features of the input image, and encoding the multiple attribute features to obtain multiple attribute feature encodings of the input image; using a pre-trained multi-subject neighborhood relationship encoding module to extract multiple subject neighborhood relationship features of the input image, and encoding the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings of the input image; using a pre-trained image feature encoding module to extract the overall aesthetic features of the input image, and encoding the overall aesthetic features to obtain the overall aesthetic feature encoding.

[0007] In some embodiments of the present disclosure, a pre-trained multi-subject neighborhood relationship encoding module is used to extract multiple subject neighborhood relationship features of an input image, and the multiple subject neighborhood relationship features are encoded to obtain multiple subject neighborhood relationship feature encodings of the input image, including: using a pre-trained subject detection model to determine the position coordinates of each subject among multiple subjects in the input image; using the position coordinates of each subject among the multiple subjects to construct an aesthetic structure feature adjacency matrix; using a pre-trained multi-subject neighborhood relationship encoding module to extract multiple subject neighborhood relationship features in the aesthetic structure feature adjacency matrix, and encoding the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings.

[0008] In some embodiments of the present disclosure, constructing an aesthetic structure feature adjacency matrix using the position coordinates of each subject in a plurality of subjects includes: determining the position coordinates of a first subject and the position coordinates of a second subject in a plurality of subjects; determining aesthetic structure features corresponding to the first subject and the second subject using the position coordinates of the first subject and the position coordinates of the second subject respectively, the aesthetic structure features including the width and height of the first subject, the width and height of the second subject, and the center point distance between the first subject and the second subject; determining the edge between the first subject and the second subject based on the width and height of the first subject, the width and height of the second subject, and the center point distance between the first subject and the second subject; and constructing the aesthetic structure feature adjacency matrix using the edge between the first subject and the second subject.

[0009] In some embodiments of the present disclosure, an aesthetic evaluation is performed on the aesthetic feature encoding to determine the aesthetic evaluation result of the input image. Before that, the method includes: using an aesthetic multi-level evaluation module to perform aesthetic evaluation training on training samples to obtain the probabilities of multiple preset score values ​​corresponding to the training samples; determining the loss function corresponding to the training samples based on the variances corresponding to the probabilities of the multiple preset score values; and training the aesthetic multi-level evaluation module based on the loss function to obtain the trained aesthetic multi-level evaluation module.

[0010] In some embodiments of the present disclosure, performing an aesthetic evaluation on the aesthetic feature code and determining the aesthetic evaluation result of the input image includes: using a trained aesthetic multi-level evaluation module to perform an aesthetic evaluation on the aesthetic feature code to obtain the probabilities of multiple preset score values ​​corresponding to the aesthetic feature code; based on the magnitudes of the probabilities of the multiple preset score values, taking a preset score value whose probability of the multiple preset score values ​​is a first probability as an initial score value; and adjusting the initial score value based on the probabilities of the multiple preset score values, and taking the adjusted initial score value as the aesthetic evaluation result.

[0011] A second aspect of the present disclosure provides an image aesthetics evaluation device, the device comprising:

[0012] A feature extraction unit is used to encode multiple attribute features, multiple subject-neighborhood relationship features, and overall aesthetic features extracted from the input image to obtain multiple attribute feature codes, multiple subject-neighborhood relationship feature codes, and overall aesthetic feature codes of the input image;

[0013] A feature fusion unit is used to fuse multiple attribute feature codes, multiple subject neighborhood relationship feature codes and overall aesthetic feature codes to obtain an aesthetic feature code of the input image;

[0014] The aesthetic evaluation unit is used to perform aesthetic evaluation on the aesthetic feature encoding and determine the aesthetic evaluation result of the input image.

[0015] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, executes the method described in the first aspect embodiment of the present disclosure.

[0016] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0017] The fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect embodiment of the present disclosure when executed by a processor.

[0018] In summary, according to the image aesthetic evaluation method proposed in the present disclosure, multiple attribute features, multiple subject neighborhood relationship features, and overall aesthetic features extracted from the input image are encoded to obtain multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes of the input image; multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes are feature fused to obtain aesthetic feature codes of the input image; aesthetic evaluation is performed on the aesthetic feature codes to determine the aesthetic evaluation result of the input image. By combining multiple attribute features, multiple subject neighborhood relationship features, and overall aesthetic features in the input image during the image aesthetic evaluation process, the aesthetic structural characteristics of multiple attributes are strengthened, thereby improving the accuracy of the aesthetic evaluation result.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0021] Figure 1 A flowchart of an image aesthetics evaluation method provided by an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of an image aesthetics evaluation method provided by an embodiment of the present disclosure;

[0023] Figure 3 A flowchart of an image aesthetics evaluation method provided by an embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram of a specific image aesthetics evaluation method provided by an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram of the structure of an image aesthetics evaluation device provided by an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions.

[0028] Image aesthetics evaluation (also known as visual aesthetics evaluation) is a very important basic task in image analysis, which aims to reflect people's overall aesthetics of the image by analyzing the image aesthetic structure, color distribution, image brightness, etc. The difficulty of computer evaluation lies in the subjectivity and uncertainty of aesthetics itself.

[0029] Traditional image aesthetics evaluation methods mainly analyze image color, brightness and other information by counting image pixel values. In terms of composition, by judging the color block information of the image, the distribution of the subject and the background is analyzed to see whether it meets the set rules, such as the golden ratio rule and the rule of thirds.

[0030] There are two main deep learning image aesthetic assessment methods in related technologies. The first is a method based on the region of interest. First, the regional features are obtained using a detection algorithm or an attention mechanism. Then, the regional features are combined with the overall features of the image, input into the aesthetic assessment neural network, and the aesthetic assessment results are output. The second is a multimodal method. This type of method usually requires combining text semantic information, image theme information, etc. for image aesthetic assessment.

[0031] However, the image aesthetics evaluation methods in related technologies rely on artificially set rules, which makes it difficult to guarantee the effectiveness of aesthetic rules and cannot cover all aesthetic features. Most of them focus on the image itself and rarely consider the aesthetic structural characteristics of the image. At the same time, in the model training process in related technologies, since the aesthetic labels themselves have certain subjectivity and uncertainty, treating all data equally may lead to instability in the model prediction results.

[0032] The method proposed in the present disclosure can perform image aesthetic evaluation efficiently and reliably, and can be specifically applied to complex image aesthetic evaluation scenarios in the field of artificial intelligence, such as social media content filtering, social media content filtering, advertising image optimization, etc., which are not limited in the embodiments of the present disclosure.

[0033] The image aesthetics evaluation method provided by the present application is described in detail below in conjunction with the accompanying drawings.

[0034] Figure 1 Flow chart of an image aesthetic evaluation method provided by an embodiment of the present disclosure. Figure 1 As shown, the image aesthetics evaluation method includes steps 101-103.

[0035] Step 101, encoding multiple attribute features, multiple subject-neighborhood relationship features, and overall aesthetic features extracted from an input image to obtain multiple attribute feature codes, multiple subject-neighborhood relationship feature codes, and overall aesthetic feature codes of the input image.

[0036] In an embodiment of the present disclosure, the input image is an image that currently needs to be aesthetically evaluated. The image aesthetic evaluation system of the present disclosure may include an image multi-attribute feature encoding module, a multi-subject neighborhood relationship encoding module, and an image feature encoding module. The image aesthetic evaluation system of the present disclosure can extract multiple attribute features in the input image through the image multi-attribute feature encoding module, and encode the multiple attribute features to obtain multiple attribute feature encodings; extract multiple subject neighborhood relationship features in the input image through the multi-subject neighborhood relationship encoding module, and encode the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings; extract the overall aesthetic features in the input image through the image feature encoding module, and encode the overall aesthetic features to obtain the overall aesthetic feature encoding.

[0037] Among them, attribute features refer to the inherent properties or characteristics of objects or regions in the input image, which can be color, texture, shape, size, position, etc. For example, in a landscape photo, the color of the sky, the texture of the mountains, the reflection of the lake, etc. can all be attribute features. In a person photo, the person's hairstyle, clothing, skin color, etc. are also attribute features.

[0038] Subject neighborhood relationship features refer to the relationship between the input image and the subject (or main object) or multiple subjects in the input image. Subject neighborhood relationships can include spatial relationships (such as position, distance, direction), interactive relationships (such as occlusion, overlap, and proximity), and semantic relationships (such as the subject relevance between the subject and the background). For example, in a photo containing people and scenery, the positional relationship between the person and the mountains or lakes, the visual harmony between the person and the image, etc. are all subject neighborhood relationship features. These features are very important for understanding the content of the image and evaluating the composition and balance of the image.

[0039] Overall aesthetic features refer to features that are abstracted from the entire input image and can reflect the overall aesthetic quality of the image. Overall aesthetic features are usually based on human's general cognition and aesthetic standards of aesthetics, such as image clarity, contrast, color balance, light and shadow effects, emotional expression, etc. Overall aesthetic features not only take into account the local details of the image, but also comprehensively consider the overall layout, theme expression and emotional transmission of the image, which is crucial for the comprehensive evaluation of the aesthetic quality of the image.

[0040] Step 102 , feature fusion is performed on multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes to obtain aesthetic feature codes of the input image.

[0041] In an embodiment of the present disclosure, the image aesthetic evaluation system in the present disclosure may further include a feature aggregation module, which is used to perform feature fusion on the multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes after obtaining the multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes in the input image, so as to strengthen the multi-attribute image structural features and the overall image features, thereby outputting the aesthetic feature code.

[0042] Step 103: perform aesthetic evaluation on the aesthetic feature code to determine an aesthetic evaluation result of the input image.

[0043] In an embodiment of the present disclosure, the image aesthetic evaluation system in the present disclosure may further include an aesthetic multi-level evaluation module. The present disclosure evaluates the aesthetic feature encoding according to a preset score value through the aesthetic multi-level evaluation module to obtain the probability corresponding to the preset score value, and determines the aesthetic evaluation result of the input image according to the probability of the preset score value.

[0044] In summary, according to the image aesthetic evaluation method proposed in the present disclosure, multiple attribute features, multiple subject neighborhood relationship features, and overall aesthetic features extracted from the input image are encoded to obtain multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes of the input image; multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes are feature fused to obtain aesthetic feature codes of the input image; aesthetic evaluation is performed on the aesthetic feature codes to determine the aesthetic evaluation result of the input image. By combining multiple attribute features, multiple subject neighborhood relationship features, and overall aesthetic features in the input image during the image aesthetic evaluation process, the aesthetic structural characteristics of multiple attributes are strengthened, thereby improving the accuracy of the aesthetic evaluation result.

[0045] As a possible implementation, Figure 2 The flowchart of an image aesthetic evaluation method shown in FIG. 1 is a flowchart of an image aesthetic evaluation method. Based on the above embodiment, multiple attribute features, multiple subject-neighborhood relationship features, and overall aesthetic features extracted from an input image are encoded to obtain a specific process of encoding multiple attribute features, multiple subject-neighborhood relationship features, and overall aesthetic features of the input image, including the following steps:

[0046] Step 201 , using a pre-trained image multi-attribute feature encoding module to extract multiple attribute features of an input image, and performing enhanced encoding on the multiple attribute features to obtain multiple attribute feature encodings of the input image.

[0047] In the embodiments of the present disclosure, the image multi-attribute feature encoding module is mainly used to identify the multi-attribute features of the image. Different attribute features in the present disclosure can be defined as scenery, cartoon, art, etc. The image multi-attribute feature encoding module of the present disclosure can adopt the CNN+FC+Softmax deep learning network structure; in order to strengthen the possibility information of the category corresponding to each attribute feature, the present disclosure can remove the Softmax module in the prediction stage (i.e., the image aesthetic evaluation process), and use CNN+FC to output the full type features to avoid the Softmax function from amplifying the gap between the categories corresponding to the attribute features.

[0048] Specifically, in an optional embodiment of the present disclosure, in the training phase, the image multi-attribute feature encoding module of the present disclosure uses a deep learning network structure of CNN (convolutional neural network) + FC (fully connected layer) + Softmax to train a classifier for identifying multiple attribute features of the input image (such as landscape, cartoon, art, etc.). Among them, CNN is responsible for extracting attribute features from the input image, FC maps these attribute features to different category spaces, and Softmax is used to convert the output of the FC layer into a probability distribution, indicating the possibility that the image belongs to each category.

[0049] In the prediction stage, the present disclosure can remove the Softmax module and use the deep learning network structure output of CNN+FC as the full type feature. Because the main function of the Softmax function is to convert the output into a probability distribution, and in order to strengthen the possibility information of the category corresponding to each attribute feature, the present disclosure hopes to obtain the original feature representation (i.e., the output of FC) instead of the probability value after Softmax processing, so as to avoid the problem of amplified category gaps that may be caused by the Softmax function.

[0050] In the prediction stage, the n attribute features output by the image multi-attribute feature encoding module through CNN (convolutional neural network) are: a = {a 0 ,a 1 ,...,a n}; The obtained n attribute features are encoded into n extensions of a transpose through FC (fully connected layer): {A 0 ,A 1 ,...,A n}={a T ,a T ,...,a T}.

[0051] Step 202: extract multiple subject neighborhood relationship features of the input image using a pre-trained multi-subject neighborhood relationship encoding module, and encode the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings of the input image.

[0052] In an embodiment of the present disclosure, the present disclosure can specifically use a pre-trained subject detection model to determine the position coordinates of each subject among multiple subjects in an input image; use the position coordinates of each subject among the multiple subjects to construct an aesthetic structure feature adjacency matrix; use a pre-trained multi-subject neighborhood relationship encoding module to extract multiple subject neighborhood relationship features in the aesthetic structure feature adjacency matrix, and encode the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings.

[0053] Specifically, the multi-subject neighborhood relationship encoding module in the present disclosure is mainly used to extract the position relationship features between the image and each of the multiple subjects and between each of the multiple subjects. The input image can first pass through the subject detection model to output the position coordinates of m subjects.

[0054] For example, the position coordinates of subject i include [x i1 (upper left corner X value), y i1 (upper left corner Y value), x i2 (lower right corner X value),y i2 (lower right corner Y value)]. The position coordinates of the image, including [x m1 (upper left corner X value, origin coordinate X value: 0), y m1 (Upper left corner Y value, origin coordinate Y value: 0), x m2 (lower right corner X value, image width), y m2 (lower right corner Y value, image height)].

[0055] It can be understood that the coordinate structures of the m subjects and the image are consistent and are unified as the vertices of the graph, with a total of m+1 vertices, to construct the aesthetic structure feature adjacency matrix.

[0056] Among them, in the present disclosure, using the position coordinates of each subject in a plurality of subjects to construct an aesthetic structure feature adjacency matrix includes: determining the position coordinates of a first subject and the position coordinates of a second subject in a plurality of subjects; determining the aesthetic structure features corresponding to the first subject and the second subject using the position coordinates of the first subject and the position coordinates of the second subject respectively, the aesthetic structure features including the width and height of the first subject, the width and height of the second subject, and the center point distance between the first subject and the second subject; determining the edge between the first subject and the second subject based on the width and height of the first subject, the width and height of the second subject, and the center point distance between the first subject and the second subject; and constructing the aesthetic structure feature adjacency matrix using the edge between the first subject and the second subject.

[0057] Based on the width and height of the first body, the width and height of the second body, and the center point distance between the first body and the second body, determining the side between the first body and the second body specifically includes: obtaining the ratio of the height to the width of the first body, the ratio of the width of the first body to the width of the second body, and the ratio of the height of the first body to the height of the second body through the width and height of the first body and the width and height of the second body; obtaining the side between the first body and the second body through the center point distance between the first body and the second body, the ratio of the height to the width of the first body, the ratio of the width of the first body to the width of the second body, and the ratio of the height of the first body to the height of the second body.

[0058] Taking vertex i as an example, the position coordinates of vertex i are [x i1 ,y i1 , x i2 ,y i2 ], the position coordinates of vertex i can be calculated by formula 1 to get the center point coordinates [cx i ,cy i ]:

[0059] xx i =(x i1 +x i2 ) / 2,cy i =(y i1 +y i2 ) / 2 Formula 1

[0060] By calculating the center point coordinates of the subject i, the width w of vertex i can be calculated by formula 2 i , high h i :

[0061] w i = abs(x i2 -x i1 ), h i = abs(y i2 -y i1 ) Formula 2

[0062] Take vertices i and j as examples, where the position coordinates of vertex j are [x j1 ,y j1 , x j2 ,y j2 ], the coordinates of the center point of vertex j [cx j ,cy j The calculation method of ] is the same as that of vertex i, and both can be calculated using formula 1. Similarly, the height w of vertex j j and width j It can also be calculated by Formula 2, which will not be repeated here.

[0063] Among them, through the center point coordinates of vertex i and vertex j, the center point distance between vertex i and vertex j can be calculated by formula 3:

[0064] x ij =cx i -cx j ,y ij =cy i -cy j Formula 3

[0065] Through the height and width of vertex i and vertex j obtained above, as well as the coordinates of the center point between vertex i and vertex j, the edge of the vertex in the adjacency matrix (that is, the edge between vertex i and vertex j, where vertex i is the first subject and vertex j is the second subject) can be calculated by formula 4:

[0066]

[0067] Among them, w is the weight parameter to be learned, and abs represents the absolute value.

[0068] It is understandable that in the process of constructing the aesthetic structure feature adjacency matrix, the present disclosure needs to traverse all the subjects in the input image, obtain the edges between every two subjects, and construct the aesthetic structure feature adjacency matrix according to the edges between every two subjects. The present disclosure can construct the aesthetic structure feature adjacency matrix from five dimensions: the x-direction distance between subjects, the y-direction distance between subjects, the width ratio between subjects, the height ratio between subjects, and the width-to-height ratio of the subject itself.

[0069] After obtaining the edges of the vertices of the adjacency matrix, an aesthetic structure feature adjacency matrix can be constructed based on the edges of the vertices of the adjacency matrix, and multiple subject neighborhood relationship features can be extracted through a graph convolutional network (GCN) in a multi-subject neighborhood relationship encoding module, and feature encoding is performed on the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings.

[0070] Step 203: extract the overall aesthetic features of the input image using a pre-trained image feature encoding module, and encode the overall aesthetic features to obtain an overall aesthetic feature code.

[0071] In the embodiment of the present disclosure, the image feature encoding module is used to extract the overall aesthetic features of the image. The image feature encoding module in the present disclosure can use a deep learning convolutional neural network to extract the overall aesthetic features of the image and encode the overall aesthetic features, such as a network structure such as VGG. The overall aesthetic features are not for a specific part or attribute of the input image, but reflect a certain aesthetic characteristic or style of the input image as a whole.

[0072] In summary, the present invention strengthens multiple attribute classification features by removing the Softmax module in the attribute feature extraction stage; at the same time, an aesthetic structure feature adjacency matrix is ​​constructed based on the width, height and position of the subject of the input image, and multiple attribute features, multiple subject neighborhood relationship features and overall aesthetic features in the input image are used to strengthen the multi-attribute image aesthetic structure features for subsequent aesthetic analysis, thereby improving the accuracy of image aesthetic evaluation.

[0073] As a possible implementation, Figure 3 The flowchart of a method for image aesthetic evaluation shown in FIG. 1 is a flowchart of a method for image aesthetic evaluation. Based on the above embodiment, the specific process of performing aesthetic evaluation on the aesthetic feature coding and determining the aesthetic evaluation result of the input image includes the following steps:

[0074] Step 301 , using the trained aesthetic multi-level evaluation module, performs aesthetic evaluation on the aesthetic feature code to obtain the probabilities of multiple preset score values ​​corresponding to the aesthetic feature code.

[0075] In an embodiment of the present disclosure, an aesthetic evaluation is performed on the aesthetic feature encoding to determine the aesthetic evaluation result of the input image. Before that, the method includes: using an aesthetic multi-level evaluation module to perform aesthetic evaluation training on training samples to obtain the probabilities of multiple preset score values ​​corresponding to the training samples; determining the loss function corresponding to the training samples based on the variances corresponding to the probabilities of the multiple preset score values; and training the aesthetic multi-level evaluation module based on the loss function to obtain the trained aesthetic multi-level evaluation module.

[0076] In an optional embodiment of the present disclosure, an aesthetic multi-level evaluation module is used to analyze feature coding and output aesthetic evaluation results. In the training phase of the present disclosure, the training data set (i.e., training samples) adopts AVA Dataset (Aesthetic Visual Analysis Dataset), which is a large-scale data set for image aesthetic quality assessment. This data set collects a large number of images and uses voting to count people's aesthetic scores for the images, thereby providing rich aesthetic evaluation data. The aesthetic score values ​​range from 0 to 9 from low to high. The more votes there are, the more the score value meets the public's aesthetic expectations. Among them, the aesthetic evaluation value is a preset score value in the present disclosure, and the range of the preset score value in the present disclosure takes 0 to 9 as an example.

[0077] Based on the distribution of aesthetic score values ​​of the training samples, the present disclosure proposes a multi-level balanced visual aesthetic evaluation method. The voting distribution variance of the image data (i.e., the variance of the probabilities of multiple preset score values ​​corresponding to the training samples) is used to measure the concentration or dispersion of the scores. The smaller the variance of the voting distribution of the image data, the more dispersed the votes, the more uncertain the aesthetic evaluation of the image, and the need to reduce the impact factor of this type of image; the larger the variance of the voting distribution, the more concentrated the votes, the stronger the guidance of the image to the aesthetic evaluation model in the aesthetic multi-level evaluation module, and the need to strengthen the impact factor of this type of image. The present disclosure can dynamically adjust the loss function according to the variance of the probabilities of multiple preset score values ​​corresponding to the training samples to balance the impact of different preset score value distributions in the training samples on the training of the aesthetic evaluation model in the aesthetic multi-level evaluation module.

[0078] The aesthetic multi-level evaluation module mainly adopts the structure of CNN+FC+Softmax, where CNN is used to extract image features, FC (fully connected layer) is used for feature integration and classification, and Softmax layer outputs the probability distribution of 10 score values. Taking training sample i as an example, the loss function (multi-level balanced loss function loss) of the designed batch is:

[0079] Among them, l i represents the loss function of training sample i, s i represents the voting variance of training sample i (i.e., the probabilities of the 10 preset score values ​​obtained according to the training sample i, and the variances of the 10 preset score values ​​calculated according to the variance calculation formula), s all is the sum of the variance values ​​of the training batch, that is,

[0080] It should be noted that s i The larger the value, the corresponding loss function l i The greater the impact on the loss function of the training batch, the present disclosure can be based on the variance s i Dynamic changes in the training data can dynamically adjust the impact of the model parameters.

[0081] Step 302: Based on the probabilities of the plurality of preset score values, a preset score value whose probability among the plurality of preset score values ​​is a first probability is used as an initial score value.

[0082] In the embodiment of the present disclosure, in the prediction stage, the present disclosure can obtain the probabilities of multiple preset score values ​​through the aesthetic multi-level module, that is, the probabilities {p 0 ,p 1 ,...,p 9}.

[0083] Compare the probability of each of the multiple preset score values, sort the probabilities of the multiple preset score values, and select the preset score value with the largest probability as the initial score value. The first probability is the probability with the largest probability. Assume that the first probability of the initial score value is p k , k∈[0,9].

[0084] Step 303: adjusting the initial score value based on the probabilities of the plurality of preset score values, and using the adjusted initial score value as the aesthetic evaluation result.

[0085] In an embodiment of the present disclosure, after determining the initial score value, the present disclosure may adjust the initial score value k based on the probabilities of multiple preset score values ​​and the differences between the multiple preset score values ​​and the initial score value, so as to obtain an aesthetic evaluation result. The adjustment formula may be specifically shown as Formula 5:

[0086]

[0087] Among them, sore is the final aesthetic evaluation result. The initial score value is based on k, and the deviation between other preset score values ​​j and the initial score value is calculated, j∈[0,9]. When jk<0, the value of score decreases, and when jk>0, the value of score increases. sore combines the probability information of each preset score value to obtain the final aesthetic evaluation result.

[0088] In summary, the image aesthetic evaluation method disclosed in the present invention is based on the perceptual distribution characteristics of aesthetic evaluation, designs a multi-level distribution variance balanced loss function, and dynamically adjusts the loss function by the variance of the probability corresponding to the preset score value of the training sample to adjust the influence factor of the loss function on the model parameters and improve the accuracy of aesthetic evaluation. At the same time, the present invention fine-tunes the aesthetic evaluation results based on the predicted multi-level quantization probability to fit the public visual aesthetics.

[0089] based on Figures 1 to 3 In the embodiment shown, the present disclosure provides a schematic diagram of a specific method for image aesthetics evaluation, such as Figure 4 shown.

[0090] In an embodiment of the present disclosure, the image aesthetic evaluation method in the present disclosure can be executed by an image aesthetic evaluation system, and the image aesthetic evaluation system may include: an image multi-attribute feature encoding module, a multi-subject neighborhood relationship encoding module, an image feature encoding module, a feature aggregation module, and an aesthetic multi-level evaluation module.

[0091] The image multi-attribute feature encoding module is used to extract multiple attribute features of the input image (such as natural landscapes, people, vehicles, etc.), and perform enhanced encoding on the multiple attribute features to obtain multiple attribute feature encodings of the input image. The multi-subject neighborhood relationship encoding module is used to extract multiple subject neighborhood relationship features of the input image (such as trees, people, sky, etc.), and encode the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings of the input image; the image feature encoding module is used to extract the overall aesthetic features of the input image, and encode the overall aesthetic features to obtain the overall aesthetic feature encoding. Among them, the execution steps of the image multi-attribute feature encoding module, the multi-subject neighborhood relationship encoding module and the image feature encoding module are not limited in the embodiments of the present disclosure, and can be executed in sequence according to a preset order, or can be executed simultaneously, depending on the actual situation.

[0092] The feature aggregation module is used to fuse multiple attribute feature codes, multiple subject neighborhood relationship feature codes, and overall aesthetic feature codes to obtain the aesthetic feature code of the input image. The feature aggregation module can use a Transformer network structure to strengthen the multi-attribute image structure features and the overall image features, and output the overall aesthetic feature code.

[0093] The aesthetic multi-level evaluation module adopts the structure of CNN+FC+Softmax to perform aesthetic evaluation on the aesthetic feature encoding, obtain the probabilities of multiple preset score values ​​corresponding to the aesthetic feature encoding, and use the probabilities of multiple preset score values ​​to fine-tune the aesthetic evaluation result to obtain the final aesthetic evaluation result. Among them, during the training stage, the aesthetic accumulation evaluation module can determine the loss function (multi-level balanced Loss) according to the variance of the probability of the preset score value of the training sample to improve the accuracy of the aesthetic evaluation.

[0094] In summary, the present invention strengthens multiple attribute features, outputs multiple attribute feature codes, and integrates multiple attribute features, multiple subject neighborhood relationship features, and overall aesthetic features to strengthen the multi-attribute image aesthetic structure features, output aesthetic feature codes, and then conduct aesthetic analysis. In the analysis stage, based on the perceptual distribution characteristics of aesthetic evaluation points, a multi-level distribution variance equalization loss function is designed to dynamically adjust the influence factors of different training data on model parameters to improve the accuracy of aesthetic evaluation; at the same time, through the predicted multi-level quantization probability, the aesthetic evaluation results are fine-tuned to be closer to the public's visual aesthetics.

[0095] Corresponding to the methods provided in the above-mentioned embodiments, the present disclosure also provides an image aesthetics evaluation device. Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation method of the method is also applicable to the device provided in this embodiment and will not be described in detail in this embodiment.

[0096] Figure 5 FIG. 5 is a schematic diagram of the structure of an image aesthetics evaluation device 500 provided in an embodiment of the present disclosure. Figure 5 As shown, the image aesthetics evaluation device comprises:

[0097] The feature extraction unit 510 is used to encode the multiple attribute features, multiple subject-neighborhood relationship features, and overall aesthetic features extracted from the input image to obtain multiple attribute feature codes, multiple subject-neighborhood relationship feature codes, and overall aesthetic feature codes of the input image;

[0098] A feature fusion unit 520 is used to fuse multiple attribute feature codes, multiple subject neighborhood relationship feature codes and overall aesthetic feature codes to obtain an aesthetic feature code of the input image;

[0099] The aesthetic evaluation unit 530 is used to perform aesthetic evaluation on the aesthetic feature coding to determine the aesthetic evaluation result of the input image.

[0100] In some embodiments of the present disclosure, the feature extraction unit 510 is used to: use a pre-trained image multi-attribute feature encoding module to extract multiple attribute features of an input image, and perform enhanced encoding on the multiple attribute features to obtain multiple attribute feature encodings of the input image; use a pre-trained multi-subject neighborhood relationship encoding module to extract multiple subject neighborhood relationship features of the input image, and encode the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings of the input image; use a pre-trained image feature encoding module to extract the overall aesthetic features of the input image, and encode the overall aesthetic features to obtain the overall aesthetic feature encoding.

[0101] In some embodiments of the present disclosure, the feature extraction unit 510 is used to: determine the position coordinates of each subject among multiple subjects in the input image using a pre-trained subject detection model; construct an aesthetic structure feature adjacency matrix using the position coordinates of each subject among the multiple subjects; extract multiple subject neighborhood relationship features in the aesthetic structure feature adjacency matrix using a pre-trained multi-subject neighborhood relationship encoding module, and encode the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings.

[0102] In some embodiments of the present disclosure, the feature extraction unit 510 is used to: determine the position coordinates of a first subject and the position coordinates of a second subject among multiple subjects; determine the aesthetic structural features corresponding to the first subject and the second subject using the position coordinates of the first subject and the position coordinates of the second subject respectively, the aesthetic structural features including the width and height of the first subject, the width and height of the second subject, and the center point distance between the first subject and the second subject; determine the edge between the first subject and the second subject based on the width and height of the first subject, the width and height of the second subject, and the center point distance between the first subject and the second subject; and construct an aesthetic structural feature adjacency matrix using the edge between the first subject and the second subject.

[0103] In some embodiments of the present disclosure, the aesthetic evaluation unit 530 is used to: before performing an aesthetic evaluation on the aesthetic feature encoding to determine the aesthetic evaluation result of the input image, use the aesthetic multi-level evaluation module to perform aesthetic evaluation training on the training samples to obtain the probabilities of multiple preset score values ​​corresponding to the training samples; determine the loss function corresponding to the training samples based on the variance corresponding to the probabilities of the multiple preset score values; and train the aesthetic multi-level evaluation module based on the loss function to obtain the trained aesthetic multi-level evaluation module.

[0104] In some embodiments of the present disclosure, the aesthetic evaluation unit 530 is used to: use the trained aesthetic multi-level evaluation module to perform aesthetic evaluation on the aesthetic feature code to obtain the probabilities of multiple preset score values ​​corresponding to the aesthetic feature code; based on the sizes of the probabilities of the multiple preset score values, use the preset score value whose probability of the multiple preset score values ​​is a first probability as the initial score value; based on the probabilities of the multiple preset score values, adjust the initial score value, and use the adjusted initial score value as the aesthetic evaluation result.

[0105] In summary, through the image aesthetic evaluation device, multiple attribute features, multiple subject neighborhood relationship features and overall aesthetic features extracted from the input image are encoded to obtain multiple attribute feature codes, multiple subject neighborhood relationship feature codes and overall aesthetic feature codes of the input image; multiple attribute feature codes, multiple subject neighborhood relationship feature codes and overall aesthetic feature codes are feature fused to obtain aesthetic feature codes of the input image; aesthetic feature codes are aesthetically evaluated to determine the aesthetic evaluation result of the input image. By combining multiple attribute features, multiple subject neighborhood relationship features and overall aesthetic features in the input image during the image aesthetic evaluation process, the aesthetic structural characteristics of multiple attributes are strengthened, and the accuracy of the aesthetic evaluation results is improved.

[0106] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0107] Figure 6 is a block diagram of an electronic device 600 for implementing the above-mentioned image aesthetic evaluation method according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0108] Reference Figure 6 The electronic device 600 may include a communication interface 601, which can interact with other devices; a processor 602, which is connected to the communication interface 601 to interact with other devices and is used to execute the method provided by one or more of the above technical solutions when running a computer program; and a memory 603, on which the computer program is stored. Specifically, the specific processing process of the processor 602 can refer to the image aesthetics evaluation method described in the above embodiment of the present disclosure.

[0109] Of course, in actual application, the various components in the electronic device 600 are coupled together through the bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus system 604 .

[0110] The memory 603 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 600. Examples of such data include: any computer program used to operate on the electronic device 600.

[0111] The method disclosed in the above embodiment of the present application can be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 602. The above processor 602 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 602 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 603, and the processor 602 reads the information in the memory 603 and completes the steps of the above method in combination with its hardware.

[0112] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0113] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the image aesthetics evaluation method described in the above embodiments of the present disclosure.

[0114] The embodiments of the present disclosure further provide a computer program product, including a computer program, which executes the image aesthetics evaluation method described in the above embodiments of the present disclosure when a processor is used to execute the computer program.

[0115] An embodiment of the present disclosure also proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the image aesthetics assessment method described in the above embodiments of the present disclosure.

[0116] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0117] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0118] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0119] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0120] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0121] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0122] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0123] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for image aesthetics assessment, characterized in that: The method comprises: Encoding a plurality of attribute features, a plurality of subject-neighborhood relationship features, and an overall aesthetic feature extracted from an input image to obtain a plurality of attribute feature codes, a plurality of subject-neighborhood relationship feature codes, and an overall aesthetic feature code of the input image; Performing feature fusion on the multiple attribute feature codes, the multiple subject-neighborhood relationship feature codes, and the overall aesthetic feature code to obtain an aesthetic feature code of the input image; An aesthetic evaluation is performed on the aesthetic feature code to determine an aesthetic evaluation result of the input image.

2. The method according to claim 1, characterized in that: The step of encoding the multiple attribute features, the multiple subject-neighborhood relationship features, and the overall aesthetic features extracted from the input image to obtain the multiple attribute feature codes, the multiple subject-neighborhood relationship feature codes, and the overall aesthetic feature codes of the input image comprises: Extracting multiple attribute features of the input image using a pre-trained image multi-attribute feature encoding module, and performing enhanced encoding on the multiple attribute features to obtain multiple attribute feature encodings of the input image; Extracting multiple subject neighborhood relationship features of the input image using a pre-trained multi-subject neighborhood relationship encoding module, and encoding the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings of the input image; The overall aesthetic features of the input image are extracted using a pre-trained image feature encoding module, and the overall aesthetic features are encoded to obtain the overall aesthetic feature code.

3. The method according to claim 2, characterized in that The extracting of multiple subject neighborhood relationship features of the input image by using a pre-trained multi-subject neighborhood relationship encoding module, and encoding the multiple subject neighborhood relationship features to obtain multiple subject neighborhood relationship feature encodings of the input image includes: Determine the position coordinates of each of the plurality of subjects in the input image using a pre-trained subject detection model; Using the position coordinates of each subject in the plurality of subjects, constructing an aesthetic structure feature adjacency matrix; The pre-trained multi-agent neighborhood relationship encoding module is used to extract multiple subject neighborhood relationship features in the aesthetic structure feature adjacency matrix, and the multiple subject neighborhood relationship features are encoded to obtain the multiple subject neighborhood relationship feature encodings.

4. The method according to claim 3, characterized in that The step of constructing an aesthetic structural feature adjacency matrix by using the position coordinates of each subject in the plurality of subjects comprises: determining position coordinates of a first subject and a second subject among the plurality of subjects; Determine aesthetic structural features corresponding to the first body and the second body using the position coordinates of the first body and the position coordinates of the second body respectively, wherein the aesthetic structural features include the width and height of the first body, the width and height of the second body, and the distance between the center points of the first body and the second body; Determine a side between the first body and the second body based on the width and height of the first body, the width and height of the second body, and the distance between the center points of the first body and the second body; The aesthetic structural feature adjacency matrix is ​​constructed using the edges between the first body and the second body.

5. The method according to claim 1, characterized in that Before performing aesthetic evaluation on the aesthetic feature encoding to determine the aesthetic evaluation result of the input image, the method comprises: Using the aesthetic multi-level evaluation module, the training samples are trained in aesthetic evaluation to obtain the probabilities of multiple preset score values ​​corresponding to the training samples; Determining a loss function corresponding to the training sample according to variances corresponding to the probabilities of the plurality of preset score values; Based on the loss function, the aesthetic multi-level evaluation module is trained to obtain a trained aesthetic multi-level evaluation module.

6. The method according to claim 5, characterized in that The step of performing aesthetic evaluation on the aesthetic feature code to determine the aesthetic evaluation result of the input image comprises: Using the trained aesthetic multi-level evaluation module, perform aesthetic evaluation on the aesthetic feature code to obtain the probabilities of multiple preset score values ​​corresponding to the aesthetic feature code; Based on the probabilities of the plurality of preset score values, taking a preset score value whose probability among the plurality of preset score values ​​is a first probability as an initial score value; Based on the probabilities of the plurality of preset score values, the initial score value is adjusted, and the adjusted initial score value is used as the aesthetic evaluation result.

7. An image aesthetics evaluation device, characterized in that: The device comprises: A feature extraction unit, used for encoding a plurality of attribute features, a plurality of subject-neighborhood relationship features, and an overall aesthetic feature extracted from an input image, to obtain a plurality of attribute feature codes, a plurality of subject-neighborhood relationship feature codes, and an overall aesthetic feature code of the input image; A feature fusion unit, configured to perform feature fusion on the plurality of attribute feature codes, the plurality of subject-neighborhood relationship feature codes and the overall aesthetic feature code to obtain an aesthetic feature code of the input image; The aesthetic evaluation unit is used to perform aesthetic evaluation on the aesthetic feature code to determine an aesthetic evaluation result of the input image.

8. An electronic device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the method described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that A computer program is included which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images