Image aesthetics quality evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides an image aesthetics quality evaluation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring text feature information and image feature information; determining the global similarity between the global image feature and the target text feature and the local similarity between the local image feature and the target text feature according to the target text feature and the image feature information in the text feature information; according to the global similarity and the local similarity, determining the average similarity of the target text feature information and the image feature information; determining a matching degree weight of the target text feature in the text feature information and a weight value of the matching degree weight according to an average similarity between the target text feature and the image feature information; and obtaining an aesthetic quality evaluation result of the to-be-evaluated image according to the matching degree weight of the target text feature in the text feature information and a weight value of the matching degree weight. According to the method, the accuracy of picture aesthetics quality evaluation is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image aesthetic quality assessment method, device, electronic device and storage medium. Background Art

[0002] Aesthetic quality is a measure of visually perceived beauty, measuring the visual appeal of an image to the human eye. Research in computational aesthetics explores how to use computable techniques to predict human emotional responses to visual stimuli so that computers can simulate the human aesthetic process and thus automatically predict the aesthetic quality of an image through computable methods.

[0003] Traditional aesthetic evaluation methods usually adopt methods based on professional knowledge or general feature design, which evaluate images by extracting fixed feature sets and following certain rules. In the field of deep learning, the basic method of image aesthetic evaluation is to input a single image, and after a series of processing by the deep learning network, output the image's aesthetic evaluation score. By guiding the training of the deep learning network, it is equipped with powerful image aesthetic feature perception and extraction capabilities, so that the aesthetic quality can be perceived during the image encoding and decoding process, and image aesthetic evaluation can be achieved.

[0004] However, existing image aesthetic quality assessment methods suffer from poor evaluation results. Summary of the invention

[0005] The present application provides an image aesthetic quality assessment method, device, electronic device and storage medium, which are used to solve the problem of poor effect when assessing the aesthetic quality of keyword-generated images.

[0006] In a first aspect, the present application provides a method for evaluating image aesthetic quality, comprising: Acquire text feature information and image feature information, wherein the text feature information is a text feature obtained by extracting features from the prompt word text according to preset evaluation keywords, the evaluation keywords represent the degree of matching between the prompt word text and the image to be evaluated, and the image feature information includes global image features of the image to be evaluated and local image features of a target local image in the image to be evaluated; Determine the global similarity between the global image feature and the target text feature, and the local similarity between the local image feature and the target text feature according to the target text feature and the image feature information in the text feature information; According to the global similarity and the local similarity, the average similarity between the target text feature and the image feature information is determined; According to the average similarity between the target text feature and the image feature information, determining the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight, wherein the weight of the matching degree weight is determined according to a preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information; According to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight, the aesthetic quality evaluation result of the image to be evaluated is obtained.

[0007] In the embodiment of the present application, obtaining text feature information and image feature information includes: Obtain prompt word text and image feature information; Obtaining a prompt word evaluation text according to the prompt word text and preset evaluation keywords, wherein the number of evaluation keywords is not less than 2; The prompt word evaluation text is input into the text encoder of the visual language model to obtain text feature information.

[0008] In the embodiment of the present application, obtaining text feature information and image feature information includes: Obtain text feature information and the image to be evaluated; According to the preset size requirement, the image to be evaluated is scaled to obtain a global image of the image to be evaluated; According to the preset size requirements, the image to be evaluated is intercepted and processed to obtain the target partial image; The global image and the target local image of the image to be evaluated are respectively input into the image encoder of the visual language model to obtain the global image features and the local image features; Image feature information is obtained according to global image features and local image features.

[0009] In the embodiment of the present application, the average similarity between the target text feature and the image feature information is determined based on the global similarity and the local similarity, including: Perform weighted averaging on the local similarities to obtain the initial average similarity; According to the initial average similarity and the global similarity, the average similarity of the target text feature and image feature information is obtained.

[0010] In the embodiment of the present application, the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight are determined according to the average similarity between the target text feature and the image feature information, including: Perform weighted summation on the average similarity of target text features and image feature information to obtain the global average similarity; According to the global average similarity and the average similarity, the matching degree weight of the target text feature in the text feature information is determined; According to the matching degree weight of the target text feature in the text feature information and the preset weight assignment rule, the weight of the matching degree weight is obtained.

[0011] In the embodiment of the present application, according to the matching degree weight of the target text feature in the text feature information and the preset weight assignment rule, the weight of the matching degree weight is obtained, including: Sorting the matching degree weights of the target text features in the text feature information to obtain a matching degree weight sequence; According to the weight assignment rule, the matching degree weights in the matching degree weight sequence are assigned to obtain the weights of the matching degree weights, wherein the earlier the position of the matching degree weight in the matching degree weight sequence is, the greater the weight of the matching degree weight is.

[0012] In the embodiment of the present application, the aesthetic quality evaluation result of the image to be evaluated is obtained according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight, including: According to the matching degree weight and the weight of the matching degree weight, a matching degree weight value is obtained; Perform weighted summation on the matching degree weight values ​​to obtain the matching degree weight value; According to the preset weight distribution range, the matching degree weight is adjusted to obtain the aesthetic quality evaluation result of the image to be evaluated.

[0013] In a second aspect, the present application provides an image aesthetic quality assessment device, comprising: An acquisition module is used to acquire text feature information and image feature information. The text feature information is a text feature obtained by extracting features from a prompt word text according to preset evaluation keywords. The evaluation keywords represent the degree of match between the prompt word text and the image to be evaluated. The image feature information includes global image features of the image to be evaluated and local image features of a target local image in the image to be evaluated. A first determination module is used to determine the global similarity between the global image feature and the target text feature, and the local similarity between the local image feature and the target text feature according to the target text feature and the image feature information in the text feature information; The second determination module is used to determine the average similarity between the target text feature and the image feature information according to the global similarity and the local similarity; A third determination module is used to determine the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight according to the average similarity between the target text feature and the image feature information, wherein the weight of the matching degree weight is determined according to a preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information; The module is used to obtain the aesthetic quality evaluation result of the image to be evaluated according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; Memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the image aesthetic quality assessment method of the embodiment of the present application.

[0015] In a fourth aspect, a computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the image aesthetic quality assessment method of an embodiment of the present application.

[0016] The image aesthetic quality assessment method, device, electronic device and storage medium provided by the present application generate an image to be assessed through a prompt word text, then generate a global image and a local image based on the image to be assessed, obtain global image features and local image features, respectively calculate similarity between the global image features and the local image features and target text features obtained according to the prompt word text and evaluation keywords, average the obtained similarities to obtain the average similarity of the target text features, determine the matching degree weight of the target text features in the text feature information based on the average similarity between the target text features and the image feature information, and then determine the means of the aesthetic quality assessment result of the image to be assessed based on a preset weight assignment rule and the size of the matching degree weight of the target text features in the text feature information, thereby improving the effect of the aesthetic quality assessment of the image generated according to the keywords. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] Figure 1 A schematic diagram of the process of the image aesthetic quality assessment method provided in the embodiment of the present application; Figure 2 A schematic diagram of a flow chart of another method for evaluating image aesthetic quality provided in an embodiment of the present application; Figure 3A schematic diagram of the structure of an image aesthetic quality assessment device provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0019] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0020] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0021] Glossary: AI generative models: Deep learning technology can be used to learn and train through massive data sets, and models that can recognize and understand the meaning of keywords and generate images that meet the requirements. Users can enter a set of keywords to describe a scene, object, etc., and the model will generate corresponding images based on these keywords, such as GauGAN, GauGAN2, BigGAN, and DF-GAN.

[0022] Visual Language Model: A technology that combines image and natural language processing to understand and interpret the association between images and text, converting image and text data into representations in the same space, allowing cross-modal interaction and joint learning. This model can be applied to various tasks such as visual question answering, image description generation, converting text and images into vectors, etc.

[0023] In the existing technology, traditional image aesthetics evaluation methods are limited to image content and mostly rely on human subjective evaluation or expert opinions. The evaluation process is relatively cumbersome and lacks consideration of text information. It ignores the correlation between AI (Artificial Intelligence) generated images and the text prompt words used. Therefore, it is limited in effect when evaluating the aesthetics of AI-generated images.

[0024] The embodiment of the present application can extract text features of multiple matching degrees for the prompt word text based on preset evaluation keywords. In order to comprehensively consider the local features and complete semantic information of the image to be evaluated, the text feature information is respectively calculated with the global image feature information and the local image feature information. Then, the similarity between the image to be evaluated and the text features of each matching degree and the matching degree weight of each text feature in all text features are obtained, so as to obtain the aesthetic quality evaluation result according to the preset weight assignment rules and the matching degree weight. As a result, the image aesthetic evaluation is performed according to the similarity between the image and the prompt word of the generated image, thereby improving the evaluation accuracy.

[0025] The embodiments of the present application provide a method, device, electronic device and storage medium for evaluating image aesthetic quality.

[0026] Figure 1 The flowchart of the method for evaluating the aesthetic quality of an image provided in the embodiment of the present application is shown in FIG. The execution subject of the method may be a server or other servers, and this embodiment is not particularly limited here. Figure 1 As shown, the method may include: S101, obtaining text feature information and image feature information, wherein the text feature information is text features obtained by extracting features from a prompt word text according to preset evaluation keywords, the evaluation keywords represent the degree of matching between the prompt word text and the image to be evaluated, and the image feature information includes global image features of the image to be evaluated and local image features of a target local image in the image to be evaluated.

[0027] Among them, text feature information may refer to multiple iconic data extracted from text data that can describe the content or nature of the text. In an embodiment of the present application, text feature information may refer to vector representations processed by a text encoder, which can capture the semantic and contextual information in the text. One text feature information is for one evaluation keyword, and the text feature information contains feature information of multiple evaluation keywords with different matching degrees.

[0028] The preset evaluation keywords may refer to words that describe different degrees of matching between the image and the prompt word text, and may be words of different degrees from light to heavy, such as "level one", "level two", "level three", etc.

[0029] The prompt word text may refer to text input by a user for generating an image to be evaluated, such as “sunset”, “ocean”, “cat”, etc.

[0030] Image feature information can refer to the numerical representation extracted from the image that can describe the image content, structure, texture, style and other attributes. These feature information are usually extracted by pre-trained image encoders (such as ResNet, VGG, EfficientNet, etc.) and converted into points in a high-dimensional vector space.

[0031] The global image feature of the image to be evaluated may refer to a sub-image that can represent the complete semantic information of the image, and the local image feature of the image to be evaluated may refer to a sub-image that can represent the local features and basic quality of the image.

[0032] The image to be evaluated may be an image generated by an AI generation model based on a prompt word text.

[0033] Among them, in the embodiment of the present application, the method of obtaining text feature information and image feature information may include: Obtain prompt word text and image feature information; Obtaining a prompt word evaluation text according to the prompt word text and preset evaluation keywords, wherein the number of evaluation keywords is not less than 2; The prompt word evaluation text is input into the text encoder of the visual language model to obtain text feature information.

[0034] Among them, the prompt word evaluation text can refer to a text composed of a prompt word text and evaluation keywords. The expression of the prompt word evaluation text can be a template such as "This is an image of {evaluation keywords} matching {prompt word text}". By embedding the same prompt word text and different evaluation keywords into the text template, multiple text features can be extracted. For example, when the evaluation keywords are "excellent", "average" and "very poor", and the prompt word text is "sunset", the prompt word evaluation text includes "excellent sunset", "average sunset" and "very poor sunset".

[0035] The prompt word evaluation text is input into the text encoder of the visual language model. The text encoder will output multiple text feature vectors. Each vector corresponds to a sentence describing different degrees of relevance. Multiple text feature vectors describing different degrees of relevance between the image and the prompt word can be obtained.

[0036] Among them, in the embodiment of the present application, the method of obtaining text feature information and image feature information may include: Obtain text feature information and the image to be evaluated; According to the preset size requirement, the image to be evaluated is scaled to obtain a global image of the image to be evaluated; According to the preset size requirements, the image to be evaluated is intercepted and processed to obtain the target partial image; The global image and the target local image of the image to be evaluated are respectively input into the image encoder of the visual language model to obtain the global image features and the local image features; Image feature information is obtained according to global image features and local image features.

[0037] The preset size requirement may be set according to the size requirement of the image encoder.

[0038] According to the preset size requirement, the image to be evaluated is intercepted and processed, and the method for obtaining the target local image may include using a sliding window method to use the image obtained by each movement as a local image, and then randomly selecting (or selecting at intervals) a fixed number of images as the target local images, or automatically determining the position of the target local image to be selected according to an algorithm, and then performing targeted interception.

[0039] The global image and the target local image of the image to be evaluated are respectively input into the image encoder of the visual language model. The obtained global image features can represent the information of all regions in the global image and can be used to describe the content, style and layout of the entire image. The local image features can represent the local attributes and information of the region.

[0040] S102: Determine the global similarity between the global image feature and the target text feature, and the local similarity between the local image feature and the target text feature, according to the target text feature and the image feature information in the text feature information.

[0041] The target text feature may refer to one of a plurality of text feature information describing different degrees of relevance between an image and a prompt word. For example, when the evaluation keywords are "excellent", "average" and "very poor", and the prompt word text is "book", the target text feature may be feature information of "excellent book". In the embodiment of the present application, by continuously repeating step S102, the global similarity and local similarity of all text features and image feature information in the text feature information may be obtained.

[0042] Similarity may refer to the degree of match between image features and text features, global similarity may refer to the degree of match between features of complete semantic information of an image and text features, local similarity may refer to the degree of match between local features of an image and text features, and in some embodiments, methods for determining similarity may include Euclidean distance, Manhattan distance, and Pearson correlation coefficient, etc. In an embodiment of the present application, global similarity may be obtained by determining the cosine similarity between global image features and target text features, and local similarity may be obtained by determining the cosine similarity between local image features and target text features, wherein cosine similarity may refer to the similarity between two vectors evaluated by measuring the cosine value of the angle between them.

[0043] S103: Determine the average similarity between the target text feature and the image feature information according to the global similarity and the local similarity.

[0044] The average similarity may refer to the similarity of the target text features obtained by calculating the average of the global similarity and the local similarity. By evaluating the calculation of the similarity, information in two dimensions, namely, the original quality of the image and the complete semantics, may be comprehensively considered.

[0045] In the embodiment of the present application, the method for determining the average similarity between the target text feature and the image feature information according to the global similarity and the local similarity may include: Perform weighted averaging on the local similarities to obtain the initial average similarity; According to the initial average similarity and the global similarity, the average similarity of the target text feature and image feature information is obtained.

[0046] The initial average similarity may refer to the average value of all local similarities of the target text feature. For example, if the local similarities of the target text feature include three values ​​of 23, 36, and 67, the initial average similarity is calculated as (23+36+67) / 3, and the result is 42.

[0047] The average similarity can be obtained based on the initial average similarity and the global similarity. For example, when the initial average similarity is 42 and the global similarity is 64, the average similarity is 53.

[0048] In some embodiments, the method for determining local similarity may further include: According to the number of local images and the preset cleaning rules, the similarity cleaning threshold of the local image is determined, and then the initial local similarity between the target text feature and the local image is calculated. The initial local similarity is screened according to the cleaning threshold to obtain the local similarity. For example, when the number of remaining images is 64, due to the large number of local images, the difference in the relevance of each image to the prompt word text is large, so the similarity cleaning threshold is set to a higher 60%. When the similarity of the local image is lower than 60%, the image is deleted. When the similarity of the local image is higher than 60%, the image is retained. The unreliable data with large deviation values ​​are eliminated, leaving reasonable data with higher credibility, thereby improving the accuracy of image aesthetic evaluation.

[0049] S104. Determine the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight according to the average similarity between the target text feature and the image feature information, wherein the weight of the matching degree weight is determined according to a preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information.

[0050] The matching degree weight may refer to the proportion of the average similarity of the target text feature to the average similarity of all text features.

[0051] The weight of the matching degree weight may refer to different weights assigned according to different rankings of the proportion of the matching degree weight.

[0052] The method for determining the weight of the matching degree weight according to the preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information may include sorting the weights from large to small, from small to large, etc., and then assigning values ​​according to the weight assignment rule, assigning the largest value to the largest weight, or assigning the largest value to the smallest weight. The value range of the weight may also be determined. For example, when the weight is between 0 and 0.5, the weight is 1, and when the weight is between 0.5 and 1, the weight is 1.5.

[0053] Among them, in the embodiment of the present application, according to the average similarity between the target text feature and the image feature information, the method for determining the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight may include: Perform weighted summation on the average similarity of target text features and image feature information to obtain the global average similarity; According to the global average similarity and the average similarity, the matching degree weight of the target text feature in the text feature information is determined; According to the matching degree weight of the target text feature in the text feature information and the preset weight assignment rule, the weight of the matching degree weight is obtained.

[0054] Among them, the global average similarity can refer to the weighted sum of the average similarities of all text features and image feature information in the text feature information. The global average similarity can be obtained by weighted summing up all the average similarities. For example, when the average similarity of the first text feature and image feature information is 0.3, the average similarity of the second text feature and image feature information is 0.2, and the average similarity of the third text feature and image feature information is 0.7, the global average similarity is calculated as 0.3+0.2+0.7, and the result is 1.2.

[0055] The method for determining the matching degree weight of the target text feature in the text feature information based on the global average similarity and the average similarity may include calculating the proportion of the similarity of each target text feature in the global average similarity, and the proportion is the matching degree weight of each target text feature. For example, when the average similarity between the first text feature and the image feature information is 0.3, the average similarity between the second text feature and the image feature information is 0.2, the average similarity between the third text feature and the image feature information is 0.7, and the global average similarity is 1.2, the matching degree weight of the average similarity between the second text feature and the image feature information in the text feature information is 0.17.

[0056] Among them, in the embodiment of the present application, according to the matching degree weight of the target text feature in the text feature information and the preset weight assignment rule, the method for obtaining the weight of the matching degree weight may include: Sorting the matching degree weights of the target text features in the text feature information to obtain a matching degree weight sequence; According to the weight assignment rule, the matching degree weights in the matching degree weight sequence are assigned to obtain the weights of the matching degree weights, wherein the earlier the position of the matching degree weight in the matching degree weight sequence is, the greater the weight of the matching degree weight is.

[0057] Among them, the matching degree weight sequence can refer to the order of the matching degree weights among all weights. For example, when the matching degree weights are 0.2, 0.3, and 0.5, the matching degree weight sequence of weight 0.2 is 3, the matching degree weight sequence of weight 0.3 is 2, and the matching degree weight sequence of weight 0.5 is 1.

[0058] The weight assignment rule can be determined based on the evaluation keywords. For example, when there are five evaluation keywords, the weights used for assignment can be five. When there are five evaluation keywords but the evaluation keyword levels are three, the weights used for assignment can be three. When the first-level evaluation keywords are "excellent" and "very good", the second-level evaluation keywords are "average", and the third-level evaluation keywords are "very poor" and "not good", the weights are set to 5, 3, and 1 respectively.

[0059] The weight assignment rule can also be determined according to the order of the weights of each matching degree, for example, when the weight sequence is 1, the weight is 5, when the weight sequence is 2, the weight is 3, and when the weight sequence is 3, the weight is 1. The above two weight assignment rules can also be multiplied to obtain the weight of the matching degree weight.

[0060] S105 , obtaining an aesthetic quality evaluation result of the image to be evaluated according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight.

[0061] Among them, the aesthetic quality assessment result of the image to be evaluated can refer to the aesthetic evaluation obtained based on the similarity between the prompt word text and the image generated by the prompt word text. The higher the evaluation, the better the image quality. The aesthetic quality assessment result will be notified to the user through text messages, emails, etc.

[0062] Among them, in the embodiment of the present application, according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight, the method for obtaining the aesthetic quality evaluation result of the image to be evaluated may include: According to the matching degree weight and the weight of the matching degree weight, a matching degree weight value is obtained; Perform weighted summation on the matching degree weight values ​​to obtain the matching degree weight value; According to the preset weight distribution range, the matching degree weight is adjusted to obtain the aesthetic quality evaluation result of the image to be evaluated.

[0063] The matching degree weight value may refer to a result obtained by combining the weight and the weight of the weight. For example, when the matching degree weight is 0.3 and the weight of the matching degree weight is 5, the matching degree weight value is 1.5.

[0064] The matching degree weight may refer to the result obtained by weighted summing the matching degree weight values ​​of all text features. For example, when the matching degree weight value of the first text feature is 2 and the matching degree weight value of the second text feature is 4, the matching degree weight value is 6.

[0065] In order to ensure that all features are on the same scale in terms of values, adjusting the matching degree weight may refer to standardizing the matching degree weight according to a preset weight distribution range. In an embodiment of the present application, the weight distribution range may be [0, +∞]. The maximum and minimum values ​​of the matching degree weight are first calculated, and the matching degree weight is scaled according to the maximum and minimum values ​​to ensure that the matching degree weight is standardized within a numerical range starting from 0. For example, when the distribution of the matching degree weight is [1,5], the matching degree weight is calculated as (matching degree weight-1)*5 / 4, where 1 is the minimum value of the matching degree weight, 5 is the maximum value of the matching degree weight, and 4 is the result of the maximum value-minimum value of the matching degree weight, thereby standardizing the matching degree weight.

[0066] The image aesthetic quality assessment method provided in the embodiment of the present application can extract text features of multiple matching degrees for the prompt word text based on preset evaluation keywords. In order to comprehensively consider the local features and complete semantic information of the image to be evaluated, the text feature information is respectively calculated with the global image feature information and the local image feature information. Then, the similarity between the image to be evaluated and the text features of each matching degree and the matching degree weight of each text feature in all text features are obtained, so as to obtain the aesthetic quality assessment result according to the preset weight assignment rules and the matching degree weight. As a result, the image aesthetic assessment is performed according to the similarity between the image and the prompt word of the generated image, thereby improving the accuracy of the assessment.

[0067] Figure 2 A flowchart of another method for evaluating image aesthetic quality provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes: S201, obtaining an image to be evaluated and a prompt word text used to generate the image to be evaluated.

[0068] Among them, the image to be evaluated can be an image generated by an AI generation model based on the prompt word text.

[0069] S202: Obtain evaluation text information and evaluation text information features according to the prompt word text and preset evaluation words, wherein the preset evaluation words include "very good", "average" and "very poor".

[0070] The evaluation text information may refer to text that imposes restrictions of different similarity levels on prompt words.

[0071] S203 . Obtain a global image, a local image, global image features, and local image features according to the image to be evaluated.

[0072] The global image may refer to an image including all regions of the image, and the global image feature may refer to vector information representing all regions in the global image.

[0073] A local image may refer to an image of a partial area in an image, and a local image feature may refer to vector information of the image of the partial area.

[0074] S204 , obtaining global similarity and local similarity according to global image features, local image features and evaluation text information features.

[0075] The global similarity includes the global similarity of "very good 'prompt words'", the global similarity of "average 'prompt words'" and the global similarity of "very bad 'prompt words'", and the local similarity includes the local similarity of "very good prompt words", the local similarity of "average prompt words" and the local similarity of "very bad prompt words".

[0076] S205 . Obtain the average similarity of each evaluation text information feature according to the global similarity and the local similarity.

[0077] Among them, the method for obtaining the average similarity of each evaluation text information feature may include averaging the global similarity of "very good 'prompt words'" and the local similarity of "very good 'prompt words'" to obtain the average similarity of "very good 'prompt words'", averaging the global similarity of "average 'prompt words'" and the local similarity of "average 'prompt words'" to obtain the average similarity of "average 'prompt words'", averaging the global similarity of "very poor 'prompt words'" and the local similarity of "very poor 'prompt words'" to obtain the average similarity of "very poor 'prompt words'".

[0078] S206 : Obtaining an aesthetic quality evaluation result of the image to be evaluated according to the average similarity of each evaluation text information feature.

[0079] Among them, the method for obtaining the aesthetic quality assessment result of the image to be evaluated may include summing the average similarities of each evaluation text information feature to obtain the global similarity between the evaluation text information feature and the image to be evaluated as the aesthetic quality assessment result of the image to be evaluated.

[0080] Another image aesthetic quality assessment method provided in an embodiment of the present application can limit the prompt word text according to the evaluation words of "very good", "average" and "very poor", and obtain the average similarity by comparing the local image features and the global image features with the feature information of the prompt word text under the evaluation words of different degrees, thereby enriching the dimension of aesthetic assessment and improving the accuracy of image aesthetic assessment.

[0081] Figure 3 This is a schematic diagram of the structure of the image aesthetic quality assessment device provided in the embodiment of the present application. Figure 3 As shown, the image aesthetic quality assessment device 30 includes: an acquisition module 301, a first determination module 302, a second determination module 303, a third determination module 304, and a obtaining module 305. Among them: The acquisition module 301 is used to acquire text feature information and image feature information. The text feature information is a text feature obtained by extracting features from the prompt word text according to preset evaluation keywords. The evaluation keywords represent the matching degree between the prompt word text and the image to be evaluated. The image feature information includes the global image features of the image to be evaluated and the local image features of the target local image in the image to be evaluated. A first determination module 302 is used to determine the global similarity between the global image feature and the target text feature, and the local similarity between the local image feature and the target text feature according to the target text feature and the image feature information in the text feature information; A second determination module 303 is used to determine the average similarity between the target text feature and the image feature information according to the global similarity and the local similarity; The third determination module 304 is used to determine the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight according to the average similarity between the target text feature and the image feature information, wherein the weight of the matching degree weight is determined according to a preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information; The obtaining module 305 is used to obtain the aesthetic quality evaluation result of the image to be evaluated according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight.

[0082] In the embodiment of the present application, the acquisition module 301 may also be used for: Obtain prompt word text and image feature information; Obtaining a prompt word evaluation text according to the prompt word text and preset evaluation keywords, wherein the number of evaluation keywords is not less than 2; The prompt word evaluation text is input into the text encoder of the visual language model to obtain text feature information.

[0083] In the embodiment of the present application, the acquisition module 301 may also be used for: Obtain text feature information and the image to be evaluated; According to the preset size requirement, the image to be evaluated is scaled to obtain a global image of the image to be evaluated; According to the preset size requirements, the image to be evaluated is intercepted and processed to obtain the target partial image; The global image and the target local image of the image to be evaluated are respectively input into the image encoder of the visual language model to obtain the global image features and the local image features; Image feature information is obtained according to global image features and local image features.

[0084] In the embodiment of the present application, the second determining module 303 may also be used to: Perform weighted averaging on the local similarities to obtain the initial average similarity; According to the initial average similarity and the global similarity, the average similarity of the target text feature and image feature information is obtained.

[0085] In the embodiment of the present application, the third determining module 304 may also be used to: Perform weighted summation on the average similarity of target text features and image feature information to obtain the global average similarity; According to the global average similarity and the average similarity, the matching degree weight of the target text feature in the text feature information is determined; According to the matching degree weight of the target text feature in the text feature information and the preset weight assignment rule, the weight of the matching degree weight is obtained.

[0086] In the embodiment of the present application, the third determining module 304 may also be used to: Sorting the matching degree weights of the target text features in the text feature information to obtain a matching degree weight sequence; According to the weight assignment rule, the matching degree weights in the matching degree weight sequence are assigned to obtain the weights of the matching degree weights, wherein the earlier the position of the matching degree weight in the matching degree weight sequence is, the greater the weight of the matching degree weight is.

[0087] In the embodiment of the present application, the obtaining module 305 may also be used for: According to the matching degree weight and the weight of the matching degree weight, a matching degree weight value is obtained; Perform weighted summation on the matching degree weight values ​​to obtain the matching degree weight value; According to the preset weight distribution range, the matching degree weight is adjusted to obtain the aesthetic quality evaluation result of the image to be evaluated.

[0088] As can be seen from the above, the image aesthetic quality assessment device of the embodiment of the present application comprises an acquisition module 301, which is used to acquire text feature information and image feature information, wherein the text feature information is text features obtained after feature extraction of the prompt word text according to preset evaluation keywords, and the evaluation keywords represent the degree of match between the prompt word text and the image to be evaluated, and the image feature information includes the global image features of the image to be evaluated, and the local image features of the target local image in the image to be evaluated; a first determination module 302, which is used to determine the global similarity between the global image features and the target text features, and the local similarity between the local image features and the target text features according to the target text features and the image feature information in the text feature information. similarity; a second determining module 303, used to determine the average similarity of the target text feature and the image feature information according to the global similarity and the local similarity; a third determining module 304, used to determine the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight according to the average similarity of the target text feature and the image feature information, wherein the weight of the matching degree weight is determined according to a preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information; an obtaining module 305, used to obtain the aesthetic quality evaluation result of the image to be evaluated according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight. Therefore, the embodiment of the present application can extract text features of multiple matching degrees for the prompt word text based on the preset evaluation keywords. In order to comprehensively consider the local features and complete semantic information of the image to be evaluated, the text feature information is respectively calculated with the global image feature information and the local image feature information. Then, the similarity between the image to be evaluated and the text features of each matching degree and the matching degree weight of each text feature in all text features are obtained, so as to obtain the aesthetic quality evaluation result according to the preset weight assignment rules and the matching degree weight, and produce an image aesthetic evaluation based on the similarity between the image and the prompt word of the generated image, thereby improving the evaluation accuracy.

[0089] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 40 includes: The electronic device 40 may include a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a communication component 403 and other components. The processor 401 , the memory 402 and the communication component 403 are connected via a bus 404 .

[0090] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that the at least one processor 401 executes the above-mentioned image aesthetic quality assessment method.

[0091] The specific implementation process of the processor 401 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0092] In the above Figure 4 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0093] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0094] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0095] In some embodiments, a computer program product is further proposed, comprising a computer program or instructions, which implement the steps in any of the above-mentioned image aesthetic quality assessment methods when executed by a processor.

[0096] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0097] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0098] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple instructions are stored. The instructions can be loaded by a processor to execute the steps in any one of the image aesthetic quality assessment methods provided in the embodiments of the present application.

[0099] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0100] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.

[0101] Since the instructions stored in the storage medium can execute the steps in any one of the image aesthetic quality assessment methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the image aesthetic quality assessment methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0102] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0103] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for evaluating image aesthetic quality, characterized in that: Applied to image aesthetic quality calculation system, including: Acquire text feature information and image feature information, wherein the text feature information is a text feature obtained by extracting features from a prompt word text according to preset evaluation keywords, the evaluation keywords represent the degree of matching between the prompt word text and the image to be evaluated, and the image feature information includes global image features of the image to be evaluated and local image features of a target local image in the image to be evaluated; Determining the global similarity between the global image feature and the target text feature, and the local similarity between the local image feature and the target text feature, according to the target text feature in the text feature information and the image feature information; Determining an average similarity between the target text feature and the image feature information according to the global similarity and the local similarity; Determine, according to the average similarity between the target text feature and the image feature information, a matching degree weight of the target text feature in the text feature information and a weight of the matching degree weight, wherein the weight of the matching degree weight is determined according to a preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information; The aesthetic quality evaluation result of the image to be evaluated is obtained according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight.

2. The method according to claim 1, characterized in that: The obtaining of text feature information and image feature information includes: Obtaining the prompt word text and image feature information; Obtaining a prompt word evaluation text according to the prompt word text and the preset evaluation keywords, wherein the number of the evaluation keywords is not less than 2; The prompt word evaluation text is input into a text encoder of a visual language model to obtain the text feature information.

3. The method according to claim 1, characterized in that The obtaining of text feature information and image feature information includes: Acquire the text feature information and the image to be evaluated; Scaling the image to be evaluated according to a preset size requirement to obtain a global image of the image to be evaluated; According to the preset size requirement, the image to be evaluated is intercepted to obtain the target partial image; Inputting the global image of the image to be evaluated and the target local image into an image encoder of a visual language model respectively to obtain the global image features and the local image features; The image feature information is obtained according to the global image feature and the local image feature.

4. The method according to claim 1, characterized in that: Determining the average similarity between the target text feature and the image feature information according to the global similarity and the local similarity includes: Performing weighted averaging processing on the local similarities to obtain an initial average similarity; The average similarity between the target text feature and the image feature information is obtained according to the initial average similarity and the global similarity.

5. The method according to claim 1, characterized in that: Determining the matching degree weight of the target text feature in the text feature information and the weight value of the matching degree weight according to the average similarity between the target text feature and the image feature information includes: Performing weighted sum processing on the average similarities of the target text feature and the image feature information to obtain a global average similarity; Determining a matching degree weight of the target text feature in the text feature information according to the global average similarity and the average similarity; The weight of the matching degree weight is obtained according to the matching degree weight of the target text feature in the text feature information and the preset weight assignment rule.

6. The method according to claim 5, characterized in that The step of obtaining the weight of the matching degree weight according to the matching degree weight of the target text feature in the text feature information and the preset weight assignment rule comprises: Sorting the matching degree weights of the target text features in the text feature information to obtain a matching degree weight sequence; According to the weight assignment rule, the matching degree weights in the matching degree weight sequence are assigned to obtain the weights of the matching degree weights, wherein the earlier the position of the matching degree weight in the matching degree weight sequence is, the greater the weight of the matching degree weight is.

7. The method according to claim 1, characterized in that The step of obtaining the aesthetic quality evaluation result of the image to be evaluated according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight comprises: Obtaining a matching degree weight value according to the matching degree weight and the weight of the matching degree weight; Performing weighted summation processing on the matching degree weight values ​​to obtain a matching degree weight value; According to a preset weight distribution range, the matching degree weight is adjusted to obtain an aesthetic quality assessment result of the image to be assessed.

8. An image aesthetic quality assessment device, characterized in that: include: An acquisition module, used to acquire text feature information and image feature information, wherein the text feature information is a text feature obtained by extracting features from a prompt word text according to preset evaluation keywords, wherein the evaluation keywords represent the degree of matching between the prompt word text and the image to be evaluated, and the image feature information includes global image features of the image to be evaluated and local image features of a target local image in the image to be evaluated; A first determination module, configured to determine the global similarity between the global image feature and the target text feature, and the local similarity between the local image feature and the target text feature according to the target text feature in the text feature information and the image feature information; A second determination module is used to determine the average similarity between the target text feature and the image feature information according to the global similarity and the local similarity; A third determination module is used to determine the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight according to the average similarity between the target text feature and the image feature information, wherein the weight of the matching degree weight is determined according to a preset weight assignment rule and the size of the matching degree weight of the target text feature in the text feature information; The obtaining module is used to obtain the aesthetic quality evaluation result of the image to be evaluated according to the matching degree weight of the target text feature in the text feature information and the weight of the matching degree weight.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the image aesthetic quality assessment method according to any one of claims 1 to 7.

Images