Image aesthetic quality evaluation method and system

Abstract

The present invention provides an image aesthetic quality evaluation method and system, the method comprising: inputting a target image to be evaluated into an image aesthetic evaluation model, obtaining aesthetic target evaluation values ​​corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; and the aesthetic evaluation results of the target image are obtained according to the aesthetic target evaluation values. The present invention can obtain more accurate image aesthetic evaluation results.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image aesthetic quality evaluation method and system. Background Art

[0002] With the rapid progress of text-generated image models (text-generated image models for short), higher and higher standards have been put forward for the quality of images generated by text-generated image models. How to screen out high-quality images with high clarity and excellent aesthetics has become a crucial task.

[0003] Most existing image aesthetic scoring schemes rely on image features to predict aesthetic scores. This method not only relies on a large manually annotated dataset for model training, but also often only extracts information from the image itself during the scoring process, ignoring other key aesthetic perception factors such as cultural background, shooting intention, and the audience's subjective feelings. This limitation makes the aesthetic scoring model prone to bias during prediction, which in turn affects the accuracy and reliability of the scoring results.

[0004] Therefore, there is an urgent need for an image aesthetic quality evaluation method and system to solve the above problems. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides an image aesthetic quality evaluation method and system.

[0006] The present invention provides an image aesthetic quality evaluation method, comprising: Inputting the target image to be evaluated into the image aesthetic evaluation model, obtaining the aesthetic target evaluation value of the target image corresponding to different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; According to each of the aesthetic target evaluation values, an aesthetic evaluation result of the target image is obtained.

[0007] According to an image aesthetic quality evaluation method provided by the present invention, the image aesthetic evaluation model is trained by the following steps: Obtain sample theme images of various image theme style types; Marking the sample subject image with the aesthetic sample evaluation labels corresponding to different aesthetic evaluation dimensions to obtain the sample image; The large language model is trained according to the sample image, and the image aesthetic evaluation model is obtained when the training result meets the preset training condition.

[0008] According to a method for evaluating image aesthetic quality provided by the present invention, the step of marking the sample subject image with the aesthetic sample evaluation labels corresponding to different aesthetic evaluation dimensions to obtain the sample image comprises: Determining, according to the image attributes of the sample subject image, a plurality of the aesthetic evaluation dimensions corresponding to the sample subject image; The preset aesthetic evaluation values ​​corresponding to the various aesthetic evaluation dimensions are used as the aesthetic sample evaluation labels, and the sample subject images are marked to obtain the sample images.

[0009] According to a method for evaluating image aesthetic quality provided by the present invention, determining a plurality of aesthetic evaluation dimensions corresponding to the sample subject image according to the image attributes of the sample subject image comprises: Based on the image clarity attribute, image color attribute, image light attribute and image composition attribute among the image attributes, the multiple aesthetic evaluation dimensions corresponding to the sample subject image are determined, wherein the multiple aesthetic evaluation dimensions include image clarity dimension, image color dimension, image light dimension and image composition dimension.

[0010] According to an image aesthetic quality evaluation method provided by the present invention, the large language model is trained according to the sample image, and when the training result meets the preset training condition, the image aesthetic evaluation model is obtained, including: Iteratively training the large language model according to the sample image to obtain an aesthetic evaluation prediction value output by the large language model in each training cycle and a reward value output by a preset reward function in each training cycle, wherein the preset reward function is constructed based on the preset aesthetic evaluation value and the aesthetic evaluation prediction value; Based on the reinforcement learning algorithm and the reward value obtained in the previous training cycle, the corresponding model parameters in the large language model of the current training cycle are fine-tuned. If, after the model parameters of the large language model of the current training cycle are fine-tuned, the reward value of the current training cycle obtained meets the preset reward value, the image aesthetic evaluation model is obtained.

[0011] According to an image aesthetic quality evaluation method provided by the present invention, obtaining the aesthetic evaluation result of the target image according to each of the aesthetic target evaluation values ​​includes: Determining aesthetic evaluation dimension weight information corresponding to each aesthetic target evaluation value according to the image theme style type of the target image; A weighted summation process is performed according to each of the aesthetic target evaluation values ​​and the aesthetic evaluation dimension weight information corresponding to each of the aesthetic target evaluation values ​​to obtain the aesthetic evaluation result of the target image.

[0012] The present invention also provides an image aesthetic quality evaluation system, comprising: A multi-dimensional rating module is used to input a target image to be evaluated into an image aesthetic evaluation model to obtain aesthetic target evaluation values ​​corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; The aesthetic evaluation module is used to obtain the aesthetic evaluation result of the target image according to each of the aesthetic target evaluation values.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned image aesthetic quality evaluation methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image aesthetic quality evaluation method described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image aesthetic quality assessment method described above is implemented.

[0016] The image aesthetic quality evaluation method and system provided by the present invention train a large language model by using sample images marked with aesthetic sample evaluation labels, and then use the trained image aesthetic evaluation model to perform aesthetic evaluation on the target image in different aesthetic evaluation dimensions, so as to obtain a more accurate aesthetic evaluation result based on the aesthetic target evaluation value corresponding to each aesthetic evaluation dimension output by the image aesthetic evaluation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of the process of the image aesthetic quality evaluation method provided by the present invention; Figure 2 A schematic diagram of the structure of the image aesthetic quality evaluation system provided by the present invention; Figure 3This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Judging the beauty of an image involves a comprehensive consideration of many factors. First, high-quality images often show a strong sense of photography, which includes clear subject matter, good lighting effects, harmonious symbiosis between colors, vivid expression of colors, and well-designed composition. These factors together constitute an important basis for influencing aesthetic scores.

[0021] Furthermore, different image themes will also have different applicable evaluation criteria. For example, landscape images and portrait images have different aesthetic emphases. Landscape images are more often judged on whether their colors are harmonious and unified, and whether their composition is clever and reasonable; while portrait images focus more on the clarity of the character's image, the coordination of the body proportions, and whether the character's position in the picture is appropriate.

[0022] In addition, different styles of art also follow their own unique evaluation criteria. For example, in the aesthetic judgment of oil paintings and sketches, oil paintings pay more attention to whether the use of colors is rich and varied, and whether the brushstrokes are rich in layers; while sketches emphasize the precise control and expressiveness of lines. These differences reflect that when evaluating the beauty of an image, the evaluation criteria need to be formulated and adjusted according to the specific type and style of the image.

[0023] Traditional aesthetic scoring schemes mostly focus on extracting low-level features of images using various pixel filters or encoding techniques. However, a major limitation of these methods is that they mainly focus on low-level features and ignore the rich semantic information contained in the image, thus limiting the breadth and depth of aesthetic ratings and resulting in unsatisfactory accuracy.

[0024] In contrast, deep learning solutions can automatically learn more complex and comprehensive low-level and high-level features from images by building artificial neural networks. However, to achieve this, existing deep learning models need to rely on a large number of manually annotated images for training in order to achieve accuracy that exceeds traditional methods in image aesthetic scoring tasks. Despite this, deep learning solutions also face another challenge: they often rely too much on information learned directly from images for aesthetic scoring, while ignoring other factors that are critical to aesthetic evaluation, such as cultural background, emotional expression, and audience perception. This limitation may cause a certain bias in the aesthetic scoring model, affecting the comprehensiveness and accuracy of the scoring.

[0025] In order to solve the problems existing in the existing image aesthetic scoring, this paper proposes to use a large language model and fine-tune it in combination with a reinforcement learning method to optimize the image classification and aesthetic scoring capabilities. The large language model is trained with massive data and can be widely applied to various fields, with the general ability to solve various tasks.

[0026] For the specific task of image aesthetic scoring, the present invention plans to adopt the following strategies: First, the large language model is fine-tuned using reinforcement learning technology, and further training is carried out by introducing data sets in specific fields, so that the large language model can more accurately adapt to the specific needs of aesthetic scoring. This process uses a relatively small amount of data for fine-tuning training, but can obtain aesthetic scoring results that are more consistent with aesthetic perception. In actual operation, when an image is input, the fine-tuned large language model will first describe the theme or type of artwork of the image. Subsequently, the large language model will deeply analyze multiple key aesthetic elements of the image, including but not limited to clarity, color characteristics (such as color harmony, color vividness, saturation, hue, brightness, complementary colors and the use of two-tone), light effects (such as good lighting and balanced exposure), and composition principles (such as subject clarity, application of the rule of thirds, sense of balance, repeated elements and symmetry, etc.).

[0027] Finally, based on the above detailed analysis, the large language model will give a comprehensive aesthetic score that comprehensively reflects the performance of the image in multiple aesthetic dimensions, thereby providing more accurate and comprehensive aesthetic evaluation results.

[0028] Figure 1 A schematic diagram of the process of the image aesthetic quality evaluation method provided by the present invention is shown in FIG. Figure 1 As shown, the present invention provides an image aesthetic quality evaluation method, comprising: Step 101, inputting the target image to be evaluated into the image aesthetic evaluation model, and obtaining the aesthetic target evaluation value corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions;.

[0029] In the present invention, a target image to be subjected to image aesthetic evaluation is input into an image aesthetic evaluation model, and the task of this image aesthetic evaluation model is to output specific evaluation values ​​of the target image in different aesthetic evaluation dimensions.

[0030] In the present invention, the aesthetic evaluation dimensions are pre-set and together constitute the key elements for evaluating the aesthetic quality of an image, mainly including four aspects: image clarity, image color, image light and image composition.

[0031] Specifically, the image clarity dimension is an important indicator to measure the visibility and sharpness of details in an image. In aesthetic evaluation, a high-definition image can convey information more accurately and bring a better visual experience to the audience.

[0032] The image color dimension focuses on the use and matching of colors in the image. Color harmony, color vividness, saturation, hue, brightness and other color characteristics are all key factors affecting the aesthetic quality of the image. An excellent color matching can enhance the expressiveness and attractiveness of the image.

[0033] The image light dimension mainly examines the lighting effect and exposure balance in the image. Good lighting and balanced exposure can make the image look more natural, three-dimensional and layered, thereby enhancing its aesthetic value.

[0034] The image composition dimension involves the organization and arrangement of elements in the image. A reasonable composition can guide the audience's sight, highlight the theme, and enhance the expressiveness and appeal of the image. Common composition principles include clear subject, application of the rule of thirds, sense of balance, repeated elements and symmetry.

[0035] In the present invention, the image aesthetic evaluation model is trained based on sample images marked with aesthetic sample evaluation labels. These sample images cover a variety of aesthetic styles and themes, and each sample image needs to be marked with a corresponding aesthetic sample evaluation label in different aesthetic evaluation dimensions. The sample images used in the present invention can be some reference images with relatively distinct thematic styles. These sample images have strong expressiveness and appeal in each aesthetic evaluation dimension. In the early stage of model training, the evaluation values ​​of each aesthetic evaluation dimension are obtained by combining manual evaluation and some existing image automatic evaluation technologies as the aesthetic sample evaluation labels corresponding to the sample images.

[0036] During the training process, the large language model will continuously try to predict the evaluation values ​​of sample images in different aesthetic evaluation dimensions, and compare and correct them with the real aesthetic evaluation labels. Through multiple iterations and optimizations, the large language model can eventually form a stable and accurate image aesthetic evaluation model, providing strong support for image aesthetic evaluation.

[0037] Step 102: Obtain an aesthetic evaluation result of the target image according to each of the aesthetic target evaluation values.

[0038] In the present invention, the specific evaluation scores of the target image in different aesthetic dimensions are reflected by the aesthetic target evaluation values ​​corresponding to each aesthetic evaluation dimension output by the image aesthetic evaluation model. These dimensions include clarity, color, light, and composition, which together constitute the aesthetic characteristics of the image. Each dimension will be scored according to specific evaluation criteria, which can be based on various factors such as art theory, photography skills, and audience preferences.

[0039] Specifically, after obtaining the evaluation values ​​of each aesthetic evaluation dimension, the present invention calculates a comprehensive aesthetic score based on these evaluation values. This process involves weighted summation of the evaluation values ​​of each aesthetic evaluation dimension. The distribution of weights can be adjusted according to different evaluation criteria, image themes or artwork styles to ensure that the final aesthetic evaluation result can objectively and comprehensively reflect the aesthetic value of the image.

[0040] The image aesthetic quality evaluation method provided by the present invention trains a large language model by using sample images marked with aesthetic sample evaluation labels, and then uses the trained image aesthetic evaluation model to perform aesthetic evaluation on the target image in different aesthetic evaluation dimensions, thereby obtaining a more accurate aesthetic evaluation result based on the aesthetic target evaluation value corresponding to each aesthetic evaluation dimension output by the image aesthetic evaluation model.

[0041] Based on the above embodiment, the image aesthetic evaluation model is trained through the following steps: Obtain sample theme images of various image theme style types; Marking the sample subject image with the aesthetic sample evaluation labels corresponding to different aesthetic evaluation dimensions to obtain the sample image; The large language model is trained according to the sample image, and the image aesthetic evaluation model is obtained when the training result meets the preset training condition.

[0042] In the present invention, images are divided into different subject categories according to needs, such as portraits, portrait cartoons, animals, plants, food, architecture, transportation, equipment products, design, human landscapes, natural landscapes, indoor scenes, outdoor scenes and artworks, etc. Such classification is helpful for subsequent targeted collection and organization of images. For the large category of artworks, it is necessary to further divide them according to artistic styles, including works of various styles such as stick drawings, sketches, oil paintings, ink paintings, Chinese paintings, illustrations, two-dimensional, comics and watercolors. Each style can be further subdivided into different art schools and artists' works to ensure the diversity and depth of artworks.

[0043] Furthermore, after collecting enough sample theme images of image theme style types, these images are marked with aesthetic sample evaluation labels. In the present invention, these aesthetic sample evaluation labels are obtained based on multiple aesthetic evaluation dimensions to ensure that the model can fully understand the aesthetic value of the image. In one embodiment, manual labeling can be used to invite professional photographers, artists or aesthetic experts to score or classify sample theme images from different aesthetic evaluation dimensions. Machine-assisted labeling can also be used to use existing aesthetic evaluation algorithms or models to make preliminary evaluations of images, and then make adjustments in combination with manual review.

[0044] After obtaining the labeled sample images, the large language model is trained using these data. In the present invention, the large language model has been trained with a large amount of data in the early stage and is already able to understand and process images. Furthermore, the labeled sample images are input into the large language model for training, and by continuously adjusting the parameters and structure of the model, the model can gradually learn the characteristics and laws of image aesthetic evaluation. During the training process, data enhancement, regularization and other technologies can also be used to improve the generalization ability and robustness of the model.

[0045] After a certain period of training, it is necessary to evaluate whether the performance of the large language model in the training process meets the preset training conditions. These conditions can include evaluation indicators such as the model's accuracy, recall rate, F1 score, and performance in practical applications. Specifically, the performance of the model can be evaluated using a validation set or a test set. By comparing the difference between the model's prediction results and the true labels, the model's accuracy, recall rate and other indicators can be calculated, and then the model can be optimized and adjusted based on the evaluation results. When the performance of the model meets the preset conditions, the final image aesthetic evaluation model can be obtained. This image aesthetic evaluation model can be used to perform aesthetic evaluation on new images and output corresponding evaluation results or suggestions.

[0046] On the basis of the above embodiment, the step of marking the sample subject image with the aesthetic sample evaluation labels corresponding to different aesthetic evaluation dimensions to obtain the sample image includes: Determining, according to the image attributes of the sample subject image, a plurality of the aesthetic evaluation dimensions corresponding to the sample subject image; The preset aesthetic evaluation values ​​corresponding to the various aesthetic evaluation dimensions are used as the aesthetic sample evaluation labels, and the sample subject images are marked to obtain the sample images.

[0047] When constructing an aesthetic dataset, we first need to clarify the aesthetic evaluation dimensions on which the sample subject images will be judged.

[0048] The image attributes such as the photographic sense, clear theme, good lighting, harmonious colors, vivid colors and excellent composition of the image will have a corresponding impact on the aesthetic score. Based on the above embodiment, the multiple aesthetic evaluation dimensions corresponding to the sample theme image are determined according to the image attributes of the sample theme image, including: Based on the image clarity attribute, image color attribute, image light attribute and image composition attribute among the image attributes, the multiple aesthetic evaluation dimensions corresponding to the sample subject image are determined, wherein the multiple aesthetic evaluation dimensions include image clarity dimension, image color dimension, image light dimension and image composition dimension.

[0049] When constructing the training data set, the present invention divides various image attributes according to the following aesthetic evaluation dimensions: 1. The clarity dimension is an important indicator for measuring the aesthetics of an image. A clear image can show more details, allowing viewers to understand the image content more clearly. When determining the clarity dimension, the present invention may also consider issues such as the resolution, sharpness, and whether there is blur or distortion in the image.

[0050] Second, the color dimension is an important part of image aesthetics, including hue, saturation, brightness and contrast. Beautiful images usually have harmonious and vivid color combinations, and can use colors close to the color wheel or complementary colors to create visual appeal. For example, use colors close to the color wheel (red and orange or blue and green) or complementary colors (blue and orange or red and green); at the same time, in the color dimension, factors such as hue, saturation (intensity or purity of color), brightness (brightness or darkness of color) and contrast (difference between light and dark areas) should also be considered.

[0051] 3. Light dimension is a key factor in shaping the atmosphere and emotion of an image. Good lighting and balanced exposure can make an image more vivid and attractive. When determining the light dimension, you can focus on the lighting conditions, exposure, and processing of shadow and highlight areas of the image.

[0052] Fourth, the composition dimension refers to the arrangement and organization of elements in the image, which affects the visual balance and appeal of the image. Beautiful images usually have a clear subject or follow composition principles such as the rule of thirds, balance, repetition and symmetry.

[0053] After determining the aesthetic evaluation dimensions, the present invention marks the sample subject images and assigns an aesthetic preset evaluation value to each dimension as an aesthetic sample evaluation label. Specifically, according to the characteristics of each aesthetic evaluation dimension, a corresponding scoring standard is set. For example, clarity can be divided into different levels such as blur, general clarity, and very clarity, and correspondingly, corresponding scores can be assigned to different evaluation levels; color can be scored from aspects such as harmony and vividness; light can be judged from aspects such as lighting conditions and exposure balance; composition can be scored from aspects such as subject clarity and application of composition principles. By assigning a specific evaluation value to each dimension, and then using these evaluation values ​​as aesthetic sample evaluation labels, a set of sample images containing multiple aesthetic evaluation dimensions and aesthetic preset evaluation values ​​corresponding to each aesthetic evaluation dimension is obtained for subsequent model training.

[0054] In one embodiment, when marking the various aesthetic evaluation dimensions of the sample subject images, the aesthetic attributes of the sample subject images can be evaluated and scored by experts or trained annotators to capture the subtle aesthetic features in the image. In this process, the annotator is required to pay attention not only to the overall beauty of the image, but also to evaluate the various details that constitute the overall beauty, such as the use of light, the smoothness of lines, color matching, etc. In addition, when selecting sample subject images, a part is selected from a large number of images. These sample subject images should be representative and cover multiple aspects of aesthetic judgment. At the same time, the number is moderate for easy management and annotation. The present invention selects a small but representative number of images to help reduce the workload while ensuring the quality of the data set.

[0055] Different people may have different aesthetic feelings and evaluations of the same picture. During the annotation process, the present invention requires multiple people to annotate the attribute scores of the same sample subject image to reduce the impact of personal aesthetic differences on the annotation results and obtain a more comprehensive and objective score. By summarizing and analyzing the scores of multiple annotators, the average score or median score of each picture can be calculated, thereby offsetting the impact of personal aesthetic preferences on the results to a certain extent. In addition, the present invention can also use statistical methods (such as standard deviation, coefficient of variation, etc.) to evaluate the consistency of the annotation results to ensure the reliability and validity of the data set.

[0056] On the basis of the above embodiment, the large language model is trained according to the sample image, and when the training result meets the preset training condition, the image aesthetic evaluation model is obtained, including: Iteratively training the large language model according to the sample image to obtain an aesthetic evaluation prediction value output by the large language model in each training cycle and a reward value output by a preset reward function in each training cycle, wherein the preset reward function is constructed based on the preset aesthetic evaluation value and the aesthetic evaluation prediction value; Based on the reinforcement learning algorithm and the reward value obtained in the previous training cycle, the corresponding model parameters in the large language model of the current training cycle are fine-tuned. If, after the model parameters of the large language model of the current training cycle are fine-tuned, the reward value of the current training cycle obtained meets the preset reward value, the image aesthetic evaluation model is obtained.

[0057] In the present invention, the training dataset contains sample images of various themes and styles of artworks. These sample images will be used to train the large language model so that it can generate evaluations that meet aesthetic preferences.

[0058] Furthermore, the constructed training data set is used to fine-tune the large language model so that it has the ability to describe the theme and artwork of the image. Then, the image is selected according to the theme and style ratio of the image and input into the large language model. The model will output the theme of the image, the description of the artwork, and the scores of dimensions such as clarity, color, light, and composition. In the present invention, the preset reward function is constructed based on the preset aesthetic evaluation value (which can be understood as the expert's evaluation of the aesthetics of the image) and the aesthetic evaluation prediction value output by the large language model. The purpose of this reward function is to calculate the difference between the aesthetic evaluation prediction value output by the large language model and the true aesthetic preset evaluation value. The smaller the difference, the higher the reward value; the larger the difference, the lower the reward value.

[0059] In each training cycle, the present invention uses sample images to input into the large language model to obtain the aesthetic evaluation prediction value output by the model. At the same time, the reward value between this prediction value and the actual aesthetic preset evaluation value is calculated using a preset reward function. This reward value reflects the performance of the current large language model in the aesthetic evaluation task, that is, the consistency between the evaluation output by the model and the evaluation of human experts.

[0060] Preferably, in the present invention, the parameters of the large language model are fine-tuned using a reinforcement learning algorithm, with the goal of fine-tuning the model parameters to reduce the difference between the predicted value of the aesthetic evaluation and the true evaluation value. Specifically, the present invention uses a proximal policy optimization (PPO) reinforcement learning algorithm to fine-tune the parameters of the large language model of the current training cycle according to the reward value obtained in the previous training cycle. In the present invention, the PPO algorithm calculates a gradient that points to a direction that can increase the reward value, and then updates the model parameters along this direction. If the reward value obtained after fine-tuning the parameters of the large language model of the current training cycle meets the preset reward value (that is, it reaches an acceptable performance level), it is determined that the training process can be terminated and an image aesthetic evaluation model is obtained.

[0061] The invention proposes a method of fine-tuning a large language model based on reinforcement learning, which uses a small amount of specific data to fine-tune the large language model using a reinforcement learning method, and then evaluates the aesthetic score of the image based on the fine-tuned large language model, solving the problem that the aesthetic scoring task of the existing deep learning solution requires a large amount of manually labeled data. At the same time, in order to solve the problem of learning aesthetic scores directly from images while ignoring the perception of aesthetics by other factors, when constructing a training data set, it is necessary to include information such as image themes, artwork styles, clarity scores, color scores, light scores, and composition scores, so that the large language model is more suitable for aesthetic scoring tasks.

[0062] On the basis of the above embodiment, obtaining the aesthetic evaluation result of the target image according to each of the aesthetic target evaluation values ​​includes: Determining aesthetic evaluation dimension weight information corresponding to each aesthetic target evaluation value according to the image theme style type of the target image; A weighted summation process is performed according to each of the aesthetic target evaluation values ​​and the aesthetic evaluation dimension weight information corresponding to each of the aesthetic target evaluation values ​​to obtain the aesthetic evaluation result of the target image.

[0063] In the present invention, different image themes or artwork styles require different aesthetic dimensions to be focused on. For example, a landscape painting may emphasize the use of light and color, while a portrait may focus more on clarity and composition. The aesthetic target evaluation value is a quantitative assessment of the image's performance in various aesthetic dimensions, including clarity, color, light, and composition. For each dimension, there will be one or more specific evaluation criteria for measuring the image's performance in that dimension.

[0064] In the present invention, a weight is assigned to each aesthetic dimension according to the theme style type of the image. This weight reflects the importance of the dimension in the overall aesthetic evaluation. For example, if the image is a landscape painting, the weight of light and color may be relatively high; while if it is a portrait, the weight of clarity and composition may be higher.

[0065] Furthermore, each aesthetic target evaluation value is multiplied by its corresponding weight, and then the products are added to obtain the final aesthetic evaluation result. After weighted summation, the final aesthetic evaluation result is a quantitative value that can accurately reflect the overall aesthetic performance of the image. This value can be used to compare the aesthetic quality of different images, and can also be used as a reference indicator in the image processing and optimization process.

[0066] The image aesthetic quality evaluation system provided by the present invention is described below. The image aesthetic quality evaluation system described below and the image aesthetic quality evaluation method described above can be referenced to each other.

[0067] Figure 2 The schematic diagram of the structure of the image aesthetic quality evaluation system provided by the present invention is as follows: Figure 2 As shown, the present invention provides an image aesthetic quality evaluation system, including a multi-dimensional rating module 201 and an aesthetic evaluation module 202, wherein the multi-dimensional rating module 201 is used to input a target image to be evaluated into an image aesthetic evaluation model, and obtain aesthetic target evaluation values ​​corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; and the aesthetic evaluation module 202 is used to obtain the aesthetic evaluation result of the target image according to each of the aesthetic target evaluation values.

[0068] The image aesthetic quality evaluation system provided by the present invention trains a large language model by using sample images marked with aesthetic sample evaluation labels, and then uses the trained image aesthetic evaluation model to perform aesthetic evaluation on the target image in different aesthetic evaluation dimensions, thereby obtaining a more accurate aesthetic evaluation result based on the aesthetic target evaluation value corresponding to each aesthetic evaluation dimension output by the image aesthetic evaluation model.

[0069] Based on the above embodiment, the image aesthetic quality evaluation system also includes a model training module, which is used to: obtain sample theme images of multiple image theme style types; mark the sample theme images with the aesthetic sample evaluation labels corresponding to different aesthetic evaluation dimensions to obtain the sample images; train the large language model based on the sample images, and obtain the image aesthetic evaluation model when the training results meet the preset training conditions.

[0070] Based on the above embodiment, the model training module is also used to determine the multiple aesthetic evaluation dimensions corresponding to the sample theme image according to the image attributes of the sample theme image; use the aesthetic preset evaluation value corresponding to each of the aesthetic evaluation dimensions as the aesthetic sample evaluation label, mark the sample theme image, and obtain the sample image.

[0071] Based on the above embodiments, the model training module is specifically used to determine the multiple aesthetic evaluation dimensions corresponding to the sample subject image according to the image clarity attribute, image color attribute, image light attribute and image composition attribute in the image attributes, wherein the multiple aesthetic evaluation dimensions include image clarity dimension, image color dimension, image light dimension and image composition dimension.

[0072] On the basis of the above embodiment, the model training module is specifically used to iteratively train the large language model according to the sample image, obtain the aesthetic evaluation prediction value output by the large language model in each training cycle, and the reward value output by the preset reward function in each training cycle, wherein the preset reward function is constructed based on the preset aesthetic evaluation value and the aesthetic evaluation prediction value; based on the reinforcement learning algorithm and the reward value obtained in the previous training cycle, fine-tune the corresponding model parameters in the large language model of the current training cycle; if after the model parameters of the large language model of the current training cycle are fine-tuned, the reward value of the current training cycle obtained satisfies the preset reward value, the image aesthetic evaluation model is obtained.

[0073] Based on the above embodiment, the aesthetic evaluation module is specifically used to determine the aesthetic evaluation dimension weight information corresponding to each aesthetic target evaluation value according to the image theme style type of the target image; and perform weighted summation processing on each aesthetic target evaluation value and the aesthetic evaluation dimension weight information corresponding to each aesthetic target evaluation value to obtain the aesthetic evaluation result of the target image.

[0074] The system provided in the embodiment of the present invention is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for the specific process and detailed contents, which will not be repeated here.

[0075] Figure 3 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302 and the memory 303 communicate with each other through the communication bus 304. The processor 301 may call the logic instructions in the memory 303 to execute the image aesthetic quality evaluation method, which includes: inputting the target image to be evaluated into the image aesthetic evaluation model, and obtaining the aesthetic target evaluation value corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; and the aesthetic evaluation result of the target image is obtained according to each of the aesthetic target evaluation values.

[0076] In addition, the logic instructions in the above-mentioned memory 303 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0077] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the image aesthetic quality evaluation method provided by the above-mentioned methods, and the method includes: inputting a target image to be evaluated into an image aesthetic evaluation model, and obtaining an aesthetic target evaluation value corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; and according to each of the aesthetic target evaluation values, an aesthetic evaluation result of the target image is obtained.

[0078] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the image aesthetic quality evaluation method provided in the above-mentioned embodiments, the method comprising: inputting a target image to be evaluated into an image aesthetic evaluation model, and obtaining aesthetic target evaluation values ​​corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; and according to each of the aesthetic target evaluation values, the aesthetic evaluation result of the target image is obtained.

[0079] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0080] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating image aesthetic quality, characterized in that: include: Inputting the target image to be evaluated into the image aesthetic evaluation model, obtaining the aesthetic target evaluation value of the target image corresponding to different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; According to each of the aesthetic target evaluation values, an aesthetic evaluation result of the target image is obtained.

2. The image aesthetic quality evaluation method according to claim 1, characterized in that: The image aesthetic evaluation model is trained by the following steps: Obtain sample theme images of various image theme style types; Marking the sample subject image with the aesthetic sample evaluation labels corresponding to different aesthetic evaluation dimensions to obtain the sample image; The large language model is trained according to the sample image, and the image aesthetic evaluation model is obtained when the training result meets the preset training condition.

3. The image aesthetic quality evaluation method according to claim 2, characterized in that: The step of marking the sample subject image with the aesthetic sample evaluation labels corresponding to different aesthetic evaluation dimensions to obtain the sample image comprises: Determining, according to the image attributes of the sample subject image, a plurality of the aesthetic evaluation dimensions corresponding to the sample subject image; The preset aesthetic evaluation values ​​corresponding to the various aesthetic evaluation dimensions are used as the aesthetic sample evaluation labels, and the sample subject images are marked to obtain the sample images.

4. The image aesthetic quality evaluation method according to claim 3, characterized in that: Determining the plurality of aesthetic evaluation dimensions corresponding to the sample subject image according to the image attributes of the sample subject image includes: Based on the image clarity attribute, image color attribute, image light attribute and image composition attribute among the image attributes, the multiple aesthetic evaluation dimensions corresponding to the sample subject image are determined, wherein the multiple aesthetic evaluation dimensions include image clarity dimension, image color dimension, image light dimension and image composition dimension.

5. The image aesthetic quality evaluation method according to claim 3, characterized in that: The step of training the large language model according to the sample image and obtaining the image aesthetic evaluation model when the training result meets the preset training condition comprises: Iteratively training the large language model according to the sample image to obtain an aesthetic evaluation prediction value output by the large language model in each training cycle and a reward value output by a preset reward function in each training cycle, wherein the preset reward function is constructed based on the preset aesthetic evaluation value and the aesthetic evaluation prediction value; Based on the reinforcement learning algorithm and the reward value obtained in the previous training cycle, the corresponding model parameters in the large language model of the current training cycle are fine-tuned. If, after the model parameters of the large language model of the current training cycle are fine-tuned, the reward value of the current training cycle obtained meets the preset reward value, the image aesthetic evaluation model is obtained.

6. The image aesthetic quality evaluation method according to any one of claims 2 to 5, characterized in that: Obtaining the aesthetic evaluation result of the target image according to each of the aesthetic target evaluation values ​​includes: Determining aesthetic evaluation dimension weight information corresponding to each aesthetic target evaluation value according to the image theme style type of the target image; A weighted summation process is performed according to each of the aesthetic target evaluation values ​​and the aesthetic evaluation dimension weight information corresponding to each of the aesthetic target evaluation values ​​to obtain the aesthetic evaluation result of the target image.

7. An image aesthetic quality evaluation system, characterized in that: include: A multi-dimensional rating module is used to input a target image to be evaluated into an image aesthetic evaluation model to obtain aesthetic target evaluation values ​​corresponding to the target image in different aesthetic evaluation dimensions output by the image aesthetic evaluation model, wherein the image aesthetic evaluation model is obtained by training a large language model with sample images marked with aesthetic sample evaluation labels; the aesthetic sample evaluation labels are constructed based on the aesthetic evaluation dimensions; The aesthetic evaluation module is used to obtain the aesthetic evaluation result of the target image according to each of the aesthetic target evaluation values.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the image aesthetic quality assessment method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image aesthetic quality assessment method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image aesthetic quality assessment method according to any one of claims 1 to 6 is implemented.

Images