Image processing method, training method of image evaluation model and electronic equipment

Abstract

The invention provides an image processing method, a training method of an image evaluation model and electronic equipment, and relates to the field of computer technology and artificial intelligence. The method comprises the steps that image data and inquiry text data are acquired, the image data are generated by an image generation model, and the inquiry text data are used for describing the requirement for quality evaluation of the image data by adopting a natural language; respectively inputting the image data and the inquiry text data into a plurality of image evaluation models, and respectively carrying out quality evaluation on the image data from corresponding evaluation dimensions by utilizing the plurality of image evaluation models to obtain a plurality of evaluation results; and integrating the plurality of evaluation results to obtain a quality evaluation result of the image data. According to the method and the device, the technical problems of relatively low accuracy and meticulousness of image quality evaluation in related technologies are solved.

Description

Technical Field

[0001] This application relates to the fields of computer technology and artificial intelligence, and more specifically, to an image processing method, a training method for an image evaluation model, and an electronic device. Background Technology

[0002] With the development of AI-generated content technology, including the generation and adjustment of images in interior decoration scenarios, image quality assessment has become a more significant technical challenge. Current image quality assessment methods suffer from a series of limitations and pain points. Automated assessment techniques often lack accuracy, fail to provide detailed evaluation of image aesthetics, and struggle to accurately reflect the true quality of images. They exhibit significant blind spots in aesthetic elements such as layout rationality and color harmony, resulting in low accuracy and detail in image quality assessments.

[0003] There is currently no effective solution to the above problems.

[0004] Application content This application provides an image processing method, an image evaluation model training method, and an electronic device to at least solve the technical problems of low accuracy and low detail in image quality evaluation in related technologies.

[0005] According to one aspect of the embodiments of this application, an image processing method is provided, the method comprising: acquiring image data and query text data, wherein the image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language; inputting the image data and query text data into multiple image assessment models respectively, and using the multiple image assessment models to perform quality assessment of the image data from corresponding assessment dimensions respectively, to obtain multiple assessment results; and integrating the multiple assessment results to obtain a quality assessment result of the image data.

[0006] According to another aspect of the embodiments of this application, a training method for an image evaluation model is also provided. The method includes: constructing multiple training images based on at least one original image; annotating the multiple training images respectively to obtain target evaluation results corresponding to the multiple training images; constructing training data based on the multiple training images and target evaluation results; and training multiple initial evaluation models separately using the training data according to different evaluation dimensions to obtain multiple image evaluation models, wherein the different image evaluation models are used to evaluate the quality of image data and query text data from different evaluation dimensions to obtain evaluation results.

[0007] According to another aspect of the embodiments of this application, an image processing method is also provided. The method includes: responding to an input command applied to an operation interface, displaying image data and query text data on the operation interface, wherein the image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language; inputting the image data and query text data into multiple image assessment models respectively, and using the multiple image assessment models to perform quality assessment of the image data from corresponding assessment dimensions to obtain multiple assessment results; integrating the multiple assessment results to obtain a quality assessment result of the image data; and displaying the quality assessment result on the operation interface.

[0008] According to another aspect of the embodiments of this application, an image processing apparatus is also provided, the apparatus comprising: an acquisition module for acquiring image data and query text data, wherein the image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language; an evaluation module for inputting the image data and query text data into multiple image evaluation models respectively, and using the multiple image evaluation models to perform quality assessment of the image data from corresponding evaluation dimensions to obtain multiple evaluation results; and an integration module for integrating the multiple evaluation results to obtain a quality assessment result of the image data.

[0009] According to another aspect of the embodiments of this application, a training apparatus for an image evaluation model is also provided. The apparatus includes: a first construction module for constructing multiple training images based on at least one original image; an annotation module for annotating the multiple training images respectively to obtain target evaluation results corresponding to the multiple training images; a second construction module for constructing training data based on the multiple training images and the target evaluation results; and a training module for training multiple initial evaluation models respectively using the training data according to different evaluation dimensions to obtain multiple image evaluation models, wherein the different image evaluation models are used to perform quality evaluation on image data and query text data from different evaluation dimensions to obtain evaluation results.

[0010] According to another aspect of the embodiments of this application, an image processing apparatus is also provided, the apparatus comprising: a first display module, configured to display image data and query text data on the operation interface in response to an input command applied to an operation interface, wherein the image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language; an evaluation module, configured to input the image data and query text data into multiple image evaluation models respectively, and use the multiple image evaluation models to perform quality assessment of the image data from corresponding evaluation dimensions to obtain multiple evaluation results; an integration module, configured to integrate the multiple evaluation results to obtain a quality assessment result of the image data; and a second display module, configured to display the quality assessment result on the operation interface.

[0011] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor connected to the memory via a bus for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0012] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0013] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0014] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods in various embodiments of this application.

[0015] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0016] In this embodiment, firstly, image data and query text data are acquired. The image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language. Next, the image data and query text data are input into multiple image assessment models, and each model assesses the image data from its corresponding assessment dimensions, resulting in multiple assessment results. Finally, the multiple assessment results are integrated to obtain the final image data quality assessment result. This application, by acquiring image data and query text data, helps to fully utilize the combined capabilities of multiple image assessment models in processing images and text. Guided by the query text data, the image assessment models can analyze the various assessment dimensions of image quality in greater detail. By using multiple image assessment models focused on different assessment dimensions, a more comprehensive quality assessment of the image data can be performed. Each image assessment model focuses on a specific assessment dimension, providing a professional assessment in that dimension. By integrating the scoring results from multiple dimensions, the resulting image data quality assessment result ensures that the quality assessment result more comprehensively reflects the image quality across different aesthetic dimensions, improving the comprehensiveness and accuracy of the assessment. This solves the technical problems of low accuracy and detail in image quality assessment in related technologies.

[0017] The above general description and the following detailed description are for illustrative and explanatory purposes only and do not constitute a limitation thereof. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram illustrating an application scenario of an image processing method according to an embodiment of this application; Figure 2 This is a flowchart of an image processing method according to an embodiment of this application; Figure 3 This is a flowchart of a training method for an image evaluation model according to an embodiment of this application; Figure 4 This is a schematic diagram of an optional image quality assessment process according to an embodiment of this application; Figure 5 This is a schematic diagram of an optional process for constructing multiple training images according to an embodiment of this application; Figure 6 This is a schematic diagram of an optional image quality assessment process according to an embodiment of this application; Figure 7This is a schematic diagram of an optional process for integrating multiple evaluation results according to an embodiment of this application; Figure 8 This is a flowchart of another image processing method according to an embodiment of this application; Figure 9 This is a schematic diagram of an image processing apparatus according to an embodiment of this application; Figure 10 This is a schematic diagram of a training device for an image evaluation model according to an embodiment of this application; Figure 11 This is a schematic diagram of another image processing apparatus according to an embodiment of this application; Figure 12 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some, not all, of the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort should fall within the scope of protection of the present application.

[0020] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in other orders. Wherein, "other orders" refers to orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that comprises a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed, or inherent to such processes, methods, products, or apparatus.

[0021] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0022] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows: Image Quality Assessment (IQA) is a technique used to measure the quality of an image, evaluating the degree of distortion or perceived quality. In the field of AI-generated content, where generation models are more complex and diverse, IQA is particularly important for quality control of these models and for improving user experience.

[0023] Multimodal models: These are artificial intelligence models that can process and fuse multiple modalities, such as images, text, audio, and video. They can extract information from different modalities and perform joint modeling for cross-modal understanding and generation tasks.

[0024] Mean Opinion Score (MOS): This is a subjective evaluation method for measuring the quality of images, audio, or video, reflecting the subjective perception of quality by human observers. It is calculated by having multiple observers rate the target content and then taking the average of the scores.

[0025] Supervised Fine-Tuning (SFT) is a technique that further improves a pre-trained model using labeled data. The model can be fine-tuned on top of large-scale pre-training using small-scale, task-related, labeled data to adapt to the needs of specific application scenarios.

[0026] According to an embodiment of this application, an image processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] The image processing method provided in the embodiments of this application can be applied to, for example, Figure 1 The application scenarios shown are not limited to these. In, for example... Figure 1 In the application scenario shown, server 10 can connect to one or more client devices 20 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Client devices 20 can interact with users through a graphical user interface to implement the methods provided in this application embodiment.

[0028] In this embodiment, the system consisting of a client device and a server can perform the following steps: The client device can interact with the server. The server can acquire image data and query text data; input the image data and query text data into multiple image evaluation models respectively, and use the multiple image evaluation models to evaluate the quality of the image data from the corresponding evaluation dimensions to obtain multiple evaluation results; integrate the multiple evaluation results to obtain the quality evaluation result of the image data.

[0029] It should be noted that with the rapid development of high-performance computing units, the methods provided in this application embodiment can also be applied to model-in-machine systems in other application scenarios. In one optional embodiment, the model-in-machine system has multiple built-in models. Users can select one model to adjust as needed to obtain their own model. The high-performance computing unit built into the model-in-machine system can then directly call the adjusted model to execute the methods provided in this application embodiment. In another optional embodiment, the model-in-machine system has a pre-trained model built-in. Therefore, the high-performance computing unit built into the model-in-machine system can directly call this model to execute the methods provided in this application embodiment.

[0030] Furthermore, when users need to train their own models, they can upload their own datasets via the client. These datasets are then sent to the server, allowing the server to adjust the pre-trained model using the dataset to obtain the user's customized model, which can then be deployed to the production environment. To facilitate users' model adjustment needs, the server provides complete adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies. This allows the adjusted model to better adapt to different application domains and achieve a high degree of customization.

[0031] Under the aforementioned operating environment, this application provides the following: Figure 2 The image processing method shown. Figure 2 This is a flowchart of an image processing method according to an embodiment of this application. For example... Figure 2 As shown, the method may include the following steps: Step S202: Obtain image data and query text data.

[0032] The image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language.

[0033] The image generation model mentioned above refers to an artificial intelligence model that can generate images based on input conditions or prompts. Image generation models can be used to create images that conform to specific scenarios, such as interior design and style, for subsequent quality assessment.

[0034] The image data mentioned above can refer to a specific image instance output by an image generation model. The image data can be a static image or part of a dynamic image sequence, which can be determined according to actual needs.

[0035] The aforementioned query text data can refer to natural language text data used to describe specific needs and standards for image quality assessment. As part of the input to the image assessment model, the query text data guides the model to understand assessment dimensions such as the image's aesthetic characteristics, layout rationality, and color harmony. For example, query text data could be used to assess the layout rationality of an indoor scene or to determine whether the image's color scheme is harmonious.

[0036] In one alternative embodiment, an image generation model can be used to generate image data covering different styles, layouts, and decorations based on the original image or specific prompts. Corresponding query text data can be written according to the needs of image quality assessment, such as layout, color matching, and style consistency. The query text data can indicate the dimensions and standards of assessment, providing clear guidance for the image assessment model and helping it understand the multidimensional attributes of image quality. The generated image data and corresponding query text data can also be integrated as input to the image assessment model. This allows the image assessment model to learn how to assess image quality in specific dimensions by understanding and parsing the relationship between images and text.

[0037] In the above process, acquiring image data and query text data helps to fully leverage the combined capabilities of multiple image evaluation models in processing images and text, improving the accuracy and robustness of these models in image quality assessment. Guided by the query text data, the image evaluation models can gain a more detailed understanding of the various dimensions of image quality, achieving a shift from coarse to fine evaluation, which helps identify subtle quality issues in images. The query text data can be adjusted according to different scenario requirements, enabling the image evaluation models to flexibly adapt to various image generation scenarios, including interior decoration, product display, and other scenarios.

[0038] Step S204: Input the image data and query text data into multiple image evaluation models respectively, and use the multiple image evaluation models to evaluate the quality of the image data from the corresponding evaluation dimensions to obtain multiple evaluation results.

[0039] The aforementioned multiple image evaluation models can refer to artificial intelligence models designed and trained to evaluate image quality. Different image evaluation models can evaluate image quality from different angles and standards. Multiple image evaluation models can include models that focus on multiple aesthetic dimensions such as image sharpness, color reproduction, layout rationality, and matching harmony. They can be trained using deep learning technology and can automatically identify and evaluate these aesthetic features in images.

[0040] The aforementioned evaluation dimensions can refer to specific aspects or standards for evaluating image quality. For example, in image aesthetic evaluation, evaluation dimensions could be: layout rationality, such as whether the placement and orientation of objects are reasonable; color coordination, such as whether the color scheme is harmonious and the style is consistent; and image richness, such as whether the number and variety of objects in the image are rich.

[0041] The aforementioned multiple evaluation results can refer to the evaluation results generated by multiple image evaluation models applied to the same image data, each focusing on its own evaluation dimension. For example, one image evaluation model can evaluate the layout rationality score of the image data, while another image evaluation model can evaluate the color matching score. Multiple evaluation results together constitute a more comprehensive assessment of image quality, helping to more accurately determine whether the image meets the expected aesthetic standards.

[0042] In one optional embodiment, a series of image evaluation models with different evaluation dimensions can be selected according to the image quality assessment requirements. For example, one image evaluation model can be used to evaluate the layout rationality of an image, and another image evaluation model can be used to evaluate the color matching of an image. These image evaluation models are pre-tuned and trained to adapt to specific evaluation dimensions. Specifically, image data and query text data can be input into each image evaluation model respectively. Each image evaluation model can perform image quality assessment from the corresponding evaluation dimension based on the input image data and query text data, and output multiple evaluation results. The evaluation results can be output in the form of scores or grades, such as layout rationality score: 4 / 5, color matching score: 3 / 5, etc., forming evaluation results with multiple evaluation dimensions.

[0043] In the process described above, by using multiple image evaluation models that focus on different evaluation dimensions, a more comprehensive quality assessment of image data can be performed, reflecting the overall quality of the image more fully. Each image evaluation model focuses on a specific dimension, providing professional evaluation in that dimension, which helps users develop more targeted image adjustment strategies. For example, if the layout rationality score is low, users can adjust the placement and angle of the goods in the image.

[0044] Step S206: Integrate multiple evaluation results to obtain the quality evaluation result of the image data.

[0045] The aforementioned quality assessment result refers to the comprehensive image quality assessment result obtained by integrating multiple assessment results during the image quality assessment process. The quality assessment result can be a numerical score, such as a rating between 1 and 5; a grade, such as excellent, good, average, or poor; or more detailed descriptive feedback. The quality assessment result can serve as a comprehensive evaluation of image quality and can be used to determine whether an image meets the aesthetic and quality standards of a specific scene.

[0046] In one optional embodiment, multiple image evaluation models can be received to evaluate image data across different evaluation dimensions. Examples include scores for layout, matching, and color harmony. The weight of each evaluation dimension in the overall score can be determined based on evaluation requirements. These weights reflect the importance of different aesthetic dimensions in a specific scenario; for example, in interior design, layout and matching might have a higher weight than color harmony. Furthermore, multiple evaluation results can be integrated based on the weights of each evaluation dimension in the overall score to obtain a quality evaluation result for the image data.

[0047] Integrating scoring results from multiple dimensions in the above process ensures that the quality assessment results can more comprehensively reflect the quality of images across different aesthetic dimensions, improving the comprehensiveness and accuracy of the assessment. The comprehensive scoring results can also serve as feedback signals, guiding the training and adjustment of the image generation model, forming a closed loop of quality improvement from image generation to image evaluation, and then to the improvement of the image generation model, continuously enhancing the quality level of generated images. In practical applications, such as studio image measurement or reinforcement learning training, high-precision quality assessment results can support user decision-making, helping to select high-quality images or improve image generation strategies, meeting user needs while enhancing the user experience.

[0048] In this embodiment, firstly, image data and query text data are acquired. The image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language. Next, the image data and query text data are input into multiple image assessment models, and each model assesses the image data from its corresponding assessment dimensions, resulting in multiple assessment results. Finally, the multiple assessment results are integrated to obtain the final image data quality assessment result. This application, by acquiring image data and query text data, helps to fully utilize the combined capabilities of multiple image assessment models in processing images and text. Guided by the query text data, the image assessment models can analyze the various assessment dimensions of image quality in greater detail. By using multiple image assessment models focused on different assessment dimensions, a more comprehensive quality assessment of the image data can be performed. Each image assessment model focuses on a specific assessment dimension, providing a professional assessment in that dimension. By integrating the scoring results from multiple dimensions, the resulting image data quality assessment result ensures that the quality assessment result more comprehensively reflects the image quality across different aesthetic dimensions, improving the comprehensiveness and accuracy of the assessment. This solves the technical problems of low accuracy and detail in image quality assessment in related technologies.

[0049] In the above embodiments of this application, the method further includes: determining the evaluation scene corresponding to the image data; determining multiple evaluation dimensions that match the evaluation scene; obtaining image evaluation models corresponding to the multiple evaluation dimensions respectively, thereby obtaining multiple image evaluation models, wherein the image evaluation models are trained according to the corresponding evaluation dimensions using multiple training images and the target evaluation results corresponding to the multiple training images respectively, the multiple training images are constructed based on at least one original image, and the target evaluation results are obtained by annotating the multiple training images respectively.

[0050] The aforementioned evaluation scenario can refer to a specific background or environment for image evaluation, such as an indoor home decoration scene or an outdoor landscape scene. The evaluation scenario can influence the focus and standards of image quality evaluation. For example, in an indoor home decoration scene, the evaluation can focus on dimensions such as the rationality of the layout of items, the harmony of color matching, and the unity of the overall style.

[0051] The aforementioned original image can refer to the image used as a base reference during image generation and evaluation. The original image can serve as the starting point for generating diverse and high-quality training images. Through analysis and processing of the original image, a series of training images that meet the needs of a specific evaluation scenario can be generated. For example, in an interior design evaluation scenario, the original image could be a real photograph of a room or a designer's initial sketch.

[0052] The training images mentioned above can refer to images generated based on the original images and used to train the image evaluation model. Training images can include variant images with different styles, layouts, and color schemes, enabling the image evaluation model to learn on diverse image data, thereby improving the accuracy and robustness of the image evaluation model in recognizing image quality in the evaluation scenario.

[0053] The aforementioned target evaluation results can refer to the results obtained by manually annotating training images, and can be used as the target standard for evaluating training image models. Target evaluation results may include, but are not limited to, layout rationality scores, color harmony scores, or overall aesthetic quality scores. Target evaluation results can be calculated based on the average opinions of multiple people to reduce the influence of individual preferences on the scores and improve the objectivity and consistency of the evaluation results.

[0054] In one optional embodiment, a specific evaluation scenario, such as an interior design scenario, can be determined. Then, based on the characteristics of the evaluation scenario, evaluation dimensions matching the scenario can be determined. For example, layout and matching can be typical evaluation dimensions in an interior design scenario. Next, based on the evaluation scenario and the original image, diverse training images can be generated. Each training image can be manually annotated, and the average opinion can be calculated as the target evaluation result, providing a quality benchmark for training the image evaluation model. Then, based on the determined evaluation dimensions, dedicated image evaluation models can be selected or trained respectively. Through supervised fine-tuning, the image evaluation models learn to evaluate image quality from the specified dimensions to output evaluation scores that match the manually annotated results.

[0055] In the above process, defining the evaluation scenario and matching evaluation dimensions allows the image evaluation model to focus on the aesthetic features of a specific scenario, improving the relevance and accuracy of the evaluation. For example, an image evaluation model based on layout dimension can more accurately capture the rationality of object placement, while an image evaluation model based on matching dimension can accurately assess the harmony of color and style. By using manually labeled target evaluation results as a training set and fine-tuning the image evaluation model, the model can learn the patterns of image quality evaluation from a large amount of data, improving its generalization ability and enabling it to make accurate quality assessments even when faced with new images.

[0056] In the above embodiments of this application, image data and query text data are respectively input into multiple image evaluation models, and the image data is evaluated for quality from corresponding evaluation dimensions using multiple image evaluation models to obtain multiple evaluation results. This includes: for any image evaluation model, inputting image data and query text data into the image evaluation model, evaluating the image data for quality from corresponding evaluation dimensions using the image evaluation model to obtain evaluation text data, wherein the evaluation text data is used to describe the quality of the image data in the corresponding evaluation dimension using natural language; and generating evaluation results based on the evaluation text data, wherein the evaluation results are used to characterize the quality quantification score of the image data in the corresponding evaluation dimension.

[0057] The aforementioned evaluation text data refers to textual information output by an image evaluation model, using natural language to describe the quality of an image across a specific evaluation dimension. It can serve as an expression of the image quality analysis results from the image evaluation model, such as a reasonable layout, appropriate furniture placement, no obvious spatial conflicts, or inconsistent color matching, clashes between primary and secondary colors, and poor visual effects. Evaluation text data provides a qualitative description of image quality and can also serve as the data basis for further quantification into evaluation results.

[0058] In one alternative embodiment, the image evaluation model can receive image data and query text data as input. Based on the image data and query text data, and using internally learned image aesthetics and quality standards, the image evaluation model can output evaluation text data in natural language. This evaluation text data can then be converted into a quantitative evaluation result. This process can employ predefined scoring rules or machine learning models to map descriptive text to scores or ratings on specific evaluation dimensions. For example, by training a classifier, text describing layout rationality can be automatically mapped to a 5-point rating, or a rule engine can be used to convert text describing color matching into corresponding scores. The output evaluation result can be a quantitative score of the image on each evaluation dimension. The evaluation result can be a number, such as a rating from 1 to 5. By integrating evaluation results from different dimensions, a comprehensive evaluation of the overall image quality can be formed.

[0059] In the above process, the evaluation text data is presented in natural language, which is easy to understand and interpret, providing users with intuitive quality feedback. Natural language descriptions are more readily accepted by non-technical users and also facilitate subsequent analysis and processing. Converting the evaluation text data into quantitative evaluation results provides objective quality assessment standards. This quantitative approach reduces the uncertainty of subjective judgment, making image quality assessment more standardized and suitable for automated quality assessment of large-scale images.

[0060] In the above embodiments of this application, the image evaluation model is used to evaluate the quality of image data from the corresponding evaluation dimensions to obtain evaluation text data, including: using the image evaluation model to extract features from the image data according to the corresponding evaluation dimensions to obtain target image features; and using the image evaluation model to evaluate the quality of the target image features to obtain evaluation text data.

[0061] The aforementioned target image features refer to the visual features extracted from the input image by the image evaluation model during the image evaluation process, which are related to a specific evaluation dimension. These features can include the image's layout structure, color distribution, object recognition, and texture details. For example, when evaluating the layout dimension, the image evaluation model can extract image features such as the position, orientation, and interrelationships of objects; when evaluating the matching dimension, the image evaluation model can extract image features such as color harmony and texture consistency.

[0062] In one optional embodiment, the image evaluation model can automatically identify and extract target image features related to a specific evaluation dimension from the input image data. Then, based on the extracted target image features, the image evaluation model can continue to perform image quality evaluation. The image evaluation model can analyze the relationships between image features and evaluate whether the image features meet the criteria of the evaluation dimension. The image evaluation model can output descriptive evaluation text data in natural language. For example, the image evaluation model can output evaluation text data such as reasonable layout, harmonious object placement, no visual conflict, or color matching conflict, disharmony between primary and secondary colors, and poor visual effect.

[0063] In the above process, by extracting target image features from image data, the image evaluation model can gain a deeper understanding of image content and structure, and identify visual elements closely related to the evaluation dimensions. Extracting target image features removes information irrelevant to the evaluation dimensions, focusing on more critical image attributes, thereby improving the accuracy and robustness of the evaluation. For example, focusing solely on layout structure without being affected by color ensures the accuracy of layout dimension evaluation.

[0064] In the above embodiments of this application, the evaluation text data includes: probabilities corresponding to multiple quality evaluation levels; generating evaluation results based on the evaluation text data includes: normalizing the probabilities corresponding to the quality evaluation levels to obtain normalized probabilities corresponding to the quality evaluation levels; and weighting the normalized probabilities corresponding to multiple quality evaluation levels to obtain the evaluation results.

[0065] The aforementioned quality assessment levels refer to several grades obtained by dividing image quality into levels based on its quality during image quality assessment. These levels can be used to visually represent the relative quality of an image. Quality assessment levels can be descriptive labels such as Excellent, Good, Average, and Poor.

[0066] In one optional embodiment, the evaluation text data may include probabilities assigned to each quality assessment level. To ensure the comparability of probabilities output by different image evaluation models, probability normalization can be performed. This is done by converting the probabilities corresponding to each quality assessment level into normalized probabilities, such that the sum of the probabilities corresponding to each quality assessment level is 1. Next, the weights of each quality assessment level can be determined based on the evaluation scenario and requirements. Then, the normalized probabilities of each quality assessment level are weighted and summed with their corresponding weights to obtain the evaluation result. For example, in a studio shooting scenario, if the weight corresponding to an excellent quality assessment level is higher, then in the weighted summation, the normalized probability of an excellent quality assessment level can be assigned more weight.

[0067] In the above process, determining the probabilities of multiple quality assessment levels output by the image evaluation model can more accurately reflect the true distribution of image quality. Adjusting the weights of the quality assessment levels according to specific application needs allows for flexible customization of the scoring criteria to adapt to different image quality requirements in different scenarios. For example, in some scenarios, more emphasis can be placed on the "Excellent" and "Good" levels, while in other scenarios, more attention can be paid to the "Average" and "Poor" levels.

[0068] In the above embodiments of this application, integrating multiple evaluation results to obtain a quality evaluation result of image data includes: normalizing the multiple evaluation results to obtain multiple normalized results, wherein the multiple normalized results conform to a normal distribution; determining the target weight value corresponding to the multiple normalized results; weighting the multiple normalized results based on the target weight value to obtain a weighted result; and mapping the weighted result to a preset range to obtain a quality evaluation result.

[0069] The aforementioned target weight values ​​refer to the weight values ​​determined for each evaluation dimension in a multi-dimensional image quality assessment. These target weight values ​​indicate the importance of that evaluation dimension to the overall image quality. They can reflect the relative importance of different evaluation dimensions within an application scenario or user preference, and thus influence the calculation of the image quality assessment results. For example, in interior design scenarios, the target weight values ​​for layout and color matching can be higher than those for color saturation or sharpness.

[0070] The aforementioned preset range can refer to a pre-defined specific numerical range within which the weighted results need to be mapped or restricted. Setting a preset range helps to standardize and normalize the quality assessment results, making them easier to quantify and compare with other assessment standards. For example, restricting the image quality assessment results to a five-point scale of [1, 5] makes it easier for users to intuitively judge the quality of the image.

[0071] In one optional embodiment, multiple evaluation results across different evaluation dimensions can be normalized. Multiple normalized results ensure that the scores for each evaluation dimension are on the same scale, facilitating subsequent weight adjustment and weighted summation. Normalization converts each evaluation result into a standard normal distribution with a mean of 0 and a standard deviation of 1, eliminating differences in the scoring base and scale between different dimensions. Next, target weight values ​​for each evaluation dimension can be determined according to application requirements, considering the impact of each evaluation dimension on the overall image quality and user experience. Then, the determined target weight values ​​can be used to perform a weighted summation of the multiple normalized results to obtain a weighted result. This process ensures that more important dimensions are reflected in the scoring, while the impact of secondary dimensions is relatively reduced. Finally, the weighted result can be mapped to a preset range, such as a five-point scale of [1, 5], as the image quality evaluation result. This mapping process quantifies and unifies the quality evaluation result, making it more intuitive and easier to understand.

[0072] In the above process, through normalization and mapping to a preset range, the quality assessment results are unified in scale, facilitating management and application. Standardized quality assessment results are easier for users and other systems to understand and use. Setting target weight values ​​provides the ability to adjust the importance of different assessment dimensions, enabling image quality assessment results to better reflect the quality requirements of specific scenarios. For example, in e-commerce product images, users value the quality of layout and arrangement more, while their requirements for color saturation are relatively lower.

[0073] In the above embodiments of this application, determining the target weight values ​​corresponding to multiple normalization results includes: determining the target evaluation dimension corresponding to the normalization result; obtaining the preset weight value corresponding to the target evaluation dimension to obtain the target weight value, wherein the preset weight value is determined during the training process of multiple image evaluation models.

[0074] The aforementioned target evaluation dimensions can refer to the aesthetic or functional evaluation dimensions that need to be considered during the image quality evaluation process. The selection of target evaluation dimensions can be based on the application scenario and needs of the image. For example, in an interior design scenario, layout and matching can be used as target evaluation dimensions.

[0075] In one alternative embodiment, the target evaluation dimensions to be evaluated can be determined based on the specific needs of image evaluation. For example, in an interior design scenario, the rationality of the layout, the harmony of the matching, and the coordination of colors can all be target evaluation dimensions.

[0076] During the training phase of an image evaluation model, the importance of each evaluation dimension to the overall image quality can be determined by analyzing a large amount of labeled image data, thereby establishing the pre-defined weight values ​​for each evaluation dimension. These pre-defined weight values ​​can reflect the statistical distribution of the evaluation dimensions in the training data or be manually set based on expert knowledge and application requirements. These pre-defined weight values ​​can be learned and memorized by the image evaluation model during the training process and used for subsequent image quality assessments.

[0077] In the above process, by determining the target evaluation dimensions corresponding to the normalized results, image quality assessment can focus more on important evaluation dimensions, ensuring the relevance and effectiveness of the assessment. For example, in indoor scenes, focusing more on layout and matching can avoid the quality assessment from overemphasizing other dimensions such as color and sharpness, which could lead to evaluation results deviating from application requirements.

[0078] According to an embodiment of this application, a training method for an image evaluation model is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order not used here.

[0079] Figure 3 This is a flowchart of a training method for an image evaluation model according to an embodiment of this application, such as... Figure 3 As shown, the specific steps may include the following: Step S302: Construct multiple training images based on at least one original image.

[0080] Step S304: Label the multiple training images respectively to obtain the target evaluation results corresponding to the multiple training images.

[0081] Step S306: Construct training data based on multiple training images and target evaluation results.

[0082] Step S308: Train multiple initial evaluation models separately using training data according to different evaluation dimensions to obtain multiple image evaluation models.

[0083] Different image evaluation models are used to evaluate the quality of image data and query text data from different evaluation dimensions to obtain evaluation results.

[0084] In one optional embodiment, based on at least one original image, similar images can be filtered and selected. Different prompt words can be generated from the remaining images. Furthermore, a variety of diverse prompt words can be generated. Combining the two types of prompt words from the original images, along with multiple image generation models, multiple images with different aesthetic styles and layout structures are generated to form training images. This step ensures the diversity and richness of the training data, helping the image evaluation model learn a wide range of quality standards and aesthetic perceptions. For each generated training image, a multi-level scoring system can be used for multiple annotations to obtain multiple initial annotation results. By calculating the average of these annotation results, the target evaluation result for each training image can be obtained, representing aspects such as image layout, arrangement, and overall quality. Simultaneously, correlation processing and normalization operations can be performed to remove outliers and subjective biases, further improving the quality of the annotated data. Next, training data can be constructed based on the training images and their corresponding target evaluation results. The target annotation results of the training images can contain information from multiple evaluation dimensions, constituting the labels for the training data. Then, the constructed training data can be used to fine-tune the training of multiple initial evaluation models, training different evaluation dimensions of each initial evaluation model separately. During training, the initial evaluation model learns the correlation between image features and evaluation results, gradually mastering the ability to extract image features from image data and convert these features into quality assessment scores. By combining image and natural language processing, the initial evaluation model enhances its comprehensive understanding of image quality. A well-trained image evaluation model can receive image data and query text data as input, outputting evaluation results for the corresponding evaluation dimensions. This includes the image evaluation model's deeper and more detailed understanding of the image, as well as accurate responses to natural language queries, thereby achieving higher-precision image quality evaluation.

[0085] In the above process, by constructing diverse training images, the initial evaluation model can learn quality standards under different scenarios, improve evaluation accuracy on unseen images, and enhance the generalization ability of the initial evaluation model. Training multiple image evaluation models enables overall image quality assessment, and also allows for refined scoring based on specific aesthetic evaluation dimensions such as layout and composition, meeting the multi-faceted and multi-layered needs of image quality in different application scenarios. By constructing diverse training images, performing image quality annotation, building training data, and training multiple image evaluation models, a systematic and refined method for training image evaluation models is provided. This significantly improves the generalization ability and evaluation accuracy of image evaluation models, providing strong technical support for image quality assessment.

[0086] In the above embodiments of this application, constructing multiple training images based on at least one original image includes: performing prompt word recognition on at least one original image to obtain at least one original prompt word; expanding the at least one original prompt word to obtain multiple target prompt words; and generating multiple training images based on the multiple target prompt words.

[0087] The aforementioned original prompt words can refer to the original prompt words obtained by performing prompt word recognition on the original image, which are used to describe the image's theme, style, layout, or other features.

[0088] The aforementioned target cue words can refer to cue words obtained by expanding and transforming the original cue words. They can be used to guide image generation models to produce images that meet more diverse needs. Target cue words can include expanded specific layout requirements, color matching suggestions, or style descriptions, enriching the guidance information for image generation and enhancing the diversity and creativity of the generated images.

[0089] In one optional embodiment, at least one original image can be analyzed, and machine learning algorithms can be used to identify and extract semantic features to obtain original prompt words. Then, the identified original prompt words can be input into a prompt word generation model for expansion and transformation to generate multiple target prompt words. This may include operations such as synonym replacement, context-related vocabulary addition, and style shifting, aiming to create richer and more diverse generation guidance information, covering more aesthetic dimensions and scenario requirements. Next, the expanded multiple target prompt words can be combined with an image generation model to generate multiple training images. These training images retain the important features of the original images and incorporate new elements or styles described in the target prompt words, increasing the diversity and creativity of the training data and providing more comprehensive samples for training the image evaluation model.

[0090] In the process described above, generating and applying original and target prompts produces multiple training images. This helps to create more diverse and creative images while preserving the important features of the original images. The diverse training images enable the image generation model to learn broader aesthetic standards and quality judgment criteria. Covering more evaluation scenarios with training images allows the image evaluation model to encounter various image quality evaluation conditions during training, thereby improving the accuracy of the image evaluation model in actual evaluations.

[0091] In the above embodiments of this application, expanding at least one original prompt word to obtain multiple target prompt words includes: inputting at least one original prompt word into a prompt word generation model, expanding the original prompt word using the prompt word generation model to obtain multiple expanded prompt words; and obtaining multiple target prompt words based on at least one original prompt word and multiple expanded prompt words.

[0092] The aforementioned cue word generation model refers to a model designed to generate diverse, creative, or style-specific extended cue words from a given original cue word. Based on natural language processing technology, it can understand the semantics of the original cue word and generate a series of extended cue words according to preset styles, contexts, or creative requirements. These extended cue words can contain more specific information, different perspectives, or innovative expressions, helping to guide image generation models to create more diverse images that meet the needs of specific scenarios.

[0093] In one optional embodiment, the original cue words can be derived from the analysis of the original image, containing important information such as the image's theme, style, color, or layout. At least one original cue word can be used as input to a cue word generation model for processing. The model can leverage its language understanding and generation capabilities to expand the original cue words, generating a series of expanded cue words. These expanded cue words can include additional layout requirements, color matching suggestions, style transition instructions, etc., aiming to create diverse image generation guidance information. Then, the original cue words can be merged or combined with the generated expanded cue words to obtain multiple target cue words. These target cue words retain the important features of the original image and incorporate new stylistic elements or aesthetic requirements, providing richer creative guidance for the image generation model.

[0094] In the process described above, the generation of target prompts guides the image generation model to produce more diverse and creative images, covering various aesthetic styles, layouts, and color schemes. This helps to construct diverse training data, providing rich examples for training the image evaluation model. The construction of diverse training images allows the image quality evaluation model to encounter various image quality assessment situations during training, thus improving its ability to accurately evaluate images with different styles, layouts, and color schemes.

[0095] In the above embodiments of this application, generating multiple training images based on multiple target prompt words includes: for any target prompt word, inputting the target prompt word into at least one image generation model, and using multiple image generation models to generate at least one generated image corresponding to the target prompt word; summarizing the at least one generated image corresponding to the multiple target prompt words to obtain multiple training images.

[0096] In one alternative embodiment, each target cue word can serve as a creative extension of the original image features and requirements, and may include detailed descriptions of aspects such as image subject, style, layout, and color. The target cue words can be input into at least one selected image generation model, which can then generate corresponding image content based on the guidance of the target cue words, ensuring that the generated images meet the requirements of the target cue words. To increase the diversity and richness of the images, multiple image generation models can be used, each generating at least one image based on the target cue words. Thus, based on the target cue words, the generated images will vary due to the characteristics and algorithmic differences of different image generation models, providing more diverse material for subsequent quality assessment. Subsequently, the generated images obtained using different target cue words can be collected and summarized to form multiple training images containing various image styles, layouts, and color combinations.

[0097] In the above process, using multiple image generation models to obtain multiple training images ensures the diversity of training images, covering different generation styles and image characteristics. This helps the image evaluation model to be exposed to and learn a wider range of image quality judgment criteria during training, thus improving the generalization ability of the image evaluation model.

[0098] In the above embodiments of this application, multiple training images are labeled to obtain target evaluation results corresponding to the multiple training images, including: for any training image, the training image is labeled multiple times to obtain multiple target labeling results of the training image; the average value of the multiple target labeling results is obtained to obtain the target evaluation result of the training image.

[0099] In one optional embodiment, for any training image, it can be manually annotated multiple times. Each annotation can be approached from different angles, such as layout, arrangement, color, and texture, ensuring the comprehensiveness and depth of the evaluation. Then, the annotation results from these multiple annotations can be collected to obtain multiple target annotation results. The arithmetic average of these multiple target annotation results can then be calculated to obtain the target evaluation result. Calculating the average helps eliminate subjective biases that may occur with individual manual annotations, making the manual evaluation result more objective, stable, and closer to the true quality level of the image.

[0100] In the above process, the training images are labeled multiple times and the average value is calculated as the target evaluation result. This can effectively reduce the impact of human subjective judgment on image quality evaluation and make the evaluation results more objective and reliable.

[0101] In the above embodiments of this application, the training image is annotated multiple times to obtain multiple target annotation results of the training image, including: annotating the training image multiple times to obtain multiple initial annotation results of the training image; performing correlation processing on the multiple initial annotation results to obtain correlation coefficients of the initial annotation results; performing normalization processing on the multiple initial annotation results to obtain multiple normalized annotation results; and filtering the multiple initial annotation results based on the correlation coefficients and the multiple normalized annotation results to obtain multiple target annotation results.

[0102] The initial annotation results mentioned above can refer to the original scores or descriptions obtained after manually annotating and evaluating the training images.

[0103] The correlation coefficient mentioned above can refer to a coefficient used to measure the degree of linear correlation between multiple initial annotation results. In scenarios where training images are annotated multiple times, the correlation between different initial annotation results can be calculated to help identify and eliminate those initial annotation results with large deviations, thereby improving the reliability and consistency of the annotation results.

[0104] In one optional embodiment, the training image can be annotated multiple times to obtain multiple initial annotation results. Then, correlation analysis is performed on the collected initial annotation results to calculate the correlation between different initial annotation results. The correlation coefficient reflects the consistency of the quality assessment of the training image by different initial annotation results, helping to identify abnormal annotation results with scores that are too high or too low. After obtaining the initial annotation results, normalization processing can be performed to distribute the scoring data within a common range, such as [0,1] or [1,5]. This helps to eliminate data inconsistency caused by differences in human evaluation scales. Next, based on the calculated correlation coefficient and the normalized annotation results, the multiple initial annotation results can be screened to remove those with poor consistency and large scoring deviations, ensuring that the obtained multiple target annotation results are more reliable and objective in quality, reducing the impact of subjective bias on the overall evaluation results.

[0105] In the above process, correlation analysis and outlier screening can effectively remove abnormal annotation results caused by human bias or misjudgment, making the target annotation results closer to the true quality level of the image and reducing subjective bias. Normalization ensures the numerical comparability of the annotation results, and screening further improves the quality of the annotation results, which is conducive to improving the efficiency of image evaluation model training and the accuracy of evaluation.

[0106] In the above embodiments of this application, the method further includes: for any training image, inputting the training image and query text data into multiple image evaluation models respectively, and using the multiple image evaluation models to evaluate the quality of the image data from the corresponding evaluation dimensions to obtain multiple training evaluation results; normalizing the multiple training evaluation results to obtain multiple training normalization results, wherein the multiple training normalization results conform to a normal distribution; and adjusting the initial weight values ​​of different evaluation dimensions based on the multiple training normalization results and the target evaluation results to obtain preset weight values ​​for different evaluation dimensions.

[0107] In one optional embodiment, any training image and query text data can be input into multiple image evaluation models designed for different evaluation dimensions. Each image evaluation model can evaluate the image quality based on its own training experience and evaluation criteria, outputting multiple training evaluation results. Then, these multiple training evaluation results can be normalized to a form conforming to a normal distribution, facilitating comparison and fusion of scores from different evaluation dimensions on the same scale and avoiding evaluation bias caused by different scoring scales. Next, based on the normalized training results and the target evaluation results, the initial weight values ​​of different evaluation dimensions can be adjusted by improving the algorithm to find preset weight values ​​for different evaluation dimensions. This may include minimizing the loss function, such as using a loss function to adjust the weight allocation, ensuring that the image evaluation model's evaluation across multiple dimensions is more balanced and accurate.

[0108] In the process described above, adjusting the initial weight values ​​helps the image evaluation model more accurately reflect the importance of each evaluation dimension to image quality, thus improving the overall accuracy of the evaluation and enabling the model to make more detailed and comprehensive judgments on image quality. In reinforcement learning scenarios, the multi-dimensional quality evaluation results after weight adjustment can serve as reward signals, guiding the image generation model to improve itself, enhancing the aesthetic characteristics of the generated images, such as layout and color matching, and increasing the visual appeal and application value of the images.

[0109] In the above embodiments of this application, the initial weight values ​​of different evaluation dimensions are adjusted based on multiple training normalization results and target evaluation results to obtain preset weight values ​​for different evaluation dimensions, including: obtaining loss function values ​​based on multiple training normalization results, initial weight values ​​of different evaluation dimensions and target evaluation results; and adjusting the initial weight values ​​of different evaluation dimensions based on the loss function values ​​to obtain preset weight values ​​for different evaluation dimensions.

[0110] The loss function value mentioned above can refer to the difference between the training normalized result given by the image evaluation model from different evaluation dimensions and the target evaluation result. By minimizing the loss function value, the initial weight values ​​of different evaluation dimensions can be improved, making the evaluation of the image evaluation model more accurate and closer to real needs.

[0111] In one optional embodiment, initial weight values ​​can be set for each evaluation dimension before training the image evaluation model. Next, the differences between multiple training normalization results and the initial weight values ​​of different evaluation dimensions can be compared to calculate the loss function value. Specifically, a suitable model loss function can be used to measure the relative advantage between different evaluation results, incorporating multiple training normalization results, the initial weight values ​​of different evaluation dimensions, and the target evaluation result into the loss function calculation. Using the model loss function, the difference between the training normalization results and the target evaluation result is calculated to obtain the loss function value. Then, based on the calculated loss function value, the initial weight values ​​of different evaluation dimensions can be adjusted. The adjustment process can be based on gradient descent, updating the weights through backpropagation by calculating the gradient of the loss function relative to the weights. This process—model prediction, loss comparison, and weight adjustment—can be repeated until the loss function value converges to a small value or meets a predetermined stopping condition. This helps the image evaluation model learn a better combination of dimension weights, making the evaluation results closer to the quality of real images.

[0112] In the above process, by adjusting the initial weights of the evaluation dimensions, the image evaluation model can highlight certain important evaluation dimensions in multi-dimensional evaluation. For example, in interior decoration scenarios, if layout is more important, the weight of the layout dimension can be adjusted to be higher, thus making the image evaluation model focus more on the evaluation of layout and improving the relevance and effectiveness of the evaluation. Different application scenarios may have different emphases on different evaluation dimensions of image quality. By adjusting the weights guided by the loss function value, the image evaluation model can quickly adapt to new evaluation requirements, improving the application adaptability and flexibility of the image evaluation model.

[0113] The technical solution proposed in this application is described below with reference to an optional embodiment. This application proposes a multi-dimensional quality assessment method based on image aesthetics and a multi-modal model. Based on image aesthetics and a multi-modal model, it provides an accurate image quality assessment method, solving the technical challenge of judging the quality of generated images in interior decoration scenarios. In practical applications, it can be used as a reward model for reinforcement learning to improve the image generation effect in actual studio shooting applications; it can also filter high-quality images for actual mapping applications to meet users' needs for the placement and use of interior decoration images. This application constructs a high-quality data baseline, combines simulation software and clustering algorithms for filtering, uses multiple image generation models to create diverse artificial intelligence image datasets, and achieves high-precision scoring and outlier cleanup. Based on the improvement of the multi-modal model, it utilizes the capabilities of the multi-modal model to fuse image and language input, achieving a high-precision quality score. The multi-dimensional aesthetic score is improved, breaking through the limitations of single-dimensional evaluation, and outputting a multi-dimensional aesthetic score through normalization and loss function calculation.

[0114] This application can be applied to practical applications, specifically for interior decoration scenarios, to conduct multi-dimensional image quality assessment based on image aesthetics and multimodal models. It can be applied to AI-powered scene image generation tools designed for the home furnishing and e-commerce industries, which offer various functions such as AI studio shooting, AI models, and AI scene replacement. This application focuses on the AI ​​studio shooting function, helping users efficiently and intelligently generate high-quality aesthetic scene images that match the characteristics of the product by uploading a product image, such as a white background image. These scene images are widely applicable to e-commerce main images, product detail pages, and social media promotions. In the studio shooting application, the image generation model is trained through reinforcement learning, and an image evaluation model can be used as a reward model to provide feedback signals to improve the generation results. In the image testing application, the image generation model can use the image evaluation model to filter the generated AI images, selecting high-quality images for users to test and use, ensuring that the image quality meets application requirements. In the studio shooting image testing application, high-quality AI-generated images are filtered based on a self-built quality assessment capability. The evaluation process can assess quality from the following dimensions: layout quality assessment based on an image evaluation model; combination quality assessment based on an image evaluation model; and image richness assessment based on a suitable image detection model. By comprehensively scoring and ranking multiple generated images, high-quality AI images are selected to meet application requirements.

[0115] This application proposes a multi-dimensional quality assessment method based on image aesthetics and a multimodal model. Taking interior decoration scenarios as an example, image quality can be assessed using two typical aesthetic dimensions: layout and matching. The specific definitions of each dimension are as follows: Layout: The placement of items should be reasonable, avoiding unconventional placements such as floating or reversed orientations, ensuring compliance with spatial logic; items should be placed in a reasonable space according to their design functions, following design common sense, and reflecting the professionalism of the layout; the number of accessories should be moderate, avoiding too many, too few, or homogeneous accessories that affect the overall aesthetics. Matching: The overall design style should be unified, avoiding obvious disharmony; it should conform to current popular interior aesthetics, avoiding overly old-fashioned or aesthetically unappealing soft and hard furnishing designs; color matching should be harmonious and consistent with the design style.

[0116] This application constructs a diverse and high-quality image quality assessment data baseline and designs a high-precision image quality assessment algorithm chain based on a multimodal model. Through multi-dimensional combined evaluation, it effectively improves the calculation of aesthetic scores, enhancing the accuracy and practicality of image quality assessment. The application includes: constructing a diverse and high-quality image quality assessment data baseline; filtering highly similar original images using simulation software and clustering algorithms to deduce corresponding prompt words; generating diverse prompt words using a prompt word generation model, and acquiring two types of prompt words from the original images; combining multiple image generation models to generate various types of AI images; allowing for multi-user annotation of each image; employing a five-level rating system to calculate the average subjective score as the image quality score to reduce subjective bias; eliminating abnormal annotators using correlation coefficients and cleaning outliers using normalization to further improve data quality; and fine-tuning the multimodal model based on the high-precision image quality assessment algorithm chain using image quality assessment data. The input to the multimodal model can include image data and natural language query text data, with the output being the corresponding evaluation text data. The image evaluation result is obtained by normalizing the probability of quality levels and performing a weighted summation, which can serve as a direct quantitative indicator for evaluating image quality. A multi-dimensional combined improved aesthetic score evaluation is implemented. For the aesthetic dimension scores output by different image evaluation models, normalization is used to eliminate biases caused by differences in data distribution. The combined weights of each dimension score are improved through the model loss function, and a weighted total score is calculated. Finally, normalization can be used to map the total score to a five-point quality score within the range of [1,5], serving as a comprehensive aesthetic evaluation result for image quality.

[0117] This application achieves high-precision image quality assessment through the following steps: constructing a diverse and high-quality image quality assessment data baseline; generating various types of AI images using simulation software, clustering algorithms, and image generation models; calculating average subjective scores through multi-person annotation; and improving data quality through correlation coefficients and normalization methods; fine-tuning a multimodal model based on the image quality assessment data; inputting images and natural language questions; outputting quality descriptions; and calculating image quality scores through normalization and weighting; integrating scores from different aesthetic dimensions; improving weights through normalization and loss functions; and normalizing the weighted sum to a five-point range of [1,5] to generate a comprehensive aesthetic quality assessment result.

[0118] Figure 4 This is a schematic diagram of an optional image quality assessment process according to an embodiment of this application, such as... Figure 4 As shown, diverse training data is constructed to train multiple image evaluation models. User image data is acquired and input into multiple image evaluation models to perform quality evaluation on the image data, resulting in evaluation text data. Based on the evaluation text data, evaluation results are generated. Multiple evaluation results are integrated to obtain and output the final quality evaluation result.

[0119] Construct a diverse and high-quality image quality assessment data baseline. Data production involves filtering highly similar original images using simulation software and clustering algorithms to deduce corresponding prompts. A diverse range of prompts are generated using a prompt model, along with two prompts derived from the original images. These are then combined with various mainstream image generation models to generate multiple types of AI-generated images. Data annotation involves multiple annotating teams for each image, employing a five-level rating system to calculate the average subjective score as the image's quality score, thus reducing subjective bias. Annotation content is divided into two aesthetic dimensions: layout and arrangement. Layout focuses on the rationality and appropriate quantity of items, while arrangement focuses on the overall style consistency, color coordination, and adherence to design aesthetics. Average subjective scores are calculated for each dimension. Data cleaning follows annotation, using correlation coefficients to remove anomalies and normalizing outliers to further improve data quality.

[0120] Figure 5 This is a schematic diagram illustrating an optional process for constructing multiple training images according to an embodiment of this application, such as... Figure 5 As shown, the original image is subjected to cue word recognition to obtain the original cue words; the original cue words are input into the cue word generation model to expand the cue words, resulting in multiple expanded cue words; based on the original cue words and multiple expanded cue words, multiple target cue words can be obtained; the multiple target cue words are input into the image generation model to generate images; the generated images are then summarized to obtain multiple training images.

[0121] A high-precision image quality assessment algorithm based on a multimodal model is implemented. The process includes: Model training, fine-tuning the multimodal model using image quality assessment data, evaluating the effects of fine-tuning and full-parameter fine-tuning respectively, and selecting the superior model; Model inference, where the image assessment model input consists of image data and natural language quality questions, and the output is the corresponding quality description; Calculating the aesthetic score, where a quality score is generated based on the probability of the output quality description in the image assessment model inference. Specifically, the probabilities corresponding to multiple quality assessment levels can be normalized to a sum of 1, and a weighted sum is performed to calculate the assessment result. This assessment result reflects the level of image quality, and combined with model fine-tuning, achieves a more accurate quantitative assessment of quality.

[0122] Figure 6 This is a schematic diagram of an optional image quality assessment process according to an embodiment of this application, such as... Figure 6 As shown, training data is constructed and fine-tuned to obtain multiple image evaluation models; image data and query text data are input into multiple image evaluation models; quality evaluation is performed on the image data to obtain evaluation text data; the probabilities corresponding to the quality evaluation levels contained in the text data are normalized and then weighted to obtain multiple evaluation results; the multiple evaluation results are integrated to obtain the quality evaluation result.

[0123] Aesthetic score assessment improved by combining multiple dimensions.

[0124] The loss function, which calculates optimal weights through likelihood estimation, can be expressed as follows: ; For multi-dimensional weighted summation, it can be expressed as follows: ; Figure 7 This is a schematic diagram of an optional process for integrating multiple evaluation results according to an embodiment of this application, such as... Figure 7 As shown, the evaluation results of the matching dimension are normalized to obtain the normalized result of the matching dimension; the evaluation results of the layout dimension are normalized to obtain the normalized result of the layout dimension; the multiple normalized results are weighted based on the target weight value to obtain the weighted result; the weighted result is mapped to a preset range to obtain the quality evaluation result.

[0125] For the multiple aesthetic dimension scores output by different image quality assessment models, the following steps can be used to integrate them and generate the overall image quality score. Normalization, specifically standard normal distribution normalization, transforms the data into a distribution with a mean of 0 and a standard deviation of 1. In this application, normalization is performed based on the mean and variance of the test set data to eliminate bias caused by differences in data distribution across different aesthetic dimensions. Loss function weight improvement: The loss function is used to improve and estimate the rating or intensity parameters in the probability model. By minimizing the loss function and combining it with likelihood estimation, the weights of different dimension scores are improved to obtain a better combination for calculating the overall image quality score. Normalization maps the overall quality score to the range [1,5]. Based on the upper and lower limits of the test set data, the normalized five-point score better reflects the user's intuitive perception of image quality and serves as the quality assessment result. This application constructs a diverse and high-quality image quality assessment data baseline and designs a high-precision image quality assessment algorithm chain based on a multimodal model. Through multi-dimensional combined assessment, the calculation of aesthetic scores is effectively improved, enhancing the accuracy and practicality of image quality assessment.

[0126] This application proposes an effective solution through innovative end-to-end design, from data construction and algorithm improvement to multi-dimensional scoring strategies, specifically as follows: Improving scoring accuracy and refining aesthetic sub-dimensional assessment: A high-quality image quality assessment data baseline is established. Similar images are selected by combining simulation software and clustering algorithms, and diverse prompts are integrated with the generated image generation model to generate rich and diverse image data. This is further supplemented by multi-person, five-level scoring and outlier cleanup to create a high-precision, high-reliability image quality assessment dataset, improving model learning ability and assessment accuracy from the fundamental data level. Based on multimodal model improvements, the strong perceptual capabilities of multimodal models are utilized to combine images with natural language, fine-tuning the image quality assessment task. Through normalization and weighting strategies, accurate quality scores are output, achieving more refined and robust perceptual quality labels. A multi-dimensional quality assessment mechanism is introduced, eliminating data bias through normalization and improving dimensional weight combinations using a loss function, effectively supporting refined analysis and improvement of different aesthetic sub-dimensions. To reduce reliance on reference images and improve generalization, this application leverages the cross-modal extension capabilities of multimodal models, combining text prompts with generated images to enhance the image evaluation model's understanding of diverse scenarios. This eliminates the need for strong reliance on reference images, significantly improving the method's generalization ability in reference-less scenarios. Through an independent multi-dimensional quality assessment strategy, aesthetic quality is quantified into interpretable sub-dimensional scores, such as color, composition, and style, enabling fine-grained evaluation and improvement in reference-less scenarios and enhancing the image evaluation model's adaptability to diverse image generation tasks. A unified and intuitive comprehensive score output, employing a combination of normalization and multi-dimensional improvement, maps the quality assessment results to a score range of [1,5], providing a unified and intuitive scoring system that meets the practical needs of multiple scenarios while improving the applicability and interpretability of the solution.

[0127] Through the aforementioned improvements, this application achieves at least the following technical effects: High-precision evaluation capability: In aesthetic image quality assessment tasks, it improves the overall scoring accuracy of the image assessment model and enables refined evaluation of different aesthetic sub-dimensions, providing support for multi-dimensional analysis in complex scenarios. Enhanced generalization capability: Overcoming the strong dependence on reference images, it improves the cross-domain adaptability of the method across various scenarios and models, providing efficient and reliable quality evaluation even in scenarios without reference images. Unified scoring system: Through intuitive comprehensive score output, it facilitates user understanding and practical application, setting a new benchmark for image quality assessment in multiple fields such as interior decoration scenarios. Overall, this application enhances the technical effectiveness and application value of aesthetic image assessment in practical applications.

[0128] This application proposes an innovative and systematic aesthetic image quality assessment framework, integrating a complete process from high-quality data construction to multimodal model improvement. It combines multi-dimensional quality assessment improvement strategies to comprehensively enhance the accuracy and practical application value of aesthetic-based image quality assessment. A diverse and high-quality image quality assessment data baseline is established. Similar images are selected by combining simulation software and clustering algorithms, and diverse prompts are generated and integrated with various mainstream image generation models to generate diverse AI images. A multi-person, five-level scoring system is adopted, combined with correlation coefficients to remove outlier annotations and normalization to clean up outliers, creating an accurate and high-quality image quality assessment dataset, laying a solid foundation for image assessment model training. A high-precision image quality assessment algorithm based on a multimodal model leverages the strong scalability of the multimodal model to fine-tune and improve the image quality assessment task. By combining images and natural language questions, and incorporating normalization and weighting strategies, the quality score is accurately quantified, providing highly robust and accurate perceptual quality labels for complex images, overcoming key technical challenges in aesthetic assessment tasks. This invention improves aesthetic scores through multi-dimensional combinations, employs normalization to eliminate cross-dimensional data bias, and utilizes a loss function to refine the dimensional weight combination. Normalization then maps the scores to an intuitive range of [1,5], outputting a unified and quantifiable comprehensive aesthetic quality assessment result. This provides high-value application support for image quality evaluation in various scenarios. Through end-to-end innovation from data to algorithms, this application achieves multi-dimensional and multi-level precision breakthroughs in the field of aesthetic image quality assessment, setting a new benchmark for next-generation aesthetic perception and quality assessment technologies, and demonstrating broad practical value and commercial prospects.

[0129] According to an embodiment of this application, an image processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order not used here.

[0130] Figure 8 This is a flowchart of an image processing method according to an embodiment of this application, such as... Figure 8 As shown, the specific steps may include the following: Step S802: In response to the input command applied to the operation interface, display image data and query text data on the operation interface.

[0131] The image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language.

[0132] Step S804: Input the image data and query text data into multiple image evaluation models respectively, and use the multiple image evaluation models to evaluate the quality of the image data from the corresponding evaluation dimensions to obtain multiple evaluation results.

[0133] Step S806: Integrate multiple evaluation results to obtain the quality evaluation result of the image data.

[0134] Step S808: Display the quality assessment results on the operation interface.

[0135] According to another aspect of the embodiments of this application, an image processing apparatus is also provided, which can execute the image processing method of the above embodiments. The specific implementation method and preferred application scenarios are the same as those of the above embodiments, and will not be repeated here.

[0136] Figure 9 This is a schematic diagram of an image processing apparatus according to an embodiment of this application, such as... Figure 9 As shown, the device includes the following: an acquisition module 902, an evaluation module 904, and an integration module 906.

[0137] The acquisition module 902 is used to acquire image data and query text data. The image data is generated by an image generation model, and the query text data is used to describe the requirements for quality assessment of the image data using natural language. The evaluation module 904 is used to input the image data and query text data into multiple image evaluation models respectively, and use the multiple image evaluation models to evaluate the quality of the image data from the corresponding evaluation dimensions to obtain multiple evaluation results. The integration module 906 is used to integrate the multiple evaluation results to obtain the quality assessment result of the image data.

[0138] The evaluation module is also used to determine the evaluation scenario corresponding to the image data; determine multiple evaluation dimensions that match the evaluation scenario; obtain image evaluation models corresponding to the multiple evaluation dimensions, and obtain multiple image evaluation models. The image evaluation models are trained using multiple training images and the target evaluation results corresponding to the multiple training images according to the corresponding evaluation dimensions. The multiple training images are constructed based on at least one original image, and the target evaluation results are obtained by labeling the multiple training images respectively.

[0139] The evaluation module is further configured to, for any given image evaluation model, input image data and query text data into the image evaluation model, use the image evaluation model to perform quality evaluation on the image data from the corresponding evaluation dimension, and obtain evaluation text data, wherein the evaluation text data is used to describe the quality of the image data in the corresponding evaluation dimension in natural language; based on the evaluation text data, an evaluation result is generated, wherein the evaluation result is used to characterize the quality quantification score of the image data in the corresponding evaluation dimension.

[0140] The evaluation module is also used to extract features from image data according to the corresponding evaluation dimensions using an image evaluation model to obtain target image features; and to evaluate the quality of the target image features using the image evaluation model to obtain evaluation text data.

[0141] The evaluation text data includes: the probabilities corresponding to multiple quality evaluation levels; the evaluation module is also used to normalize the probabilities corresponding to the quality evaluation levels to obtain the normalized probabilities corresponding to the quality evaluation levels; and to perform weighted processing on the normalized probabilities corresponding to multiple quality evaluation levels to obtain the evaluation results.

[0142] The integration module is also used to normalize multiple evaluation results to obtain multiple normalized results, wherein the multiple normalized results conform to a normal distribution; determine the target weight values ​​corresponding to the multiple normalized results; perform weighted processing on the multiple normalized results based on the target weight values ​​to obtain weighted results; and map the weighted results to a preset range to obtain quality evaluation results.

[0143] The integration module is also used to determine the target evaluation dimension corresponding to the normalization result; obtain the preset weight value corresponding to the target evaluation dimension, and obtain the target weight value, wherein the preset weight value is determined during the training process of multiple image evaluation models.

[0144] According to another aspect of the embodiments of this application, a training device for an image evaluation model is also provided. This device can execute the training method of the image evaluation model in the above embodiments. The specific implementation method and preferred application scenarios are the same as those in the above embodiments, and will not be repeated here.

[0145] Figure 10 This is a schematic diagram of a training device for an image evaluation model according to an embodiment of this application, as shown below. Figure 10 As shown, the device includes the following: a first construction module 1002, a labeling module 1004, a second construction module 1006, and a training module 1008.

[0146] The first construction module 1002 is used to construct multiple training images based on at least one original image; the annotation module 1004 is used to annotate the multiple training images respectively to obtain the target evaluation results corresponding to the multiple training images; the second construction module 1006 is used to construct training data based on the multiple training images and the target evaluation results; the training module 1008 is used to train multiple initial evaluation models according to different evaluation dimensions using the training data to obtain multiple image evaluation models, wherein the different image evaluation models are used to evaluate the quality of image data and query text data from different evaluation dimensions to obtain evaluation results.

[0147] The first construction module is further configured to perform cue word recognition on at least one original image to obtain at least one original cue word; expand the at least one original cue word to obtain multiple target cue words; and generate multiple training images based on the multiple target cue words.

[0148] The first construction module is further used to input at least one original prompt word into the prompt word generation model, expand the original prompt word using the prompt word generation model to obtain multiple expanded prompt words, and obtain multiple target prompt words based on at least one original prompt word and multiple expanded prompt words.

[0149] The first construction module is further configured to, for any given target prompt word, input the target prompt word into at least one image generation model, and use multiple image generation models to generate at least one generated image corresponding to the target prompt word; and summarize the at least one generated image corresponding to multiple target prompt words to obtain multiple training images.

[0150] The annotation module is also used to annotate any training image multiple times to obtain multiple target annotation results for the training image; and to obtain the average value of the multiple target annotation results to obtain the target evaluation result of the training image.

[0151] The annotation module is also used to annotate the training image multiple times to obtain multiple initial annotation results; to perform correlation processing on the multiple initial annotation results to obtain the correlation coefficient of the initial annotation results; to perform normalization processing on the multiple initial annotation results to obtain multiple normalized annotation results; and to filter the multiple initial annotation results based on the correlation coefficient and the multiple normalized annotation results to obtain multiple target annotation results.

[0152] The training module is further configured to input the training image and query text data into multiple image evaluation models for any given training image, and use these models to evaluate the image data from their respective evaluation dimensions, resulting in multiple training evaluation results. These training evaluation results are then normalized to obtain multiple training normalized results, which conform to a normal distribution. Based on these normalized results and the target evaluation results, the initial weights for different evaluation dimensions are adjusted to obtain preset weights for each evaluation dimension.

[0153] The training module is also used to obtain loss function values ​​based on multiple training normalization results, initial weight values ​​of different evaluation dimensions, and target evaluation results; and to adjust the initial weight values ​​of different evaluation dimensions based on the loss function values ​​to obtain preset weight values ​​for different evaluation dimensions.

[0154] According to another aspect of the embodiments of this application, an image processing apparatus is also provided, which can execute the image processing method of the above embodiments. The specific implementation method and preferred application scenarios are the same as those of the above embodiments, and will not be repeated here.

[0155] Figure 11 This is a schematic diagram of an image processing apparatus according to an embodiment of this application, such as... Figure 11 As shown, the device includes the following: a first display module 1102, an evaluation module 1104, an integration module 1106, and a second display module 1108.

[0156] The first display module 1102 is used to respond to input commands applied to the operation interface and display image data and query text data on the operation interface. The image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language. The evaluation module 1104 is used to input the image data and query text data into multiple image evaluation models respectively, and use the multiple image evaluation models to perform quality assessment of the image data from the corresponding evaluation dimensions to obtain multiple evaluation results. The integration module 1106 is used to integrate the multiple evaluation results to obtain the quality assessment result of the image data. The second display module 1108 is used to display the quality assessment result on the operation interface.

[0157] Embodiments of this application may provide an electronic device. Figure 12 This is a structural block diagram of an electronic device according to an embodiment of this application, such as... Figure 12 As shown, the electronic device may include: an input / output device 1202; a memory 1204; and a processor 1206, wherein the processor 1206 is connected to the input / output device 1202 and the memory 1204 via a bus 1208.

[0158] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0159] The processor can invoke an executable program stored in memory via a transmission device to perform the following methods: acquiring image data and query text data, wherein the image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language; inputting the image data and query text data into multiple image assessment models respectively, and using the multiple image assessment models to perform quality assessment of the image data from corresponding assessment dimensions to obtain multiple assessment results; integrating the multiple assessment results to obtain the quality assessment result of the image data; and performing the methods in the various embodiments of this application.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0161] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause the processing unit to execute the methods in the various embodiments of this application.

[0163] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0164] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store program code executed by the method provided in the above embodiments.

[0165] Optionally, in this embodiment, the storage medium may be located in a computing device.

[0166] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, which, when the executable program is running, controls the device where the computer-readable storage medium is located to execute the method described in any of the above embodiments.

[0167] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.

[0168] The aforementioned computer program products can refer to software programs that have been written, tested, and released, and can run on computers or other devices. Computer program products can include application programs, operating systems, utility software, etc., used to achieve specific functions or solve specific problems.

[0169] Embodiments of this application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0170] The aforementioned non-volatile computer-readable storage medium can refer to a medium for storing data. Non-volatile computer-readable storage media can retain data without loss when power is off and can be used to store long-term data, such as operating systems, applications, and user files. Non-volatile storage media can include hard disk drives, solid-state drives, optical disks, and flash memory storage devices, etc.

[0171] Embodiments of this application also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the method provided in the above embodiments.

[0172] The aforementioned computer program can refer to a set of instructions used to tell the computer to perform specific tasks or operations. Computer programs can be written by programmers using specific programming languages ​​and can include algorithms, data structures, logic, and control flow. Computer programs can be used for a variety of purposes, including application software, operating systems, etc.

[0173] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0175] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0176] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0178] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Acquire image data and query text data, wherein the image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language; The image data and the query text data are respectively input into multiple image evaluation models, and the image data is evaluated from the corresponding evaluation dimensions using the multiple image evaluation models to obtain multiple evaluation results; The multiple evaluation results are integrated to obtain the quality evaluation result of the image data.

2. The method according to claim 1, characterized in that, The method further includes: Determine the evaluation scenario corresponding to the image data; Identify multiple evaluation dimensions that match the evaluation scenario; Obtain the image evaluation models corresponding to the multiple evaluation dimensions respectively, and obtain the multiple image evaluation models. The image evaluation models are obtained by training multiple training images and the target evaluation results corresponding to the multiple training images according to the corresponding evaluation dimensions. The multiple training images are constructed based on at least one original image, and the target evaluation results are obtained by annotating the multiple training images respectively.

3. The method according to claim 1 or 2, characterized in that, The image data and the query text data are respectively input into multiple image evaluation models, and the image data is evaluated for quality from corresponding evaluation dimensions using the multiple image evaluation models to obtain multiple evaluation results, including: For any image evaluation model, the image data and the query text data are input into the image evaluation model, and the image evaluation model is used to evaluate the quality of the image data from the corresponding evaluation dimension to obtain evaluation text data. The evaluation text data is in natural language and is used to describe the quality of the image data under the corresponding evaluation dimension. Based on the evaluation text data, the evaluation result is generated, wherein the evaluation result is used to characterize the quality quantification score of the image data under the corresponding evaluation dimension.

4. The method according to claim 3, characterized in that, The process of using the image evaluation model to evaluate the image data from the corresponding evaluation dimensions to obtain evaluation text data includes: The image evaluation model is used to extract features from the image data according to the corresponding evaluation dimensions to obtain the target image features; The image evaluation model is used to evaluate the quality of the target image features to obtain the evaluation text data.

5. The method according to claim 3, characterized in that, The evaluation text data includes: the probabilities corresponding to multiple quality evaluation levels; generating the evaluation result based on the evaluation text data includes: The probability corresponding to the quality assessment level is normalized to obtain the normalized probability corresponding to the quality assessment level. The normalized probabilities corresponding to the multiple quality assessment levels are weighted to obtain the assessment result.

6. The method according to claim 1 or 2, characterized in that, The process of integrating the multiple evaluation results to obtain the quality evaluation result of the image data includes: The multiple evaluation results are normalized to obtain multiple normalized results, wherein the multiple normalized results conform to a normal distribution; Determine the target weight values ​​corresponding to the multiple normalization results; The multiple normalization results are weighted based on the target weight value to obtain a weighted result. The weighted result is mapped to a preset range to obtain the quality assessment result.

7. The method according to claim 6, characterized in that, Determining the target weight values ​​corresponding to the plurality of normalization results includes: Determine the target evaluation dimension corresponding to the normalization result; Obtain the preset weight value corresponding to the target evaluation dimension to obtain the target weight value, wherein the preset weight value is determined during the training process of the plurality of image evaluation models.

8. A training method for an image evaluation model, characterized in that, include: Multiple training images are constructed based on at least one original image; The multiple training images are labeled respectively to obtain the target evaluation results corresponding to the multiple training images respectively; Training data is constructed based on the multiple training images and the target evaluation results; The training data is used to train multiple initial evaluation models according to different evaluation dimensions to obtain multiple image evaluation models. The different image evaluation models are used to evaluate the quality of image data and query text data from different evaluation dimensions to obtain evaluation results.

9. The method according to claim 8, characterized in that, The construction of multiple training images based on at least one original image includes: Each of the at least one original image is subjected to prompt word recognition to obtain at least one original prompt word; The at least one original prompt word is expanded to obtain multiple target prompt words; The multiple training images are generated based on the multiple target prompt words.

10. The method according to claim 9, characterized in that, The expansion of the at least one original prompt word to obtain multiple target prompt words includes: The at least one original prompt word is input into the prompt word generation model, and the original prompt word is expanded using the prompt word generation model to obtain multiple expanded prompt words; Based on the at least one original prompt word and the plurality of extended prompt words, the plurality of target prompt words are obtained.

11. The method according to claim 9, characterized in that, The process of generating the multiple training images based on the multiple target prompt words includes: For any target prompt word, the target prompt word is input into at least one image generation model, and at least one generated image corresponding to the target prompt word is generated using the multiple image generation models; The generated images corresponding to each of the multiple target prompt words are summarized to obtain the multiple training images.

12. The method according to claim 8, characterized in that, The step of annotating the multiple training images to obtain target evaluation results corresponding to the multiple training images includes: For any training image, the training image is annotated multiple times to obtain multiple target annotation results for the training image; The average value of the multiple target annotation results is obtained to obtain the target evaluation result of the training image.

13. The method according to claim 12, characterized in that, The process of repeatedly annotating the training image to obtain multiple target annotation results for the training image includes: The training image is annotated multiple times to obtain multiple initial annotation results for the training image; The correlation coefficients of the initial annotation results are obtained by performing correlation processing on the multiple initial annotation results. The initial annotation results are normalized to obtain multiple normalized annotation results; Based on the correlation coefficient and the multiple normalized annotation results, the multiple initial annotation results are filtered to obtain the multiple target annotation results.

14. The method according to any one of claims 8 to 13, characterized in that, The method further includes: For any training image, the training image and the query text data are respectively input into the multiple image evaluation models, and the multiple image evaluation models are used to evaluate the quality of the image data from the corresponding evaluation dimensions to obtain multiple training evaluation results; The multiple training evaluation results are normalized to obtain multiple training normalization results, wherein the multiple training normalization results conform to a normal distribution. Based on the multiple training normalization results and the target evaluation results, the initial weight values ​​of different evaluation dimensions are adjusted to obtain the preset weight values ​​of different evaluation dimensions.

15. The method according to claim 14, characterized in that, The step of adjusting the initial weight values ​​of different evaluation dimensions based on the multiple training normalization results and the target evaluation results to obtain preset weight values ​​for different evaluation dimensions includes: Based on the multiple training normalization results, the initial weight values ​​of different evaluation dimensions, and the target evaluation results, the loss function value is obtained; The initial weight values ​​of different evaluation dimensions are adjusted based on the loss function value to obtain the preset weight values ​​of different evaluation dimensions.

16. An image processing method, characterized in that, include: In response to input commands applied to the user interface, image data and query text data are displayed on the user interface, wherein the image data is generated by an image generation model, and the query text data is used to describe the need for quality assessment of the image data using natural language. The image data and the query text data are respectively input into multiple image evaluation models, and the image data is evaluated from the corresponding evaluation dimensions using the multiple image evaluation models to obtain multiple evaluation results; The multiple evaluation results are integrated to obtain the quality evaluation result of the image data; The quality assessment results are displayed on the user interface.

17. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor, connected to a memory via a bus, is used to run the program, wherein the program, when running, performs the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 16.

19. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 16.

Images