Image processing method and device, electronic equipment, storage medium and program product

Abstract

Embodiments of the present disclosure provide an image processing method and device, electronic equipment, storage medium and program product. The method comprises: obtaining a plurality of first images, generating a plurality of second images according to the plurality of first images and an image processing model; obtaining first prompt information, determining image description information of each second image according to the first prompt information, the second image and an image detection model; and determining the image processing effect of the image processing model according to the image description information of the plurality of second images. The technical solution of the embodiments of the present disclosure can realize targeted detection of the image processed by the image processing model, thereby enabling more accurate detection of the image processing effect of the image processing model, making the detection result of the image processing model more reliable and more convincing.

Description

Technical Field

[0001] This disclosure relates to the field of computer processing technology, and more particularly to an image processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the development of image processing and machine learning technologies, more and more image processing scenarios are leveraging pre-trained image processing models to achieve diverse image processing. Different image processing models often process images in different ways, resulting in varying image quality.

[0003] In related technologies, the image processing performance of image processing models often relies on subjective evaluations from the trainer or user regarding the convergence of the loss function during training and the number of iterations. However, this evaluation method typically only reflects the training level of the model and fails to reflect its specific processing effectiveness on the input image. In summary, the methods for determining the image processing performance of image processing models in related technologies struggle to accurately and objectively assess their effectiveness. Summary of the Invention

[0004] This disclosure provides an image processing method, apparatus, electronic device, storage medium, and program product, which can perform targeted detection on images processed by an image processing model, thereby enabling more accurate detection of the image processing effect of the image processing model.

[0005] In a first aspect, embodiments of this disclosure provide an image processing method, the method comprising:

[0006] Multiple first images are acquired, and multiple second images are generated based on the multiple first images and an image processing model; wherein, the image processing model is trained on a first machine learning model based on the multiple first sample images and a first expected image corresponding to the first sample images;

[0007] Obtain first prompt information, and determine image description information for each of the second images based on the first prompt information, the second image, and the image detection model; wherein, the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model;

[0008] The image processing effect of the image processing model is determined based on the image description information of multiple second images.

[0009] Secondly, embodiments of this disclosure also provide an image processing apparatus, the apparatus comprising:

[0010] An image determination module is used to acquire multiple first images and generate multiple second images based on the multiple first images and an image processing model; wherein, the image processing model is trained on a first machine learning model based on the multiple first sample images and a first expected image corresponding to the first sample images;

[0011] An image description information determination module is used to acquire first prompt information and determine image description information for each of the second images based on the first prompt information, the second image, and the image detection model; wherein, the image detection model is trained on a second machine learning model based on multiple second sample images and a second expected image corresponding to the second sample images; the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model;

[0012] The image processing effect determination module is used to determine the image processing effect of the image processing model based on the image description information of multiple second images.

[0013] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0014] One or more processors;

[0015] Storage device for storing one or more programs.

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any of the embodiments of this disclosure.

[0017] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the image processing method as described in any of the embodiments of this disclosure.

[0018] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the image processing method as described in any of the embodiments of this disclosure.

[0019] The technical solution of this disclosure, by acquiring multiple first images and generating multiple second images based on the multiple first images and an image processing model, can obtain image data that directly reflects the image processing effect. Furthermore, using multiple second images as the basis for detecting the image processing effect makes the detection results more comprehensive and objective. Then, by acquiring first prompt information and determining the image description information of each second image based on the first prompt information, the second images, and the image detection model, the image description information of the multiple second images generated by the image processing model can be automatically and quickly determined. Since the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model, targeted detection can be achieved using this first prompt information. By providing targeted guidance to the image detection model, indicating the direction of detection and the content to be focused on, the image detection model can perform targeted detection of specific content in the second image, thereby obtaining more targeted image description information. In this case, since this image description information is the result of detecting specific aspects common to multiple second images, the data will be more convincing and accurate. Therefore, the presentation characteristics of multiple second images can be measured based on the image description information of multiple second images, providing a more reliable basis for detecting the image processing effect. Thus, by combining the presentation characteristics of multiple second images, a more comprehensive, objective, and accurate detection of the image processing effect of the image processing model can be achieved, and the detected image processing effect is more realistic, reliable, and convincing. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure;

[0022] Figure 2 A schematic flowchart of another image processing method provided in an embodiment of this disclosure;

[0023] Figure 3 A schematic flowchart of another image processing method provided in an embodiment of this disclosure;

[0024] Figure 4 This is a schematic flowchart illustrating an optional example of an image processing method for implementing embodiments of this disclosure.

[0025] Figure 5This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device for implementing an embodiment of the present disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0034] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0037] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0038] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this disclosure. This embodiment is applicable to scenarios where images are processed using an image processing model. The method can be executed by an image processing device, which can be implemented in software and / or hardware, optionally through an electronic device, such as a mobile terminal, PC, or server. Figure 1 As shown, the method in this embodiment may specifically include:

[0039] S110. Obtain multiple first images, and generate multiple second images based on the multiple first images and the image processing model; wherein, the image processing model is trained on the first machine learning model based on the multiple first sample images and the first expected image corresponding to the first sample images.

[0040] Image processing models are used to process images to achieve the desired image processing effect. Specifically, an image processing model is trained on a first machine learning model using a first sample image and a corresponding first expected image. The first expected image is the image corresponding to the first sample image and exhibiting the desired image processing effect compared to the first sample image. After training the first machine learning model using the first sample image and its corresponding first expected image, the image processing model is obtained. The resulting image processing model can achieve a specific image processing effect, which is related to the expected effect set during model training. For example, the resulting image processing model could be a beauty effect model for beautifying human faces, or an art style conversion effect model for applying an oil painting effect to landscape images, and so on.

[0041] Accordingly, the first image is the image that needs to be processed, and the second image is the image corresponding to the first image and having the expected image processing effect. There are multiple first images, which can be pre-selected or obtained based on an image generation model.

[0042] In this embodiment of the disclosure, the methods for acquiring multiple first images may include, but are not limited to, at least one of the following methods: generating multiple first images based on an image generation model (e.g., a machine learning model with the ability to generate images from text); acquiring multiple first images based on an image acquisition device; reading multiple first images from an image storage space with access rights; receiving multiple first images transmitted through a preset transmission path (e.g., a preset program data interface, etc.), etc.

[0043] As an optional technical solution in this embodiment, after acquiring multiple first images, these images can be input into an image processing model. The image processing model then generates multiple second images after image processing. There is a one-to-one correspondence between the first and second images. For example, taking a beautification feature model capable of enhancing facial features as an example, assuming 1000 first images are acquired, these 1000 first images can be input into the beautification feature model. After processing by the model, 1000 corresponding second images can be generated.

[0044] As another optional technical solution in this disclosure, generating multiple second images based on multiple first images and an image processing model may include: obtaining text description data corresponding to the first images, and inputting the first images and their corresponding text description data into the image processing model to obtain multiple second images. It should be noted that the text description data corresponding to different first images may be the same or different.

[0045] S120. Obtain first prompt information, and determine image description information for each second image based on the first prompt information, the second image, and the image detection model; wherein, the image detection model is trained on a second machine learning model based on multiple second sample images and the second expected image corresponding to the second sample images; the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model.

[0046] Image detection models can be used to detect and semantically understand input images, identifying their specific content and outputting corresponding image descriptions. These descriptions are textual descriptions of the image content, reflecting aspects such as the main objects, the scene, and the relationships between different objects. For example, an image detection model could be a Visual Question Answering (VQA) model, outputting the image description "A black dog is chasing a red ball on the grass."

[0047] It should be noted that the image detection model can be a model trained on a single image processing model or a model trained on multiple image detection models. That is, the image detection model can be a dedicated detection model for a single image processing model or a general image detection model applicable to multiple image processing models. Specifically, the second sample images used to train the image detection model can be sample images that exhibit the same characteristics as the output images of a single image processing model in terms of preset features, or they can include sample images that exhibit the same characteristics as the output images of multiple image processing models in terms of preset features corresponding to those image processing models. The preset features corresponding to the output images of different image processing models can be one or more, and can be completely identical, partially identical, or completely different; no specific limitations are imposed here. The image detection model trained on the second machine learning model based on multiple second sample images and the corresponding second expected images enables the image detection model to possess the expected image detection capabilities, improving the accuracy and robustness of image processing.

[0048] The first hint information includes descriptive data about the image content that the image detection model needs to detect and / or the content it needs to output. The image content to be detected indicates the main content and direction of detection; the descriptive data of the output content refers to the specific content the image detection model needs to output. Using this first hint information, the image detection model can be specifically guided to identify the detection direction and the content to be prioritized, enabling it to detect specific content within an image with specialization and focus.

[0049] Specifically, the first prompt can be pre-set or generated based on the functional effects of the image processing model. The first prompt can be in the form of a question or a statement. The display format and the amount of information contained in the first prompt are not specified here. For example, the first prompt could be "What is the hairstyle of the person in the image?", or it could be "Please detect the background color in the image and the type of clothing worn by the person in the image," etc.

[0050] In embodiments of this disclosure, the image description data is used to describe at least a portion of the image content of the second image. The image description data can also be used to describe quality data of the second image, such as image sharpness.

[0051] After obtaining the initial prompt information, the initial prompt information and multiple second images can be input into the image detection model. The image detection model will perform targeted detection on each second image based on the initial prompt information, and then output the corresponding image description information for each second image. It should be noted that the initial prompt information can guide the image detection model to detect multiple second images. The process of inputting the initial prompt information and multiple second images into the image detection model can be as follows: first input the initial prompt information, then input the multiple second images; second input the multiple second images, then input the initial prompt information; or third, package both together and input them into the image detection model.

[0052] S130. Determine the image processing effect of the image processing model based on the image description information of multiple second images.

[0053] In this embodiment of the disclosure, the image processing effect can be understood as the processing effect of the image processing model on the input image. The image processing effect can be obtained by analyzing the image description data of the second image output by the image processing model. For example, the image processing effect can be determined based on the degree of similarity between multiple second images processed by the image processing model. If the degree of similarity between multiple second images is higher, it indicates that the multiple second images are more similar, and the processing effect of the image processing model is considered to be worse, and the processed images are relatively similar; conversely, if the degree of similarity between multiple second images is lower, it indicates that the processing effect of the image processing model is better, and the processed images are less similar.

[0054] Specifically, since the image detection mode performs targeted detection on the second image, the determined image description information can essentially reflect the detection results of each second image in terms of specific content. These detection results can then be used to measure whether the images processed by the image processing model are highly similar; in other words, the image processing effect of the image processing model can be measured based on the image description information of multiple second images. In some embodiments, the image processing effect can be determined based on the degree of similarity between the image description information of multiple second images. The higher the similarity of the image description information, the worse the image processing effect; the lower the similarity of the image description information, the better the image processing effect.

[0055] For example, suppose the first prompt instructs the image detection model to detect objects contained in the background of an image, and to detect the color of each object. If the image detection model detects three second images and obtains three image descriptions: the first is "The background of image 1 contains white clouds, a river, green mountains, white flowers, and yellow stones"; the second is "The background of image 2 contains a river, clouds, mountains, trees, white flowers, and yellow stones"; and the third is "The background of image 3 contains green mountains, clouds, a river, mountains, and white flowers." It is evident that based on these three image descriptions, it can be determined that the background content of these three second images is highly similar. Therefore, it can be considered that the image processing effect of the corresponding image processing model is poor, and the processed images are identical.

[0056] The technical solution of this disclosure, by acquiring multiple first images and generating multiple second images based on the multiple first images and an image processing model, can obtain image data that directly reflects the image processing effect. Furthermore, using multiple second images as the basis for detecting the image processing effect makes the detection results more comprehensive and objective. Then, by acquiring first prompt information and determining the image description information of each second image based on the first prompt information, the second images, and the image detection model, the image description information of the multiple second images generated by the image processing model can be automatically and quickly determined. Since the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model, targeted detection can be achieved using this first prompt information. By providing targeted guidance to the image detection model, indicating the direction of detection and the content to be focused on, the image detection model can perform targeted detection of specific content in the second image, thereby obtaining more targeted image description information. In this case, since this image description information is the result of detecting specific aspects common to multiple second images, the data will be more convincing and accurate. Therefore, the presentation characteristics of multiple second images can be measured based on the image description information of multiple second images, providing a more reliable basis for detecting the image processing effect. Thus, by combining the presentation characteristics of multiple second images, a more comprehensive, objective, and accurate detection of the image processing effect of the image processing model can be achieved, and the detected image processing effect is more realistic, reliable, and convincing.

[0057] Figure 2 This is a flowchart illustrating another image processing method provided in this embodiment. Based on the above embodiments, the technical solution of this embodiment further refines the method of acquiring the first image. Optionally, acquiring multiple first images includes: acquiring at least one second prompt information; generating multiple third prompt information based on at least one second prompt information and a text generation model; wherein the second prompt information includes descriptive data of the object to be described by the text generation model; the text generation model is trained on a third machine learning model based on first sample prompt information and expected prompt information corresponding to the first sample prompt information; generating multiple first images based on multiple third prompt information and an image generation model; wherein the image generation model is trained on a fourth machine learning model based on second sample prompt information and a third expected image corresponding to the second sample prompt information. For detailed implementation, please refer to the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. Figure 2 As shown, the method in this embodiment may specifically include:

[0058] S210. Obtain at least one second prompt information, and generate multiple third prompt information based on at least one second prompt information and a text generation model; wherein, the second prompt information includes descriptive data of the object to be described by the text generation model; the text generation model is trained on the third machine learning model based on the first sample prompt information and the expected prompt information corresponding to the first sample prompt information.

[0059] The text generation model is used to generate semantically coherent and grammatically correct text content. The first sample prompt is used to train the third machine learning model. The expected prompt provides the text generation model with a specific direction for text generation, guiding it to generate expected text content. For example, the first sample prompt could be "Generate text content about a puppy," and the corresponding expected prompt could be "The puppy is playing on the grass," etc. After training the third machine learning model using the first sample prompt and its corresponding expected prompt, the text generation model is obtained. For example, the trained text generation model could be a language model with a preset number of model parameters, etc.

[0060] Accordingly, the second prompt includes descriptive data about the object that the text generation model needs to describe. This descriptive data is generally text-based, and its content can consist of one or more "words" or one or more "sentences." The second prompt can be pre-set based on the user's actual needs, and at least one second prompt can be set.

[0061] After obtaining at least one second prompt, it can be input into a text generation model. The model then processes this information to generate multiple corresponding third prompts. In short, compared to the third prompt, the second prompt is essentially a coarse-grained prompt, while the third prompt is a fine-grained prompt obtained by expanding and enriching the second prompt.

[0062] For example, the text generation model can have two second prompts: "Generate text content featuring male, female, and child characters, who should be positive and energetic," and "Characters should be able to depict various postures such as standing, sitting, and walking." Based on these two second prompts and the LLM model, the following third prompts can be generated: "A little girl with a ponytail sits at a desk covered with colored pencils and paper, intently depicting her dream world, her face beaming with creative joy," "A little boy in athletic shorts and a T-shirt runs energetically on a green soccer field, a soccer ball at his feet, his eyes filled with a desire for victory," and "An elegant lady in a stylish dress and exquisite high heels, carrying a designer handbag, strolls leisurely through an art-filled street, a gentle smile on her face."

[0063] As an optional but non-limiting implementation, obtaining at least one second prompt message includes: obtaining model description information of the image processing model, and generating at least one second prompt message based on the model description information and the text generation model; wherein the text generation model is trained on a fourth machine learning model based on third sample prompt information and second expected prompt information corresponding to the third sample prompt information. By adopting this technical solution, generating at least one second prompt message based on the model description information of the image processing model and the text generation model, the second prompt message can better fit the image processing model, thereby enabling subsequent solutions to generate multiple images that better meet the requirements of the image processing model based on the second prompt message.

[0064] Specifically, the model description information of the image processing model can be obtained first, and then input into the text generation model to generate at least one second prompt message. The text generation model used in this process can be the same as the text generation model used in generating multiple third prompt messages based on the second prompt message, or they can be different text generation models, as long as they can perform the text generation function. This embodiment does not specify further details.

[0065] As an optional but non-limiting implementation, the second prompt information is associated with the image processing model; at least one second prompt information is determined based on the model description information; wherein the model description information is used to describe at least the presentation features of the image after processing by the image processing model.

[0066] The presentation features include at least one of content features, structural features, and style features. The model description information is used to describe at least the presentation features of the image after processing by the image processing model. That is, the model description information reflects what effects the image processing model can perform on the image, and what characteristics the processed image can exhibit. For example, the model description information could be "this image processing model can beautify parts of an image," or "this image processing model can process a landscape image into an oil painting style image," etc. In this embodiment, the second prompt information is associated with the image processing model; at least one second prompt information can be determined based on the model description information.

[0067] As an optional but non-limiting implementation, further, at least one second prompt message is determined based on the model description information. Specifically, this may include determining at least one second prompt message from a preset prompt message library based on the model description information. The preset prompt message library contains various types of effect keywords used to describe the image processing effects of the image processing model and corresponding associated prompt messages. Using this technical solution, based on the model description information of the image processing model, at least one second prompt message that better fits the image processing model can be determined from the preset prompt message library, thereby enabling subsequent solutions to generate multiple images that better meet the requirements of the image processing model based on the second prompt message.

[0068] Specifically, after obtaining the model description information of the image processing model, this description information can be analyzed to determine the effect keywords of the image processing model. Then, based on these effect keywords, at least one prompt message can be matched from a preset prompt information library, and the matched prompt message can be used as a second prompt message. For example, if the model description information is "this image processing model can beautify parts of an image," then the analyzed and determined functional keyword could be "part beautification," and thus at least one second prompt message can be determined from the preset prompt information library based on the functional keyword "part beautification."

[0069] S220. Generate multiple first images based on multiple third prompts and an image generation model; wherein the image generation model is obtained by training a fourth machine learning model based on second sample prompts and third expected images corresponding to the second sample prompts.

[0070] In this model, the second sample prompt is the prompt used during the training of the fourth machine learning model, and the second expected image is an image that can display the specific content described by the second sample prompt. In other words, although the second sample prompt and the corresponding second expected image are different forms of data, the information they describe is essentially harmonious and unified; that is, the content of the second expected image is consistent with the content described by the second sample prompt. For example, if the second sample prompt is "a little girl is running on the grass," then the corresponding second expected image will at least show a little girl running on the grass. The trained image generation model can be a text-to-image (T2I) model.

[0071] Specifically, after obtaining multiple third-party prompts, these prompts can be input into the image generation model to generate multiple first images.

[0072] As an optional but non-limiting implementation, multiple first images are generated based on multiple third prompts and an image generation model, including at least one of the following operations: inputting multiple third prompts into the image generation model to obtain multiple first images; inputting multiple third prompts and corresponding second prompts into the image generation model to obtain multiple first images; determining multiple fourth prompts from the multiple third prompts and inputting the multiple fourth prompts into the image generation model to obtain multiple first images. Using this technical solution, multiple corresponding first images can be generated based on the third prompts and the image generation model through various operations. In particular, multiple fourth prompts that better fit the functional characteristics of the image processing model can be determined from the multiple third prompts, thereby generating first images that better fit the functional characteristics of the image processing model, thus facilitating subsequent solutions to determine more realistic, objective, and accurate image processing effects.

[0073] The fourth prompt is a prompt that is compatible with the functional characteristics of the image processing model. Multiple fourth prompts are determined from multiple third prompts, including by filtering multiple fourth prompts based on the functional description data of the image processing model. The functional description data describes the image processing functions corresponding to the image processing model. The fourth prompts determined using the functional description data of the image processing model ensure that the first image generated subsequently is more compatible with the image processing model in subsequent solutions.

[0074] Specifically, generating multiple first images based on multiple third prompts and an image generation model may include at least one of the following operations: multiple third prompts may be directly input into the image generation model so that the image generation model generates images that correspond to the third prompts, thereby obtaining multiple first images.

[0075] Alternatively, multiple third prompts and corresponding second prompts can be input into the image generation model. This allows the image generation model to input both coarse-grained second prompts and fine-grained third prompts, with a correlation between the second and third prompts. This enables the image generation model to generate images that better meet the requirements of the prompts, thereby obtaining multiple first images.

[0076] Alternatively, multiple fourth prompts can be determined from multiple third prompts. These fourth prompts better align with the functional characteristics of the image processing model in the subsequent solution. Inputting these fourth prompts into the image generation model yields multiple first images. It's important to note that some of the third prompts generated by the text generation model may not align with the functional characteristics of the image processing model in the subsequent solution. Therefore, the first images generated based on these mismatched third prompts may not be beneficial for judging the image processing effect of the further image processing model. For example, if the image processing model is a feature effect model capable of beautifying a part, and one of the generated third prompts, 'a', is "a beautiful woman wearing a veil, her brows slightly furrowed, her eyes revealing sadness," then the features related to the part in the first image A generated based on such third prompt 'a' may not be comprehensive or clear. Therefore, processing this first image A using the feature effect model may not objectively and realistically reflect the beautification effect of the feature effect model. It is clear that such third prompts are not preferred. This implementation scheme takes these special cases into account, identifying multiple fourth prompts from among the third prompts that better match the functional characteristics of the image processing model. This allows for the generation of a first image that better matches the functional characteristics of the image processing model, facilitating subsequent schemes to determine a more realistic, objective, and accurate image processing effect.

[0077] S230. Generate multiple second images based on multiple first images and an image processing model; wherein the image processing model is trained on a first machine learning model based on a first sample image and a first expected image corresponding to the first sample image.

[0078] S240. Obtain first prompt information, and determine image description information for each second image based on the first prompt information, the second image, and the image detection model; wherein, the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model.

[0079] S250. Determine the image processing effect of the image processing model based on the image description information of multiple second images.

[0080] The technical solution of this disclosure, by acquiring at least one second prompt information and generating multiple third prompt information based on the at least one second prompt information and a text generation model, can enrich and expand the coarse-grained second prompt information into fine-grained third prompt information with the help of the text generation model, thereby improving the richness of the prompt information content and providing more content prompts when the image generation model generates images. Since the second prompt information includes descriptive data of the object to be described by the text generation model, and then by generating multiple first images based on the multiple third prompt information and the image generation model, the image generation model can generate multiple corresponding first images based on the multiple third prompt information with richer content, thereby improving the generation quality of the first images and thus improving the detection accuracy of the image processing effect of the image processing model.

[0081] Figure 3 This is a flowchart illustrating another image processing method provided in this embodiment. Based on the above embodiments, this embodiment further refines how to determine the image processing effect of an image processing model based on image description information of multiple second images. Optionally, determining the image processing effect of the image processing model based on the image description information of multiple second images includes: determining the probability distribution data of multiple first images on a preset feature dimension based on the image description information of multiple second images, and determining the image processing effect of the image processing model based on the probability distribution data of the preset feature dimension. For detailed implementation, please refer to the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. Figure 3 As shown, the method in this embodiment may specifically include:

[0082] S310. Acquire multiple first images, and generate multiple second images based on the multiple first images and the image processing model; wherein, the image processing model is trained on the first machine learning model based on the first sample image and the first expected image corresponding to the first sample image.

[0083] S320. Obtain first prompt information, and determine image description information for each second image based on the first prompt information, the second image, and the image detection model; wherein, the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model.

[0084] As an optional but non-limiting implementation, obtaining the first prompt information includes: obtaining the model description information of the image processing model, and generating the first prompt information based on the model description information, the fifth prompt information, and the text generation model; wherein, the model description information is used at least to describe the presentation features of the image generated by the image processing model; and the fifth prompt information is used at least to query the detection content of the image output by the image detection model. By adopting this technical solution, a more suitable first prompt information can be generated based on the processing effect characteristics of the image processing model, ensuring that the generated first prompt information is specific to the image processing model and more consistent with the actual situation of the model. This allows for targeted evaluation of the functional characteristics of the image processing model, enabling subsequent solutions to determine a more realistic, objective, and accurate image processing effect.

[0085] For example, the presentation features include at least one of the features such as content features, structural features, and style features. The fifth prompt is used to query the detected content of the image output by the image detection model. That is, the fifth prompt covers at least the image content that the image detection model can detect.

[0086] Specifically, since different image processing models may be applied to image processing functions with different effects, the processing effect characteristics of such image processing models need to be considered when processing them. That is, in determining the first prompt, it can be generated based on the model description information, the fifth prompt, and the text generation model. This ensures that the generated first prompt is specific to the image processing model and more consistent with its actual situation. This allows for targeted evaluation of the image processing model's functional characteristics, leading to more realistic, objective, and accurate image processing effects in subsequent solutions. For example, assuming an image processing effect evaluation of a body part enhancement effect model, since the model's function is to enhance body parts, the determined first prompt should be more closely related to body part enhancement, rather than prompts such as oil painting style or clothing color changes. Specifically, the first prompt can be generated based on the model description information, the fifth prompt, and the text generation model of the body part enhancement effect model.

[0087] S330. Determine the probability distribution data of multiple second images on a preset feature dimension based on the image description information of multiple second images, and determine the image processing effect of the image processing model based on the probability distribution data of the preset feature dimension.

[0088] The preset feature dimensions are the pre-defined feature dimensions for image processing. These preset feature dimensions are related to the functional characteristics of the image processing model. For example, if the image processing model can process the hairstyle, clothing, and posture of a person in an image, then the corresponding preset feature dimensions could include these three dimensions. Probability distribution data is used to reflect the data distribution among multiple second images under each preset feature dimension.

[0089] Specifically, based on the image description information of multiple second images, the distribution of these images across various preset feature dimensions can be determined, thereby identifying the probability distribution data. For a single feature dimension, if the data is uniformly distributed, the image processing model can be considered to have performed well on that feature dimension, avoiding the situation where all images are processed in the same extreme manner, indicating a relatively good image processing effect. Similarly, for multiple preset feature dimensions, if the data is uniformly distributed across each feature dimension, then the image processing is considered uniform, resulting in a relatively good image processing effect. In determining the image processing effect, whether to judge from a single preset feature dimension or from multiple preset feature dimensions can be differentiated based on the actual situation; this embodiment does not specify this in detail.

[0090] As an optional but non-limiting implementation, determining the probability distribution data of multiple second images on a preset feature dimension based on image description information includes: determining the attribute values ​​of the second images on the preset feature dimension based on image description information, and determining the probability distribution data corresponding to the preset feature dimension based on the attribute values ​​of the multiple second images on the preset feature dimension. By employing this technical solution, determining the attribute values ​​of the second images on the preset feature dimension through image description information of multiple second images enables the determination of more realistic, objective, and accurate image processing effects, achieving the quantification of image processing effects.

[0091] In this context, attribute values ​​are the specific values ​​corresponding to the feature dimension, or the specific data representing the feature dimension. For example, if the preset feature dimension is "hair color," the hair colors in the image processed by the image processing model might be yellow, black, and white. Therefore, the attribute values ​​corresponding to the feature dimension "hair color" would be "yellow, black, and white," etc.

[0092] Specifically, determining the attribute values ​​of the second image in a preset feature dimension based on the image description information of the second image includes: performing field matching on the image description information of the second image according to the preset feature dimension, and determining the attribute values ​​of the second image in the preset feature dimension based on the matched field information. After determining the attribute values ​​of multiple second images in the preset feature dimension, the distribution among the multiple attribute values ​​can be determined, thereby determining the probability distribution data corresponding to each preset feature dimension.

[0093] Optionally, determining the attribute values ​​of the second image in a preset feature dimension based on the image description information of the second image further includes: analyzing the image description information of the second image using a text analysis model to determine the attribute values ​​of the second image in the preset feature dimension. The text analysis model can be used for semantic understanding and analysis of text content, and can also be a language model with a preset number of model parameters. Specifically, the image description information of the second image can be input into the text analysis model, and relevant recognition content and recognition questions can be set to determine the attribute values ​​of the second image in the preset feature dimension.

[0094] It should be noted that in this embodiment, the attribute values ​​of the second image in the preset feature dimension are determined based on the image description information of the second image. This is mainly to focus on which attribute values ​​the image processed by the image processing model has, and thus to judge the image processing effect between the obtained images. For example, taking "hair color" as the preset feature dimension, if the attribute values ​​determined based on the image description information of multiple second images are yellow, black, and white, and the image processing model could actually process "hair color" to red, but since none of the current obtained second images have red hair, then red cannot be used as the current attribute value. This embodiment focuses on judging the image processing effect of the image processing model based on the multiple processed second images.

[0095] As an optional but non-limiting implementation, determining the probability distribution data corresponding to a preset feature dimension based on the attribute values ​​of multiple second images in that preset feature dimension includes: determining the frequency of occurrence of each attribute value of the preset feature dimension in the multiple second images, and determining the probability distribution data corresponding to the preset feature dimension based on the frequency of occurrence of multiple attribute values. By employing this technical solution, and by determining the frequency of occurrence of each attribute value of the preset feature dimension in the multiple second images, thereby determining the corresponding probability distribution data, the accuracy and reliability of image processing effect detection can be improved, achieving the quantification of image processing effects.

[0096] Specifically, for an attribute value of a single preset feature dimension, the frequency of occurrence of the attribute value can be calculated by dividing the number of times a certain attribute value appears by the total number of times all attribute values ​​under that preset feature dimension appear. By analogy, the frequency of occurrence of each attribute value of the preset feature dimension in multiple second images can be determined, and then the probability distribution data corresponding to the preset feature dimension can be determined based on the frequency of occurrence of multiple attribute values.

[0097] For example, if the preset feature dimension of "hair color" is determined to have three attribute values—yellow, black, and white—from the image description information corresponding to 1000 second images, and yellow appears 300 times, black appears 299 times, and white appears 297 times in these 1000 image descriptions, then dividing the occurrence frequency of each attribute value by the total number of second images (1000) will determine the occurrence frequency of each attribute value, and thus the corresponding probability distribution data.

[0098] As an optional but non-limiting implementation, the image processing effect of the image processing model is determined based on the probability distribution data of the preset feature dimensions. This includes: determining the entropy of the probability distribution data of the preset feature dimensions, normalizing the entropy, and determining the image processing effect of the image processing model based on the normalized entropy. Using this technical solution, probability distribution data can be converted into normalized entropy, and using normalized entropy as a quantitative indicator, the image processing effect of the image processing model can be determined conveniently and quickly.

[0099] There are several methods in related technologies for determining the entropy of probability distribution data for preset feature dimensions, the details of which will not be elaborated here. Specifically, determining the image processing effect of an image processing model based on the normalized entropy can include: determining the image processing effect of the image processing model based on the normalized entropy and one or more preset thresholds. One or more preset thresholds can divide the normalized entropy into multiple value ranges, each value range corresponding to a description of the image processing effect of the image processing model. For example, the image processing effect of the image processing model could be described as excellent, good, average, poor, etc.; or, the image processing effect of the image processing model could be described as achieving a certain score, etc. In this way, the value range of the normalized entropy can be determined based on the normalized entropy and one or more preset thresholds, and the preset effect description data corresponding to this value range can be determined as the image processing effect of the image processing model. It can be understood that the value range of the normalized entropy is from 0 to 1; a higher value generally indicates a better image processing effect of the image processing model, and conversely, a lower value generally indicates a worse image processing effect of the image processing model.

[0100] The technical solution of this disclosure embodiment generates multiple second images using multiple first images and an image processing model. When determining the image description information for each second image, it is specifically based on an image detection model and first prompt information. The first prompt information includes descriptive data about the image content to be detected and / or the content to be output by the image detection model. With this first prompt information, the image detection model can be specifically instructed on the direction of detection and the content to be focused on, thereby enabling the image detection model to perform targeted detection of specific content in the second images, resulting in more targeted image description information. In this case, since these image description information are the results of detecting specific aspects common to multiple second images, the data is more convincing and accurate. Therefore, the similarity between these multiple second images can be measured based on their image description information. Specifically, the probability distribution data of the multiple second images on a preset feature dimension is determined based on their image description information, and then the image processing effect of the image processing model is determined based on the probability distribution data of the preset feature dimension. This solution enables more accurate detection of the image processing effect of the image processing model, and the detected image processing effect is more realistic, reliable, and convincing.

[0101] Figure 4 This is a schematic flowchart illustrating an optional example of an image processing method for implementing embodiments of this disclosure. Figure 4 As shown, the image processing method in this example may specifically include the following steps:

[0102] Step 1: Determining the third prompt information. Specifically, at least one second prompt information can be determined based on the model description information corresponding to the image processing model. At this time, the second prompt information is a coarse-grained prompt information. Then, at least one second prompt information is input into the text generation model (such as a language model with a preset number of parameters) to generate multiple third prompt information. The third prompt information is essentially a fine-grained prompt information obtained by expanding and enriching the second prompt information.

[0103] Step 2: Generation of multiple first images. Specifically, the obtained third-party prompts are input into the image generation model to generate multiple first images. For example, the image generation model can be a text-to-image (T2I) model.

[0104] Step 3: Generation of Multiple Second Images. Specifically, multiple first images are input into the image processing model to generate multiple second images. For example, taking an image processing model that can beautify human faces as an example, assuming there are 10,000 first images, these 10,000 first images are input into the beautification feature model. After processing by the model, 10,000 corresponding second images are generated. These second images have beautification effects compared to the first images.

[0105] Step 4: Determining the First Hint. Specifically, based on the image processing model, the corresponding model description information can be obtained, and a fifth hint can be determined that can at least be used to query the detection content of the image output by the image detection model. Then, the model description information and the fifth hint are input into the text generation model to output the generated first hint. This ensures that the generated first hint is specific to the image processing model and more in line with the model's actual situation. This allows for targeted evaluation of the image processing model's functional characteristics, enabling subsequent solutions to determine a more realistic, objective, and accurate image processing effect.

[0106] Step 5: Determining Image Description Information. Specifically, the initial prompt information and multiple second images can be input into an image detection model (e.g., a Visual Question Answering (VQA) model). The image detection model can then determine the image description information for each second image. Using this initial prompt information, the image detection model can be specifically instructed on the direction of detection and the content that needs to be prioritized. This allows the image detection model to perform targeted detection of specific content in the second images, resulting in more targeted image description information.

[0107] Step 6: Determining the Image Processing Effect of the Model. Specifically, after obtaining the image description information of multiple second images, this information can be analyzed using a text analysis model to determine the probability distribution data of the multiple second images on a preset feature dimension. This allows for the determination of the normalized entropy of the probability distribution data on the preset feature dimension. Based on this normalized entropy and a preset threshold, the image processing effect of the model is determined. For example, if the normalized entropy reaches the preset threshold, the image processing effect of the model is considered good; if the normalized entropy does not reach the preset threshold but is greater than the threshold, the image processing effect of the model is considered poor.

[0108] The technical solution of this disclosure eliminates the need for manual operation throughout the entire process. It automatically generates multiple images (second images) to determine the image processing effect of the image processing model through the cooperation of multiple models. Statistical analysis of these second images provides quantitative indicators, enabling simple, efficient, and accurate detection of the image processing effect. Furthermore, the deep integration of multimodal models fully utilizes their data processing capabilities for problem generation and comprehensive analysis. Combined with quantitative indicators, it ensures accurate return of consistency evaluation metrics while providing detailed explanations, thereby improving the detection efficiency and accuracy of the image processing model in terms of image processing effect.

[0109] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure, as shown below. Figure 5 As shown, the image processing device includes: an image determination module 510, an image description information determination module 520, and an image processing effect determination module 530. The image determination module 510 is used to acquire multiple first images and generate multiple second images based on the multiple first images and an image processing model; wherein the image processing model is trained on a first machine learning model based on a first sample image and a first expected image corresponding to the first sample image. The image description information determination module 520 is used to acquire first prompt information and determine image description information for each second image based on the first prompt information, the second images, and an image detection model; wherein the image detection model is trained on a second machine learning model based on multiple second sample images and a second expected image corresponding to the second sample image; the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model. The image processing effect determination module 530 is used to determine the image processing effect of the image processing model based on the image description information of the multiple second images.

[0110] The technical solution of this embodiment acquires multiple first images through the image determination module 510, and generates multiple second images based on the multiple first images and the image processing model. This allows for the acquisition of image data that directly reflects the image processing effect. Furthermore, using multiple second images as the basis for detecting the image processing effect makes the detection results more comprehensive and objective. Then, the image description information determination module 520 acquires first prompt information, and determines the image description information for each second image based on the first prompt information, the second images, and the image detection model. This automatically and quickly determines the image description information for the multiple second images generated by the image processing model. Since the first prompt information includes descriptive data about the image content to be detected and / or the content to be output by the image detection model, such first prompt information can be used to specifically instruct the image detection model on the detection content. The direction and key content to be detected are identified, enabling the image detection model to perform targeted detection of specific content in the second image, thereby obtaining more targeted image description information. Then, the image processing effect determination module 530 determines the image processing effect of the image processing model based on the image description information of multiple second images. In this case, since these image description information are the results of detecting specific aspects common to multiple second images, the data will be more convincing and accurate. Therefore, the presentation characteristics of multiple second images can be measured based on the image description information of multiple second images, which can provide a more reliable basis for detecting the image processing effect. Thus, by combining the presentation characteristics of multiple second images, a more comprehensive, objective, and accurate detection of the image processing effect of the image processing model can be achieved, and the detected image processing effect is more realistic, reliable, and convincing.

[0111] Based on any optional technical solution in the embodiments of this disclosure, the image determination module 510 includes a first image determination submodule. The first image determination submodule is used to acquire multiple first images. Specifically, the first image determination submodule includes a prompt information determination unit and a first image determination unit. The prompt information determination unit is used to acquire at least one second prompt information, and generate multiple third prompt information based on the at least one second prompt information and a text generation model; wherein the second prompt information includes descriptive data of the object to be described by the text generation model; the text generation model is trained on a third machine learning model based on first sample prompt information and expected prompt information corresponding to the first sample prompt information; the first image determination unit is used to generate multiple first images based on the multiple third prompt information and the image generation model; wherein the image generation model is trained on a fourth machine learning model based on second sample prompt information and a second expected image corresponding to the second sample prompt information.

[0112] Based on any optional technical solution in the embodiments of this disclosure, the first image determination unit is configured to generate multiple first images according to multiple third prompt information and an image generation model through at least one of the following operations: inputting multiple third prompt information into the image generation model to obtain multiple first images; inputting multiple third prompt information and second prompt information corresponding to the third prompt information into the image generation model to obtain multiple first images; determining multiple fourth prompt information from the multiple third prompt information, and inputting multiple fourth prompt information into the image generation model to obtain multiple first images.

[0113] Based on any optional technical solution in the embodiments of this disclosure, the second prompt information is associated with the image processing model; at least one second prompt information is determined according to the model description information; wherein, the model description information is at least used to describe the presentation characteristics of the image after processing by the image processing model.

[0114] Based on any optional technical solution in the embodiments of this disclosure, the prompt information determination unit is used to obtain at least one second prompt information, specifically including: obtaining the model description information of the image processing model, and generating at least one second prompt information according to the model description information and the text generation model; wherein, the text generation model is trained on a fourth machine learning model according to the third sample prompt information and the second expected prompt information corresponding to the third sample prompt information.

[0115] Based on any optional technical solution in the embodiments of this disclosure, the image description information determination module 520 includes a first prompt information determination submodule. The first prompt information determination submodule is used to obtain model description information of the image processing model, and generate first prompt information based on the model description information, fifth prompt information, and text generation model; wherein the model description information is at least used to describe the presentation features of the image generated by the image processing model; and the fifth prompt information is at least used to query the detection content of the image output by the image detection model.

[0116] Based on any optional technical solution in the embodiments of this disclosure, the image processing effect determination module 530 includes an image processing effect determination submodule. The image processing effect determination submodule is used to determine the probability distribution data of the multiple second images on a preset feature dimension based on the image description information of the multiple second images, and to determine the image processing effect of the image processing model based on the probability distribution data of the preset feature dimension.

[0117] Based on any optional technical solution in the embodiments of this disclosure, the image processing effect determination submodule includes a probability distribution data determination unit. The probability distribution data determination unit is used to determine the attribute value of the second image on a preset feature dimension based on the image description information of the second image, and to determine probability distribution data corresponding to the preset feature dimension based on the attribute values ​​of multiple second images on the preset feature dimension.

[0118] Based on any optional technical solution in the embodiments of this disclosure, the probability distribution data determination unit is specifically used to: determine the frequency of occurrence of each attribute value of the preset feature dimension in multiple second images, and determine the probability distribution data corresponding to the preset feature dimension based on the frequency of occurrence of the multiple attribute values.

[0119] Based on any optional technical solution in the embodiments of this disclosure, the image processing effect determination submodule further includes a processing effect determination unit. The processing effect determination unit is used to determine the entropy of the probability distribution data of the preset feature dimension, normalize the entropy, and determine the image processing effect of the image processing model based on the normalized entropy.

[0120] The image processing apparatus provided in this disclosure can execute the image processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the image processing method.

[0121] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0122] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0123] like Figure 6As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0124] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0125] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0126] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0127] The electronic device provided in this disclosure and the image processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this disclosure can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0128] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the image processing method provided in the above embodiments.

[0129] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0130] According to one or more embodiments of this disclosure, [Example 1] provides an image processing method, comprising: acquiring multiple first images, generating multiple second images based on the multiple first images and an image processing model; wherein the image processing model is trained on a first machine learning model based on multiple first sample images and a first expected image corresponding to the first sample images; acquiring first prompt information, determining image description information for each second image based on the first prompt information, the second images, and an image detection model; wherein the image detection model is trained on a second machine learning model based on multiple second sample images and a second expected image corresponding to the second sample images; the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model; and determining the image processing effect of the image processing model based on the image description information of the multiple second images.

[0131] According to one or more embodiments of this disclosure, [Example 2] provides the method of Example 1, further comprising: optionally, acquiring multiple first images includes: acquiring at least one second prompt information; generating multiple third prompt information based on at least one second prompt information and a text generation model; wherein the second prompt information includes descriptive data of the object to be described by the text generation model; the text generation model is trained on a third machine learning model based on first sample prompt information and expected prompt information corresponding to the first sample prompt information; generating multiple first images based on multiple third prompt information and an image generation model; wherein the image generation model is trained on a fourth machine learning model based on second sample prompt information and a second expected image corresponding to the second sample prompt information.

[0132] According to one or more embodiments of this disclosure, Example 3 provides the method of Example 2, which further includes: Optionally, generating multiple first images based on multiple third prompt messages and an image generation model includes at least one of the following operations: inputting multiple third prompt messages into the image generation model to obtain multiple first images; inputting multiple third prompt messages and second prompt messages corresponding to the third prompt messages into the image generation model to obtain multiple first images; determining multiple fourth prompt messages from the multiple third prompt messages, and inputting the multiple fourth prompt messages into the image generation model to obtain multiple first images.

[0133] According to one or more embodiments of this disclosure, Example 4 provides the method of Example 2, which further includes: optionally, the second prompt information is associated with the image processing model; determining at least one second prompt information according to the model description information; wherein the model description information is at least used to describe the presentation features of the image processed by the image processing model.

[0134] According to one or more embodiments of this disclosure, Example 5 provides the method of Example 4, which further includes: optionally, obtaining at least one second prompt information includes: obtaining model description information of the image processing model, and generating at least one second prompt information based on the model description information and the text generation model; wherein the text generation model is trained on a fourth machine learning model based on third sample prompt information and second expected prompt information corresponding to the third sample prompt information.

[0135] According to one or more embodiments of this disclosure, Example Six provides the method of Example One, which further includes: Optionally, obtaining the first prompt information includes: obtaining model description information of the image processing model, and generating the first prompt information based on the model description information, the fifth prompt information, and the text generation model; wherein the model description information is at least used to describe the presentation features of the image generated by the image processing model; and the fifth prompt information is at least used to query the detection content of the image output by the image detection model.

[0136] According to one or more embodiments of this disclosure, Example 7 provides the method of Example 1, which further includes: Optionally, determining the image processing effect of the image processing model based on the image description information of the plurality of second images includes: determining the probability distribution data of the plurality of second images on a preset feature dimension based on the image description information of the plurality of second images, and determining the image processing effect of the image processing model based on the probability distribution data of the preset feature dimension.

[0137] According to one or more embodiments of this disclosure, [Example 8] provides the method of Example 7, which further includes: optionally, determining the probability distribution data of the plurality of second images on a preset feature dimension based on the image description information of the plurality of second images includes: determining the attribute value of the second image on the preset feature dimension based on the image description information of the second image, and determining the probability distribution data corresponding to the preset feature dimension based on the attribute value of the plurality of second images on the preset feature dimension.

[0138] According to one or more embodiments of this disclosure, [Example Nine] provides the method of Example Eight, which further includes: optionally, determining the probability distribution data corresponding to the preset feature dimension based on the attribute values ​​of the plurality of second images on the preset feature dimension includes: determining the occurrence frequency of each attribute value of the preset feature dimension in the plurality of second images, and determining the probability distribution data corresponding to the preset feature dimension based on the occurrence frequencies corresponding to the plurality of attribute values.

[0139] According to one or more embodiments of this disclosure, Example 10 provides the method of Example 7, which further includes: optionally, determining the image processing effect of the image processing model based on the probability distribution data of the preset feature dimension includes: determining the entropy of the probability distribution data of the preset feature dimension, normalizing the entropy, and determining the image processing effect of the image processing model based on the normalized entropy.

[0140] According to one or more embodiments of this disclosure, [Example 11] provides an image processing apparatus, comprising: an image determination module, configured to acquire multiple first images and generate multiple second images based on the multiple first images and an image processing model; wherein the image processing model is trained on a first machine learning model based on multiple first sample images and a first expected image corresponding to the first sample images; an image description information determination module, configured to acquire first prompt information and determine image description information for each second image based on the first prompt information, the second images, and an image detection model; wherein the image detection model is trained on a second machine learning model based on multiple second sample images and a second expected image corresponding to the second sample images; the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model; and an image processing effect determination module, configured to determine the image processing effect of the image processing model based on the image description information of the multiple second images.

[0141] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0142] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0143] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire multiple first images; generate multiple second images based on the multiple first images and an image processing model; wherein the image processing model is trained on a first machine learning model based on a first sample image and a first expected image corresponding to the first sample image; acquire first prompt information; determine image description information for each second image based on the first prompt information, the second images, and the image detection model; wherein the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model; and determine the image processing effect of the image processing model based on the image description information of the multiple second images.

[0144] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0146] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not necessarily limiting in certain circumstances; for example, an image determination module can also be described as "a module for acquiring multiple first images".

[0147] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0148] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0150] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0151] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, characterized in that, include: Multiple first images are acquired, and multiple second images are generated based on the multiple first images and an image processing model; wherein, the image processing model is trained on a first machine learning model based on the multiple first sample images and a first expected image corresponding to the first sample images; Obtain first prompt information, and determine image description information for each of the second images based on the first prompt information, the second image, and the image detection model; wherein, the image detection model is trained on a second machine learning model based on multiple second sample images and a second expected image corresponding to the second sample images; the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model; The image processing effect of the image processing model is determined based on the image description information of multiple second images.

2. The image processing method according to claim 1, characterized in that, The acquisition of multiple first images includes: At least one second prompt information is obtained, and multiple third prompt information are generated based on the at least one second prompt information and a text generation model; wherein, the second prompt information includes descriptive data of the object to be described by the text generation model; the text generation model is trained on the third machine learning model based on the first sample prompt information and the expected prompt information corresponding to the first sample prompt information; Multiple first images are generated based on multiple third prompts and an image generation model; wherein the image generation model is obtained by training a fourth machine learning model based on second sample prompts and a third expected image corresponding to the second sample prompts.

3. The image processing method according to claim 2, characterized in that, The step of generating multiple first images based on multiple third prompt messages and an image generation model includes at least one of the following operations: Multiple third prompt messages are input into the image generation model to obtain multiple first images; Multiple third prompt messages and the second prompt messages corresponding to the third prompt messages are input into the image generation model to obtain multiple first images; Multiple fourth prompts are determined from the multiple third prompts, and the multiple fourth prompts are input into the image generation model to obtain multiple first images.

4. The image processing method according to claim 2, characterized in that, The second prompt message is associated with the image processing model; At least one second prompt message is determined based on the model description information; wherein the model description information is used to describe at least the presentation features of the image after processing by the image processing model.

5. The image processing method according to claim 4, characterized in that, The acquisition of at least one second prompt message includes: Obtain the model description information of the image processing model, and generate at least one second prompt information based on the model description information and the text generation model; wherein, the text generation model is trained on the fifth machine learning model based on the third sample prompt information and the second expected prompt information corresponding to the third sample prompt information.

6. The image processing method according to claim 1, characterized in that, The process of obtaining the first prompt information includes: Obtain the model description information of the image processing model, and generate the first prompt information based on the model description information, the fifth prompt information, and the text generation model; wherein, the model description information is used at least to describe the presentation features of the image generated by the image processing model; and the fifth prompt information is used at least to query the detection content of the image output by the image detection model.

7. The image processing method according to claim 1, characterized in that, Determining the image processing effect of the image processing model based on the image description information of multiple second images includes: Based on the image description information of the multiple second images, the probability distribution data of the multiple second images on a preset feature dimension is determined, and the image processing effect of the image processing model is determined based on the probability distribution data of the preset feature dimension.

8. The image processing method according to claim 7, characterized in that, Determining the probability distribution data of the multiple second images on a preset feature dimension based on the image description information of the multiple second images includes: The attribute value of the second image on the preset feature dimension is determined based on the image description information of the second image, and the probability distribution data corresponding to the preset feature dimension is determined based on the attribute values ​​of multiple second images on the preset feature dimension.

9. The image processing method according to claim 8, characterized in that, The step of determining the probability distribution data corresponding to the preset feature dimension based on the attribute values ​​of multiple second images on the preset feature dimension includes: The frequency of occurrence of each attribute value of the preset feature dimension in multiple second images is determined, and the probability distribution data corresponding to the preset feature dimension is determined based on the frequency of occurrence of the multiple attribute values.

10. The image processing method according to claim 7, characterized in that, Determining the image processing effect of the image processing model based on the probability distribution data of the preset feature dimension includes: The entropy of the probability distribution data of the preset feature dimension is determined, and the entropy is normalized. The image processing effect of the image processing model is determined based on the entropy after normalization.

11. An image processing apparatus, characterized in that, include: An image determination module is used to acquire multiple first images and generate multiple second images based on the multiple first images and an image processing model; wherein, the image processing model is trained on a first machine learning model based on a first sample image and a first expected image corresponding to the first sample image; An image description information determination module is used to acquire first prompt information and determine image description information for each of the second images based on the first prompt information, the second image, and the image detection model; wherein, the image detection model is trained on a second machine learning model based on multiple second sample images and a second expected image corresponding to the second sample images; the first prompt information includes descriptive data of the image content to be detected and / or the content to be output by the image detection model; The image processing effect determination module is used to determine the image processing effect of the image processing model based on the image description information of multiple second images.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1-10.

13. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the image processing method as described in any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1-10.

Images