Image processing method and apparatus, computer device, and storage medium

Abstract

The present disclosure provides an image processing method and apparatus, a computer device, and a storage medium. The method comprises: determining a target style type set for a target image to be generated, wherein the target style type corresponds to a plurality of models to be screened; receiving prompt information set under the target style type, and on the basis of the prompt information, selecting a matching target model from the plurality of models to be screened; and generating the target image on the basis of the target model and the prompt information. In embodiments of the present disclosure, on the basis of prompt information input by a user, a matching target model is automatically selected from a plurality of models to be screened corresponding to a target style type selected by the user, and then a target image is generated on the basis of the target model and the prompt information. Only by setting the prompt information and the target style type, the user can automatically select the matching target model for processing, thereby improving the model selection efficiency and the image processing effect.

Description

Image processing method, device, computer equipment and storage medium

[0001] Cross-references

[0002] This application claims priority to Chinese Patent Application No. 202311024439.3 filed on August 14, 2023. The contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0003] The present disclosure relates to an image processing method, apparatus, computer equipment, and storage medium. Background Art

[0004] With the rapid development of artificial intelligence (AI), generative AI (AIGC) has gradually become part of our daily lives. AIGC-based network models can generate content, such as articles, images, and audio, based on input keywords, descriptions, or images, thereby enabling content generation.

[0005] Taking image generation as an example, after entering prompt information, users generally need to select the corresponding AIGC model to generate the corresponding image based on the selected AIGC model and the prompt information. However, since there are many types of AIGC models, the selected AIGC model will directly affect the quality of the generated image.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure at least provide an image processing method, apparatus, computer device, and storage medium.

[0008] An embodiment of the present disclosure provides an image processing method, including:

[0009] Determining a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened;

[0010] receiving prompt information set under the target style type, and selecting a matching target model from the multiple models to be screened based on the prompt information;

[0011] The target image is generated based on the target model and the prompt information.

[0012] In a possible implementation, selecting a matching target model from the multiple models to be screened based on the prompt information includes:

[0013] Determining a scene classification result corresponding to the prompt information; wherein the scene classification result is used to represent the confidence level that the prompt information belongs to each preset scene;

[0014] Based on the scene classification result, a matching target model is selected from the multiple models to be screened.

[0015] In one possible implementation, selecting a matching target model from the multiple models to be screened based on the scene classification result includes:

[0016] Determining the quality scores of the plurality of models to be screened in each of the preset scenarios;

[0017] Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.

[0018] In one possible implementation, selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score includes:

[0019] Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes;

[0020] The model to be screened whose total quality score meets the preset conditions is used as the target model.

[0021] In a possible implementation, determining a scene classification result corresponding to the prompt information includes:

[0022] The prompt information is input into a pre-trained intent recognition model to determine the confidence level that the prompt information belongs to each preset scenario.

[0023] In one possible implementation, before determining the target style type to be set for the target image to be generated, the method further includes:

[0024] Get the image to be processed;

[0025] The generating the target image based on the target model and the prompt information includes:

[0026] The target image is generated based on the target model, the prompt information, and the image to be processed.

[0027] In a possible implementation, determining a scene classification result corresponding to the prompt information includes:

[0028] Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed;

[0029] Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.

[0030] In a possible implementation, the target model includes at least one model;

[0031] The generating the target image based on the target model and the prompt information includes:

[0032] Generate at least one target image based on each target model and the prompt information;

[0033] The method further comprises:

[0034] Acquiring interaction information corresponding to the target image;

[0035] The intent recognition model is adjusted based on the interaction information.

[0036] An embodiment of the present disclosure further provides an image processing apparatus, comprising:

[0037] A determination module, configured to determine a target style type to be set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened;

[0038] a screening module, configured to receive prompt information set under the target style type, and select a matching target model from the plurality of models to be screened based on the prompt information;

[0039] A generating module is used to generate the target image based on the target model and the prompt information.

[0040] In one possible implementation, when selecting a matching target model from the multiple models to be screened based on the prompt information, the screening module is configured to:

[0041] Determining a scene classification result corresponding to the prompt information; wherein the scene classification result is used to represent the confidence level that the prompt information belongs to each preset scene;

[0042] Based on the scene classification result, a matching target model is selected from the multiple models to be screened.

[0043] In one possible implementation, when selecting a matching target model from the multiple models to be screened based on the scene classification result, the screening module is configured to:

[0044] Determining the quality scores of the plurality of models to be screened in each of the preset scenarios;

[0045] Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.

[0046] In one possible implementation, the screening module, when selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score, is configured to:

[0047] Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes;

[0048] The model to be screened whose total quality score meets the preset conditions is used as the target model.

[0049] In one possible implementation, the screening module, when determining the scene classification result corresponding to the prompt information, is configured to:

[0050] The prompt information is input into a pre-trained intent recognition model to determine the confidence level that the prompt information belongs to each preset scenario.

[0051] In one possible implementation, the determination module, before determining the target style type to be set for the target image to be generated, is further configured to:

[0052] Get the image to be processed;

[0053] In one possible implementation, when generating the target image based on the target model and the prompt information, the generation module is configured to:

[0054] The target image is generated based on the target model, the prompt information, and the image to be processed.

[0055] In one possible implementation, the screening module, when determining the scene classification result corresponding to the prompt information, is configured to:

[0056] Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed;

[0057] Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.

[0058] In a possible implementation, the generation module, when generating the target image based on the target model and the prompt information, is configured to:

[0059] In one possible implementation, the device further includes an adjustment module configured to:

[0060] Acquiring interaction information corresponding to the target image;

[0061] The intent recognition model is adjusted based on the interaction information.

[0062] An embodiment of the present disclosure also provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method described in any embodiment of the present disclosure are performed.

[0063] An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method described in any embodiment of the present disclosure are executed.

[0064] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, some embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0066] FIG1 shows a flow chart of an image processing method provided by an embodiment of the present disclosure;

[0067] FIG2 shows a schematic diagram of an image editing interface in the image processing method provided by an embodiment of the present disclosure;

[0068] FIG3 is a schematic diagram showing an image processing method in a cultural image scenario provided by an embodiment of the present disclosure;

[0069] FIG4 shows a schematic diagram of an image processing device provided by an embodiment of the present disclosure; and

[0070] FIG5 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0072] Research has shown that with the rapid development of artificial intelligence (AI), generative AI (AIGC) is gradually becoming part of our daily lives. AIGC-based network models can generate content, such as articles, images, and audio, based on input keywords, descriptions, or images, thereby enabling content generation.

[0073] Taking image generation as an example, after entering prompt information, users generally need to select the corresponding AIGC model to generate the corresponding image based on the selected AIGC model and the prompt information. However, since there are many types of AIGC models, if the user's understanding of the AIGC model is not high, the image generated by the selected AIGC model may not meet the user's needs, thereby affecting the user experience.

[0074] Based on the above research, the present disclosure provides an image processing method, apparatus, computer device, and storage medium. After determining the target style type set for the image to be processed, the method automatically selects a matching target model from multiple models to be screened corresponding to the target style type based on prompt information input by the user. The method then performs processing based on the target model and the prompt information to generate the target image. With this method, the user only needs to set the prompt information and the target style type to automatically select a matching target model for processing, improving model selection efficiency and image processing effectiveness.

[0075] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0076] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0077] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0078] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0079] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0080] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0081] To facilitate understanding of this embodiment, an image processing method disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the image processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities. The computer device includes, for example: a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a vehicle-mounted device, a wearable device, etc.

[0082] 1 , which is a flowchart of an image processing method provided by an embodiment of the present disclosure, the method includes steps 101 to 103 .

[0083] Step 101: Determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened.

[0084] Step 102: Receive prompt information set under the target style type, and select a matching target model from the multiple models to be screened based on the prompt information.

[0085] Step 103: Generate the target image based on the target model and the prompt information.

[0086] The following is a detailed description of steps 101 to 103.

[0087] Regarding step 101:

[0088] The target image may refer to an image to be generated; the target style type may refer to a style type of the target image to be generated; exemplarily, the target style type may include stylized processing, augmented reality (AR) processing, character processing, etc.

[0089] Among them, the stylization processing can be to generate an image with a specific style. For example, it can generate a target image with an animation style or a traditional art style; the AR processing refers to generating an AR image; the character processing refers to generating an image with the characteristics of a certain character, such as generating a sweet-style image of a beautiful woman.

[0090] Each style type may correspond to multiple models to be screened. The models to be screened corresponding to the style type can be understood as the models referenced for implementing this style type. Each model to be screened can process images in multiple scenarios, and the processing quality of the same model to be screened in different scenarios may be different.

[0091] The scenes may exemplarily include portraits, landscapes, animals, oil paintings, 3D styles, etc.

[0092] In another possible implementation, the stylized type may further include a null process. In this case, the to-be-screened modules corresponding to the null process are all optional to-be-screened models.

[0093] For example, if the models to be screened corresponding to the stylized processing are model 1 and model 2, the models to be screened corresponding to the AR processing are model 3 and model 4, and the models to be screened corresponding to the character processing are model 5 and model 6, then the models to be screened corresponding to the empty processing are model 1, model 2, model 3, model 4, model 5, and model 6.

[0094] It should be noted that the models to be screened corresponding to different target style types may be different, partially the same, or completely the same.

[0095] Regarding step 102:

[0096] After determining the target style type set for the target image, since the target style type corresponds to multiple models to be screened, multiple specific processing scenarios can be implemented under the target style type. For example, if the target style type is stylized processing, multiple styles of processing can be implemented under the stylized processing; therefore, the user needs to make further settings to determine what kind of processing to be performed on the image to be processed based on which specific model to be screened.

[0097] Therefore, after determining the target style type set for the image to be processed, prompt information set by the user under the target style type may be received, and based on the prompt information, a matching target model may be selected from the multiple models to be screened.

[0098] The prompt information may be, for example, a complete sentence, or one or more words that can show the user's intention.

[0099] In one possible implementation, when selecting a matching target model from the multiple models to be screened based on the prompt information, the scene classification result corresponding to the prompt information can be first determined; wherein the scene classification result is used to characterize the confidence that the prompt information belongs to each preset scene; and then based on the scene classification result, a matching target model is selected from the multiple models to be screened.

[0100] Here, the prompt information belonging to a preset scene can be understood as the scene contained in the target image to be generated indicated by the prompt information. For example, it can include physical objects contained in the target image to be generated, such as animals and people; or it can include the image type contained in the target image to be generated, such as scenery, oil painting, 3D, etc.

[0101] Optionally, when determining the scene classification result corresponding to the prompt information, a pre-trained intent recognition model can be used. Specifically, the prompt information can be input into the pre-trained intent recognition model to determine the confidence level that the prompt information belongs to each preset scene. The confidence level is used to represent the probability that the prompt information belongs to the preset scene.

[0102] Optionally, when selecting a matching target model from the multiple models to be screened based on the scene classification results, the quality scores of the multiple models to be screened in each of the preset scenes can be determined first, and then a matching target model can be selected from the multiple models to be screened based on the scene classification results and the quality scores.

[0103] Among them, the quality score is used to represent the processing quality of the model to be screened for a certain preset scenario; when determining the quality scores of the multiple models to be screened in each of the preset scenarios, the preset value can be directly used as an example, that is, the quality score of each model to be screened in each preset scenario is artificially pre-set.

[0104] Alternatively, the quality score of each to-be-screened model in each preset scene may be determined based on user interaction information of historical images generated by each to-be-screened model in each preset scene.

[0105] Specifically, images in various preset scenes can be generated based on each model to be screened, and then users can interact with the images, such as collecting, saving, liking, etc.; for any model to be screened, if the user interaction information of the image generated by the model to be screened in a certain preset scene is good (such as the number of likes exceeds the preset number, the number of collections exceeds the preset number, etc.), it can be determined that the image generated by the model to be screened in the preset scene has a high quality.

[0106] Accordingly, when determining the quality score of the model to be screened in the preset scenario, the quality score of the model to be screened in the preset scenario can be determined based on the user interaction information and the preset quality coefficient.

[0107] When selecting a matching target model from the multiple models to be screened based on the scene classification results and the quality scores, it is exemplified that the total quality scores of each model to be screened corresponding to the multiple preset scenes can be determined based on the scene classification results and the quality scores, and the model to be screened whose total quality score meets the preset conditions can be used as the target model.

[0108] Specifically, when determining the total quality score, any one of the following methods may be used:

[0109] Method 1:

[0110] When determining the total quality scores of each model to be screened in multiple preset scenarios based on the scene classification results and quality scores, for any model to be screened, the quality score of the model to be screened in any preset scenario and the confidence that the prompt information belongs to the preset scenario can be multiplied together, and the product obtained is used as the intermediate quality score of the model to be screened in the preset scenario, and the sum of all intermediate quality scores is the total quality score of the model to be screened.

[0111] For example, if the models to be screened are model A and model B, and the preset scenes include animals, portraits, and landscapes, the quality scores of the models to be screened in each preset scene are shown in Table 1 below, and the scene classification results are shown in Table 2 below. The total quality scores of the models to be screened in multiple preset scenes are shown in Table 3 below:

[0112] Table 1

[0113] Table 2

[0114] Table 3

[0115] Method 2:

[0116] Since the confidence levels of the prompt information belonging to various preset scenarios are different in the scene classification results, the preset scenarios with higher confidence levels have higher reference value for the final selection of the model to be screened. Therefore, the total quality score can be calculated based only on the confidence levels of the preset scenarios with higher confidence levels and the quality scores.

[0117] For example, the preset scenes whose corresponding confidence levels are higher than a preset confidence threshold value in the scene classification results can be first screened, and then the confidence levels of the preset scenes screened out can be multiplied by the quality scores corresponding to the models to be screened, and the products can be summed to obtain the total quality score.

[0118] Continuing with the above example, if the quality scores of the models to be screened in each preset scenario are shown in Table 1 above, the scene classification results are shown in Table 2 above, and the preset confidence threshold is 50%, then the preset scenario with a confidence higher than the preset confidence is the animal scene, and the total quality score corresponding to model A is 80*0.9=72, and the total quality score corresponding to model B is 20*0.9=18.

[0119] After the total quality score is calculated based on any of the above methods, a target model can be determined from the multiple models to be screened based on the total quality score.

[0120] For example, after obtaining the total quality scores of the models to be screened, the models to be screened whose total quality scores meet preset conditions can be used as the target models, and the specific screening conditions can be determined according to user settings.

[0121] Exemplarily, the preset condition may be: the total quality score is the highest among all models to be screened, the total quality score is in the top three among all models to be screened, or the total quality score is higher than a preset value, etc., that is, there can be at least one target model that can be screened out.

[0122] Here, it should be noted that the target models are all AI-Generated Content (AIGC) models. Different AIGC models can generate a variety of effects for users to choose from. Through step 102, one or more AIGC models can be determined as target models to generate the target image.

[0123] Regarding step 103:

[0124] When the target image is generated based on the target model and the prompt information, the prompt information may be input into the target model, and the target model may output the target image.

[0125] The target model may include at least one model. When the target image is generated based on the target model and the prompt information, at least one target image may be generated based on each target model and each prompt information.

[0126] Specifically, one target model can generate at least one target image; when there are multiple target models, the number of target images generated by each target model can be the same, for example, each target model generates one image.

[0127] Alternatively, when there are multiple target models, the target models may be sorted based on the total quality scores corresponding to the respective target models, and then the number of target images generated by each target model may be determined based on the sorting result.

[0128] Here, when determining the number of target images generated by each target model based on the sorting results, the total number of target images to be generated can be determined first. The total number can be a pre-set fixed number, or can be a number input by the user. Then, in the sorting results, the higher the ranking of the target model, the fewer target images are generated.

[0129] The total number of the target images is greater than or equal to the number of the target models.

[0130] When the total number of target images is an integer multiple of the number of target models, each target model can generate the same number of target images. For example, if the total number of target images is 10 and the number of target models is 2, each target model can generate 5 target images. When the total number of target images is not an integer multiple of the number of target models, the number of target images produced by each target model can be allocated in sequence according to the sorting result.

[0131] For example, if the total number of target images is 5, the number of target models is 3, and the sorting results are model 1, model 2, and model 3 respectively, then one image can be allocated to model 1, model 2, and model 3 respectively, and then the remaining two images can be allocated to model 1 and model 2 according to the sorting results. That is, after the allocation, model 1 and model 2 produce two target images, and model 3 produces one target image.

[0132] The above embodiment describes a scenario of a text image, that is, in this scenario, the user can select a target style type and input prompt information, and then a target model can be automatically selected, and a target image can be generated based on the target model and the prompt information.

[0133] In another possible scenario, the method provided in the above embodiment can also be applied to a picture-to-picture scenario.

[0134] In one possible implementation, before determining the target style type to be set for the target image to be generated, an image to be processed may be obtained; thus, when generating the target image, the target image may be generated based on the target model, the prompt information, and the image to be processed.

[0135] The image to be processed may refer to an image to be stylized or beautified.

[0136] When acquiring the image to be processed, you can respond to a shooting instruction, call the image acquisition device to shoot, and obtain the image to be processed. You can also receive an image uploaded by the user and use the uploaded image as the image to be processed. Here, the image uploaded by the user can be an image uploaded by the user locally, or it can be an image on the network selected by the user.

[0137] It should be noted that the above-mentioned process of obtaining the image to be processed is obtained after authorization by the user.

[0138] After the image to be processed is acquired, an image editing page may be displayed to the user, and a variety of style types may be displayed on the image editing page for the user to choose from. Different style types produce different effects on image processing.

[0139] Specifically, the image editing interface is shown in Figure 2. The image to be processed can be displayed in the first area, and the type names of multiple style types and the reference images corresponding to the style types can be displayed in the second area. The reference images of multiple style types in the second area support left and right sliding; the image editing interface also includes a third area, in which the user enters prompt information.

[0140] In the image-to-image scenario, the final target image generated may be affected not only by the prompt information but also by the image content of the image to be processed. Therefore, when selecting the target model, in addition to the prompt information, the image content of the image to be processed can also be considered.

[0141] Specifically, content recognition can be performed on the image to be processed to determine a content classification result corresponding to the image to be processed, and then based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information can be determined. Based on the scene classification result, a target model can be determined from multiple models to be screened.

[0142] Here, the performing of content recognition on the image to be processed may, for example, be performed by using a pre-selected and trained content detection network to perform content recognition on the image to be processed and determine a content classification result corresponding to the image to be processed.

[0143] The content classification result corresponding to the image to be processed can be used to characterize the first confidence level of each preset scene contained in the image to be processed; when determining the scene classification result based on the content classification result and the prompt information, it can be based on a pre-trained intention recognition model to determine the second confidence level that the prompt information belongs to each preset scene, and then the second confidence level that the prompt information belongs to each preset scene and the first confidence level that the image to be processed contains each preset scene are weightedly summed to obtain the scene classification result, that is, the confidence level that the prompt information belongs to each preset scene.

[0144] For example, if the first confidence level and the second confidence level are as shown in Table 4 below:

[0145] Table 4

[0146] If the weight corresponding to the first confidence is x and the weight corresponding to the second confidence is y, then the confidence that the prompt information belongs to scene 1 is ax+ly, the confidence that the prompt information belongs to scene 2 is bx+my, and the confidence that the prompt information belongs to scene 3 is cx+ny.

[0147] In another possible implementation, in an image-to-image scenario, the target model can be determined solely based on the image content of the image to be processed. Specifically, after obtaining the image to be processed, a target style type set for the target image to be generated can be determined. Then, based on the image content of the image to be processed, a matching target model can be selected from multiple models to be screened. The image to be processed can then be processed based on the target model to generate the target image.

[0148] In this embodiment, the user does not need to input prompt information, and the target model can directly process the image to be processed. Therefore, the application scenario of this embodiment can be an image processing scenario, such as an image beautification scenario; or, after the image to be processed is input into the target model, the target model can obtain the target image based on the default prompt information.

[0149] In one possible scenario, since the target model is determined by predicting the intent of the prompt information, and during the intent recognition process, the model accuracy of the intent recognition model cannot be completely accurate, there may be a situation where the target model cannot meet user needs after generating the target image. Therefore, in this case, the intent recognition model needs to be optimized.

[0150] Specifically, in the case where the target model includes one model, after generating a target image based on the target model, interaction information corresponding to the target image can be obtained, and then the intention recognition model can be adjusted based on the interaction information.

[0151] The interactive information may be, for example, the user's selection result and its corresponding target model; the user may select only one target image or multiple target images.

[0152] Specifically, when adjusting the intention recognition model based on the interaction information, for example, a set of sample data can be constructed based on the image to be processed, the target style type corresponding to the image to be processed, and the prompt information, and the target model corresponding to the target image in the interaction information is used as the supervision data. The intention recognition model is then trained based on the sample data, and the intention recognition model can output a prediction model selected for the image to be processed. The model parameters of the intention recognition model can then be adjusted based on the supervision data and the prediction model selected for the image to be processed output by the intention recognition model.

[0153] The above image processing method is introduced below in conjunction with specific embodiments.

[0154] See Figure 3, which is a schematic diagram of an image processing method for a cultural image scenario provided by the present disclosure. The user selects the target style type as blank processing and enters the prompt message "A gloomy puppy crouching on the street, 3D and rich in detail." The prompt intent classification algorithm then analyzes the input prompt intent, determining the confidence level that the prompt belongs to each preset scenario. The algorithm then calculates the total quality score for each model and ultimately selects a target model to generate a target image based on the prompt.

[0155] The scenario of generating a graph from a picture is similar to the above, and will not be introduced in detail here.

[0156] The image processing method provided by the disclosed embodiments can, after determining the target style type to be set for the target image to be generated, automatically select a matching target model from multiple models to be screened corresponding to the target style type based on prompt information input by the user. The target image is then generated based on the target model and the prompt information. This method allows the user to automatically select a matching target model for processing, eliminating the need for manual model selection and improving image processing performance.

[0157] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0158] Based on the same inventive concept, an image processing device corresponding to the image processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0159] 4 , which is a schematic diagram of the architecture of an image processing device provided by an embodiment of the present disclosure, the device includes: a determination module 401 , a screening module 402 , and a generation module 403 .

[0160] Determination module 401, for determining a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened;

[0161] A screening module 402 is configured to receive prompt information set under the target style type and select a matching target model from the plurality of models to be screened based on the prompt information;

[0162] The generating module 403 is configured to generate the target image based on the target model and the prompt information.

[0163] In one possible implementation, when selecting a matching target model from the multiple models to be screened based on the prompt information, the screening module 402 is configured to:

[0164] Determining a scene classification result corresponding to the prompt information; wherein the scene classification result is used to represent the confidence level that the prompt information belongs to each preset scene;

[0165] Based on the scene classification result, a matching target model is selected from the multiple models to be screened.

[0166] In one possible implementation, the screening module 402, when selecting a matching target model from the multiple models to be screened based on the scene classification result, is configured to:

[0167] Determining the quality scores of the plurality of models to be screened in each of the preset scenarios;

[0168] Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.

[0169] In one possible implementation, the screening module 402, when selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score, is configured to:

[0170] Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes;

[0171] The model to be screened whose total quality score meets the preset conditions is used as the target model.

[0172] In one possible implementation, the screening module 402, when determining the scene classification result corresponding to the prompt information, is configured to:

[0173] The prompt information is input into a pre-trained intent recognition model to determine the confidence level that the prompt information belongs to each preset scenario.

[0174] In a possible implementation, the determination module 401 is further configured to: obtain an image to be processed before determining the target style type set for the target image to be generated.

[0175] In a possible implementation, the generating module 403, when generating the target image based on the target model and the prompt information, is configured to:

[0176] The target image is generated based on the target model, the prompt information, and the image to be processed.

[0177] In one possible implementation, the screening module 402, when determining the scene classification result corresponding to the prompt information, is configured to:

[0178] Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed;

[0179] Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.

[0180] In a possible implementation, the generating module 403, when generating the target image based on the target model and the prompt information, is configured to:

[0181] In one possible implementation, the device further includes an adjustment module 404 configured to:

[0182] Acquiring interaction information corresponding to the target image;

[0183] The intent recognition model is adjusted based on the interaction information.

[0184] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0185] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Referring to Figure 5, there is a schematic diagram of the structure of the computer device 500 provided in the embodiment of the present disclosure. The computer device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., or various forms of servers, such as independent servers or server clusters. The computer device shown in Figure 5 is only an example and should not bring any limitations to the functions and scope of use of the embodiment of the present disclosure.

[0186] As shown in FIG5 , a computer device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory device (ROM) 502 or a program loaded from a storage device 508 into a random access memory device (RAM) 503. Various programs and data required for the operation of the computer device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0187] Typically, the following devices may be connected to I / O interface 505: input device 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output device 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 508 including, for example, a magnetic tape, hard disk, etc.; and communication device 509. Communication device 509 may allow computer device 500 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 illustrates computer device 500 with various devices, it should be understood that not all of the illustrated devices are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0188] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the image processing method. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0189] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the image processing method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0190] The embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the image processing method described in the above method embodiment. For details, please refer to the above method embodiment and will not be repeated here.

[0191] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0192] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0194] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0195] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0196] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising: Determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened; receiving prompt information set under the target style type, and selecting a matching target model from the multiple models to be screened based on the prompt information; The target image is generated based on the target model and the prompt information.

2. The method according to claim 1, wherein: The selecting a matching target model from the multiple models to be screened based on the prompt information includes: Determine a scene classification result corresponding to the prompt information; wherein the scene classification result is used to characterize the confidence that the prompt information belongs to each preset scene; Based on the scene classification result, a matching target model is selected from the multiple models to be screened.

3. The method according to claim 2, wherein: The selecting a matching target model from the multiple models to be screened based on the scene classification result includes: Determine the quality scores of the plurality of models to be screened in each of the preset scenarios; Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.

4. The method according to claim 3, wherein: The selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score includes: Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes respectively; The model to be screened whose total quality score meets the preset conditions is taken as the target model.

5. The method according to any one of claims 2 to 4, wherein: The determining the scene classification result corresponding to the prompt information includes: The prompt information is input into a pre-trained intent recognition model to determine the confidence that the prompt information belongs to each preset scenario.

6. The method according to claim 2 or 4, wherein: Before determining the target style type set for the target image to be generated, the method further includes: Get the image to be processed; The step of generating the target image based on the target model and the prompt information includes: The target image is generated based on the target model, the prompt information, and the image to be processed.

7. The method according to claim 6, wherein: The determining the scene classification result corresponding to the prompt information includes: Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed; Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.

8. The method according to claim 5, wherein: The target model includes at least one model; The step of generating the target image based on the target model and the prompt information includes: Generate at least one target image based on each target model and the prompt information; The method further comprises: Acquire interaction information corresponding to the target image; The intent recognition model is adjusted based on the interaction information.

9. An image processing device, comprising: A determination module is configured to determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened; A screening module is configured to receive prompt information set under the target style type, and select a matching target model from the multiple models to be screened based on the prompt information; A generating module is configured to generate the target image based on the target model and the prompt information.

10. A computer device comprising: Processor, memory and bus, The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the image processing method as described in any one of claims 1 to 8 are performed.

11. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are executed.

Images