Image processing method and device, computer equipment and storage medium
By automatically selecting the matching AIGC model to generate images, the problem of inefficiency caused by users in the prior art needs to manually select the model, and a more efficient image processing effect is achieved.
Patent Information
- Application Number
- CN202311024439.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-14
- Publication Date
- 2025-02-28
AI Technical Summary
In the prior art, users need to manually select the AIGC model when generating images, resulting in low model selection efficiency and difficult image quality to meet user needs.
By determining the target style type and receiving prompt information input from the user, the matching target model is automatically selected from multiple models to be filtered, and the target image is generated based on the target model and prompt information.
This improves model selection efficiency and image processing effect, allowing users to automatically select the matching target model for processing by simply setting prompt information and target style types.
Smart Images
Figure CN119540402A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing method, apparatus, computer equipment and storage medium. Background Art
[0002] With the rapid development of artificial intelligence technology, artificial intelligence generated content (AIGC) has gradually entered people's daily lives. The network model based on AIGC technology can generate articles, images, audio and other content that match the input content based on input keywords, descriptions or images, thereby realizing the content generation function.
[0003] Taking image generation as an example, after entering the prompt information, the user generally needs to select the corresponding AIGC model to generate the corresponding image based on the selected AIGC model and the prompt information. However, since there are many types of AIGC models, the selected AIGC model will directly affect the quality of the generated image. Summary of the invention
[0004] The embodiments of the present disclosure at least provide an image processing method, apparatus, computer equipment, and storage medium.
[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, including:
[0006] Determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened;
[0007] receiving prompt information set under the target style type, and selecting a matching target model from the multiple models to be screened based on the prompt information;
[0008] The target image is generated based on the target model and the prompt information.
[0009] In a possible implementation, selecting a matching target model from the multiple models to be screened based on the prompt information includes:
[0010] Determine a scene classification result corresponding to the prompt information; wherein the scene classification result is used to characterize the confidence that the prompt information belongs to each preset scene;
[0011] Based on the scene classification result, a matching target model is selected from the multiple models to be screened.
[0012] In a possible implementation, the selecting a matching target model from the multiple models to be screened based on the scene classification result includes:
[0013] Determine the quality scores of the plurality of models to be screened in each of the preset scenarios;
[0014] Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.
[0015] In a possible implementation, the selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score includes:
[0016] Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes respectively;
[0017] The model to be screened whose total quality score meets the preset conditions is taken as the target model.
[0018] In a possible implementation manner, determining the scene classification result corresponding to the prompt information includes:
[0019] The prompt information is input into a pre-trained intent recognition model to determine the confidence that the prompt information belongs to each preset scenario.
[0020] In a possible implementation manner, before determining the target style type set for the target image to be generated, the method further includes:
[0021] Get the image to be processed;
[0022] The step of generating the target image based on the target model and the prompt information includes:
[0023] The target image is generated based on the target model, the prompt information, and the image to be processed.
[0024] In a possible implementation manner, determining the scene classification result corresponding to the prompt information includes:
[0025] Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed;
[0026] Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.
[0027] In a possible implementation manner, the target model includes at least one model;
[0028] The step of generating the target image based on the target model and the prompt information includes:
[0029] Generate at least one target image based on each target model and the prompt information;
[0030] The method further comprises:
[0031] Acquire interaction information corresponding to the target image;
[0032] The intent recognition model is adjusted based on the interaction information.
[0033] In a second aspect, the present disclosure also provides an image processing device, including:
[0034] A determination module, used to determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened;
[0035] A screening module, configured to receive prompt information set under the target style type, and select a matching target model from the multiple models to be screened based on the prompt information;
[0036] A generating module is used to generate the target image based on the target model and the prompt information.
[0037] In a possible implementation, when the screening module selects a matching target model from the multiple models to be screened based on the prompt information, it is configured to:
[0038] Determine a scene classification result corresponding to the prompt information; wherein the scene classification result is used to characterize the confidence that the prompt information belongs to each preset scene;
[0039] Based on the scene classification result, a matching target model is selected from the multiple models to be screened.
[0040] In a possible implementation, the screening module, when selecting a matching target model from the multiple models to be screened based on the scene classification result, is used to:
[0041] Determine the quality scores of the plurality of models to be screened in each of the preset scenarios;
[0042] Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.
[0043] In a possible implementation manner, the screening module, when selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score, is configured to:
[0044] Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes respectively;
[0045] The model to be screened whose total quality score meets the preset conditions is taken as the target model.
[0046] In a possible implementation, the screening module, when determining the scene classification result corresponding to the prompt information, is used to:
[0047] The prompt information is input into a pre-trained intent recognition model to determine the confidence that the prompt information belongs to each preset scenario.
[0048] In a possible implementation manner, the determination module, before determining the target style type set for the target image to be generated, is further configured to:
[0049] Get the image to be processed;
[0050] In a possible implementation manner, when generating the target image based on the target model and the prompt information, the generating module is used to:
[0051] The target image is generated based on the target model, the prompt information, and the image to be processed.
[0052] In a possible implementation, the screening module, when determining the scene classification result corresponding to the prompt information, is used to:
[0053] Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed;
[0054] Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.
[0055] In a possible implementation manner, the generating module, when generating the target image based on the target model and the prompt information, is used to:
[0056] Generate at least one target image based on each target model and the prompt information
[0057] In a possible implementation manner, the device further includes an adjustment module, configured to:
[0058] Acquire interaction information corresponding to the target image;
[0059] The intent recognition model is adjusted based on the interaction information.
[0060] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.
[0061] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.
[0062] The image processing method, apparatus, computer device and storage medium provided by the embodiments of the present disclosure can automatically select a matching target model from a plurality of to-be-screened models corresponding to the target style type based on the prompt information input by the user after determining the target style type set for the target image to be generated, and then generate the target image based on the target model and the prompt information. In this way, the user only needs to set the prompt information and the target style type, and the matching target model can be automatically selected for processing, avoiding the process of manually selecting the model and improving the image processing effect.
[0063] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.
[0065] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure is shown;
[0066] Figure 2 A schematic diagram of an image editing interface in the image processing method provided by an embodiment of the present disclosure is shown;
[0067] Figure 3 A schematic diagram showing an image processing method in a Wensheng image scenario provided by an embodiment of the present disclosure;
[0068] Figure 4 A schematic diagram of an image processing device provided by an embodiment of the present disclosure is shown;
[0069] Figure 5 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0070] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.
[0071] Research has found that with the rapid development of artificial intelligence technology, artificial intelligence generated content (AIGC) has gradually entered people's daily lives. The network model based on AIGC technology can generate articles, images, audio and other content that match the input content based on input keywords, descriptions or images, thereby realizing the content generation function.
[0072] Taking image generation as an example, after entering the prompt information, the user generally needs to select the corresponding AIGC model to generate the corresponding image based on the selected AIGC model and the prompt information. However, since there are many types of AIGC models, if the user's understanding of the AIGC model is not high, the image generated by the selected AIGC model may not meet the user's needs, thereby affecting the user experience.
[0073] Based on the above research, the present disclosure provides an image processing method, apparatus, computer equipment and storage medium, which can automatically select a matching target model from multiple to-be-screened models corresponding to the target style type based on the prompt information input by the user after determining the target style type set for the image to be processed, and then process based on the target model and the prompt information to generate a target image. Through this method, the user only needs to set the prompt information and the target style type, and the matching target model can be automatically selected for processing, thereby improving the model selection efficiency and the image processing effect.
[0074] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0075] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.
[0076] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0077] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0078] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0079] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0080] To facilitate understanding of the present embodiment, an image processing method disclosed in an embodiment of the present disclosure is first introduced in detail. The executor of the image processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities. The computer device includes, for example: a terminal device or a server or other processing device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a vehicle-mounted device, a wearable device, etc.
[0081] See also Figure 1 FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, wherein the method includes steps 101 to 103, wherein:
[0082] Step 101: determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened.
[0083] Step 102: receiving prompt information set under the target style type, and selecting a matching target model from the multiple models to be screened based on the prompt information.
[0084] Step 103: Generate the target image based on the target model and the prompt information.
[0085] The following is a detailed description of the above steps 101 to 103.
[0086] For step 101:
[0087] The target image may refer to an image to be generated; the target style type may refer to a style type of the target image to be generated; exemplarily, the target style type may include stylized processing, augmented reality (AR) processing, character processing, etc.
[0088] Among them, the stylization processing can be to generate an image with a certain specific style. For example, it can generate a target image with an animation style or a traditional art style; the AR processing refers to generating an AR image; the character processing refers to generating an image with certain character characteristics, such as generating a sweet-style image of a beautiful woman.
[0089] Each style type may correspond to multiple models to be screened. The model to be screened corresponding to the style type can be understood as the model referenced to implement this style type; each model to be screened can process images in multiple scenes, and the processing quality of the same model to be screened in different scenes may be different.
[0090] The scenes may exemplarily include portraits, landscapes, animals, oil paintings, 3D styles, etc.
[0091] In another possible implementation, the stylized type may further include empty processing. In this case, the to-be-screened modules corresponding to the empty processing are all optional to-be-screened models.
[0092] For example, if the models to be screened corresponding to the stylized processing are model 1 and model 2, the models to be screened corresponding to the AR processing are model 3 and model 4, and the models to be screened corresponding to the character processing are model 5 and model 6, then the models to be screened corresponding to the empty processing are model 1, model 2, model 3, model 4, model 5, and model 6.
[0093] It should be noted that the models to be screened corresponding to different target style types may be different, partially the same, or completely the same.
[0094] Regarding step 102:
[0095] After determining the target style type set for the target image, since the target style type corresponds to multiple models to be screened, multiple specific processing scenarios can be processed under the target style type. For example, if the target style type is stylized processing, multiple styles of processing can be achieved under the stylized processing; therefore, the user needs to make further settings as to what kind of processing to be performed on the image to be processed based on which specific model to be screened.
[0096] Therefore, after determining the target style type set for the image to be processed, prompt information set by the user under the target style type may be received, and based on the prompt information, a matching target model may be selected from the multiple models to be screened.
[0097] The prompt information may be, for example, a complete sentence, or may be one or more words that can demonstrate the user's intention.
[0098] In one possible implementation, when selecting a matching target model from the multiple models to be screened based on the prompt information, the scene classification result corresponding to the prompt information can be first determined; wherein the scene classification result is used to characterize the confidence that the prompt information belongs to each preset scene; and then based on the scene classification result, a matching target model is selected from the multiple models to be screened.
[0099] Here, the prompt information belonging to a preset scene can be understood as the scene contained in the target image to be generated indicated by the prompt information. For example, it can include physical objects contained in the target image to be generated, such as animals and people; or it can include the image type contained in the target image to be generated, such as scenery, oil painting, 3D, etc.
[0100] Optionally, when determining the scene classification result corresponding to the prompt information, a pre-trained intent recognition model may be used. Specifically, the prompt information may be input into the pre-trained intent recognition model to determine the confidence that the prompt information belongs to each preset scene. The confidence is used to characterize the probability that the prompt information belongs to the preset scene.
[0101] Optionally, when selecting a matching target model from the multiple models to be screened based on the scene classification result, the quality scores of the multiple models to be screened in each of the preset scenes can be first determined, and then a matching target model can be selected from the multiple models to be screened based on the scene classification result and the quality scores.
[0102] Among them, the quality score is used to indicate the processing quality of the model to be screened for a certain preset scene; when determining the quality scores of the multiple models to be screened in each of the preset scenes, the preset value can be directly used as an example, that is, the quality score of each model to be screened in each preset scene is artificially pre-set.
[0103] Alternatively, the quality score of each model to be screened in each preset scene may be determined based on user interaction information of historical images generated by each model to be screened in each preset scene.
[0104] Specifically, images in various preset scenes can be generated based on each model to be screened, and then users can interact with the images, such as collecting, saving, liking, etc.; for any model to be screened, if the user interaction information of the image generated by the model to be screened in a preset scene is good (such as the number of likes exceeds the preset number, the number of collections exceeds the preset number, etc.), it can be determined that the image generated by the model to be screened in the preset scene has a high quality.
[0105] Correspondingly, when determining the quality score of the model to be screened in the preset scenario, the quality score of the model to be screened in the preset scenario can be determined based on the user interaction information and the preset quality coefficient.
[0106] When selecting a matching target model from the multiple models to be screened based on the scene classification results and the quality scores, it is exemplary possible to determine the total quality scores of each model to be screened in the multiple preset scenarios respectively based on the scene classification results and the quality scores, and use the model to be screened whose total quality score meets the preset conditions as the target model.
[0107] Specifically, when determining the total quality score, exemplary methods may be any one of the following:
[0108] Method 1:
[0109] When determining the total quality scores of each model to be screened in multiple preset scenarios based on the scene classification results and quality scores, for any model to be screened, the quality score of the model to be screened in any preset scenario and the confidence that the prompt information belongs to the preset scenario can be multiplied together, and the product obtained is used as the intermediate quality score of the model to be screened in the preset scenario, and the sum of all the intermediate quality scores is the total quality score of the model to be screened.
[0110] Exemplarily, if the models to be screened are model A and model B, and the preset scenes include animals, portraits, and landscapes; the quality scores of each model to be screened in each preset scene are shown in Table 1 below, and the scene classification results are shown in Table 2 below, then the total quality scores of each model to be screened in multiple preset scenes are shown in Table 3 below:
[0111] Table 1
[0112] animal Portrait landscape A Model 80 10 0 B Model 20 100 10
[0113] Table 2
[0114] Preset scenes Confidence animal 0.9 Portrait 0.1 landscape 0.3
[0115] Table 3
[0116] animal Portrait landscape Total quality score A Model 80*0.9 10*0.1 0*0.3 73 B Model 20*0.9 100*0.1 10*0.3 31
[0117] Method 2:
[0118] Since in the scene classification results, the confidence levels of the prompt information belonging to the preset scenes are different, the preset scenes with higher confidence levels have higher reference value for the final selection of the model to be screened. Therefore, the total quality score can be calculated based only on the confidence levels of the preset scenes with higher confidence levels and the quality scores.
[0119] Exemplarily, the preset scenes whose corresponding confidence levels are higher than a preset confidence threshold in the scene classification results may be first screened, and then the confidence levels of the preset scenes screened out may be multiplied by the quality scores corresponding to the models to be screened, and the products may be summed to obtain the total quality score.
[0120] Continuing with the above example, if the quality scores of the models to be screened in each preset scenario are as shown in Table 1 above, the scene classification results are as shown in Table 2 above, and the preset confidence threshold is 50%, then the preset scene with a confidence higher than the preset confidence is the animal scene, and the total quality score corresponding to model A is 80*0.9=72, and the total quality score corresponding to model B is 20*0.9=18.
[0121] After the total quality score is calculated based on any of the above methods, the target model can be determined from the multiple models to be screened based on the total quality score.
[0122] Exemplarily, after obtaining the total quality scores of the models to be screened, the models to be screened whose total quality scores meet preset conditions can be used as the target models, and the specific screening conditions can be determined according to user settings.
[0123] Exemplarily, the preset condition may be: the total quality score is the highest among all the models to be screened, the total quality score is in the top three among all the models to be screened, or the total quality score is higher than a preset value, etc., that is, there can be at least one target model that can be screened out.
[0124] Here, it should be noted that the target models are all AI-Generated Content (AIGC) models, and different AIGC models can generate a variety of effects for users to choose from. Through step 102, one or more AIGC models can be determined as target models to generate a target image.
[0125] Regarding step 103:
[0126] When the target image is generated based on the target model and the prompt information, the prompt information may be input into the target model, and the target model may output the target image.
[0127] The target model may include at least one model. When the target image is generated based on the target model and the prompt information, at least one target image may be generated based on each target model and each prompt information.
[0128] Specifically, one target model can generate at least one target image; when there are multiple target models, the number of target images generated by each target model can be the same, for example, each target model generates one image.
[0129] Alternatively, when there are multiple target models, the target models may be sorted based on the total quality scores corresponding to the target models, and then the number of target images generated by each target model may be determined based on the sorting result.
[0130] Here, when determining the number of target images generated by each target model based on the sorting results, the total number of target images to be generated can be determined first. The total number can be a pre-set fixed number, or can be a number input by the user. Then, in the sorting results, the higher the ranking of the target model, the fewer target images generated.
[0131] The total number of the target images is greater than or equal to the number of the target models.
[0132] When the total number of target images is an integer multiple of the number of target models, each target model can generate the same number of target images. For example, if the total number of target images is 10 and the number of target models is 2, each target model can generate 5 target images. When the total number of target images is not an integer multiple of the number of target models, the number of target images produced by each target model can be allocated in sequence according to the sorting result.
[0133] For example, if the total number of target images is 5, the number of target models is 3, and the sorting results are model 1, model 2, and model 3 respectively, then one image can be allocated to model 1, model 2, and model 3 respectively, and then the remaining two images can be allocated to model 1 and model 2 according to the sorting results. That is, after the allocation, model 1 and model 2 produce two target images, and model 3 produces one target image.
[0134] The above embodiment describes a scene of a text image, that is, in this scene, the user can select a target style type and input prompt information, and then a target model can be automatically selected, and a target image can be generated based on the target model and the prompt information.
[0135] In another possible scenario, the method provided in the above embodiment can also be applied to a picture-to-picture scenario.
[0136] In a possible implementation, before determining the target style type set for the target image to be generated, the image to be processed may be obtained; thus, when generating the target image, the target image may be generated based on the target model, the prompt information, and the image to be processed.
[0137] The image to be processed may refer to an image to be stylized or beautified.
[0138] When acquiring the image to be processed, you can respond to a shooting instruction and call the image acquisition device to shoot to obtain the image to be processed, or you can receive an image uploaded by a user and use the uploaded image as the image to be processed. Here, the image uploaded by the user can be an image uploaded by the user locally, or it can be an image on the network selected by the user.
[0139] It should be noted that the above process of acquiring the image to be processed is performed after authorization by the user.
[0140] After the acquisition of the image to be processed is completed, an image editing page can be displayed to the user, and a variety of style types can be displayed on the image editing page for the user to choose. Different style types produce different effects on image processing.
[0141] Specifically, the image editing interface is as follows: Figure 2As shown, the image to be processed can be displayed in the first area, and the type names of multiple style types and the reference images corresponding to the style types can be displayed in the second area. The reference images of multiple style types in the second area support left and right sliding; the image editing interface also includes a third area, in which the user inputs prompt information.
[0142] In the image-generated image scene, the final generated target image may be affected by the image content of the image to be processed in addition to the prompt information. Therefore, when selecting the target model, in addition to the prompt information, the image content of the image to be processed itself may also be combined.
[0143] Specifically, the content of the image to be processed may be identified to determine the content classification result corresponding to the image to be processed, and then based on the content classification result and the prompt information, the scene classification result corresponding to the prompt information may be determined. Based on the scene classification result, the target model may be determined from multiple models to be screened.
[0144] Here, the performing content recognition on the image to be processed may, for example, be to perform content recognition on the image to be processed through a pre-selected and trained content detection network to determine a content classification result corresponding to the image to be processed.
[0145] The content classification result corresponding to the image to be processed can be used to characterize the first confidence level of each preset scene contained in the image to be processed; when determining the scene classification result based on the content classification result and the prompt information, it can be based on a pre-trained intention recognition model to determine the second confidence level that the prompt information belongs to each preset scene, and then the second confidence level that the prompt information belongs to each preset scene and the first confidence level that the image to be processed contains each preset scene are weightedly summed to obtain the scene classification result, that is, the confidence level that the prompt information belongs to each preset scene.
[0146] Exemplarily, if the first confidence level and the second confidence level are as shown in Table 4 below:
[0147] Table 4
[0148] Scenario 1 Scenario 2 Scene 3 First confidence a b c Second confidence level l m n
[0149] If the weight corresponding to the first confidence is x and the weight corresponding to the second confidence is y, then the confidence that the prompt information belongs to scene 1 is ax+ly, the confidence that the prompt information belongs to scene 2 is bx+my, and the confidence that the prompt information belongs to scene 3 is cx+ny.
[0150] In another possible implementation, in the image-generated-image scenario, when determining the target model, it can be based only on the image content of the image to be processed. Specifically, after obtaining the image to be processed, the target style type set for the target image to be generated can be determined, and then based on the image content of the image to be processed, a matching target model can be selected from multiple models to be screened, and the image to be processed can be processed based on the target model to generate the target image.
[0151] In this embodiment, the user does not need to input prompt information, and the target model can directly process the image to be processed. Therefore, the application scenario of this embodiment can be an image processing scenario, such as an image beautification scenario; or, after the image to be processed is input into the target model, the target model can obtain the target image based on the default prompt information.
[0152] In one possible scenario, since the target model is determined by estimating the intent of the prompt information, and during the intent recognition process, the model accuracy of the intent recognition model cannot be completely accurate, there may be a situation where the target model cannot meet user needs after generating the target image. Therefore, in this case, the intent recognition model needs to be optimized.
[0153] Specifically, in the case where the target model includes a model, after generating a target image based on the target model, interaction information corresponding to the target image can be obtained, and then the intention recognition model can be adjusted based on the interaction information.
[0154] The interactive information may be, for example, the user's selection result and its corresponding target model; the user may select only one target image or multiple target images.
[0155] Specifically, when the intent recognition model is adjusted based on the interaction information, for example, a set of sample data can be constructed by including the image to be processed, the target style type corresponding to the image to be processed, and the prompt information, and the target model corresponding to the target image in the interaction information is used as the supervision data. The intent recognition model is then trained based on the sample data, and the intent recognition model can output a prediction model selected for the image to be processed. The model parameters of the intent recognition model can then be adjusted based on the supervision data and the prediction model selected for the image to be processed output by the intent recognition model.
[0156] The above image processing method is introduced below in conjunction with specific embodiments.
[0157] See also Figure 3As shown, it is a schematic diagram of an image processing method in a scene of a living image provided by the present disclosure. The target style type selected by the user is empty processing, and the prompt information input is "a gloomy puppy squatting on the street, 3D sense, rich details". Then, through the prompt intention classification algorithm, the input prompt is analyzed for intent, and the confidence that the prompt belongs to each preset scene is obtained. Then, the total quality score of each model can be calculated and determined, and finally the target model is selected to generate the target image based on the prompt.
[0158] The scenario of graph generation is similar to the above, and will not be introduced in detail here.
[0159] The image processing method provided by the embodiment of the present disclosure can automatically select a matching target model from a plurality of to-be-screened models corresponding to the target style type based on the prompt information input by the user after determining the target style type set for the target image to be generated, and then generate the target image based on the target model and the prompt information. Through this method, the user only needs to set the prompt information and the target style type, and the matching target model can be automatically selected for processing, avoiding the process of manually selecting the model, and improving the image processing effect.
[0160] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.
[0161] Based on the same inventive concept, an image processing device corresponding to the image processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0162] Reference Figure 4 FIG. 4 is a schematic diagram of the architecture of an image processing device provided by an embodiment of the present disclosure, wherein the device comprises: a determination module 401, a screening module 402, and a generation module 403; wherein,
[0163] A determination module 401 is used to determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened;
[0164] A screening module 402, configured to receive prompt information set under the target style type, and select a matching target model from the multiple models to be screened based on the prompt information;
[0165] The generating module 403 is used to generate the target image based on the target model and the prompt information.
[0166] In a possible implementation, the screening module 402, when selecting a matching target model from the multiple models to be screened based on the prompt information, is configured to:
[0167] Determine a scene classification result corresponding to the prompt information; wherein the scene classification result is used to characterize the confidence that the prompt information belongs to each preset scene;
[0168] Based on the scene classification result, a matching target model is selected from the multiple models to be screened.
[0169] In a possible implementation, the screening module 402, when selecting a matching target model from the multiple models to be screened based on the scene classification result, is configured to:
[0170] Determine the quality scores of the plurality of models to be screened in each of the preset scenarios;
[0171] Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.
[0172] In a possible implementation, the screening module 402, when selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score, is configured to:
[0173] Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes respectively;
[0174] The model to be screened whose total quality score meets the preset conditions is taken as the target model.
[0175] In a possible implementation, the screening module 402, when determining the scene classification result corresponding to the prompt information, is used to:
[0176] The prompt information is input into a pre-trained intent recognition model to determine the confidence that the prompt information belongs to each preset scenario.
[0177] In a possible implementation manner, the determination module 401, before determining the target style type set for the target image to be generated, is further configured to:
[0178] Get the image to be processed;
[0179] In a possible implementation manner, the generating module 403, when generating the target image based on the target model and the prompt information, is configured to:
[0180] The target image is generated based on the target model, the prompt information, and the image to be processed.
[0181] In a possible implementation, the screening module 402, when determining the scene classification result corresponding to the prompt information, is used to:
[0182] Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed;
[0183] Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.
[0184] In a possible implementation manner, the generating module 403, when generating the target image based on the target model and the prompt information, is used to:
[0185] Generate at least one target image based on each target model and the prompt information
[0186] In a possible implementation manner, the device further includes an adjustment module 404, configured to:
[0187] Acquire interaction information corresponding to the target image;
[0188] The intent recognition model is adjusted based on the interaction information.
[0189] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0190] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 5 , which is a schematic diagram of the structure of a computer device 500 provided in an embodiment of the present disclosure. The computer device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc., or various forms of servers, such as independent servers or server clusters. Figure 5 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0191] like Figure 5As shown, the computer device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only storage device (ROM) 502 or a program loaded from a storage device 508 to a random access storage device (RAM) 503. Various programs and data required for the operation of the computer device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0192] Typically, the following devices may be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509. The communication devices 509 may allow the computer device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The computer device 500 is shown with various devices, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.
[0193] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the image processing method. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0194] The present disclosure also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.
[0195] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the image processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0196] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0197] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0198] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0199] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0200] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0201] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. An image processing method, characterized in that: include: Determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened; receiving prompt information set under the target style type, and selecting a matching target model from the multiple models to be screened based on the prompt information; The target image is generated based on the target model and the prompt information.
2. The method according to claim 1, characterized in that The selecting a matching target model from the multiple models to be screened based on the prompt information includes: Determine a scene classification result corresponding to the prompt information; wherein the scene classification result is used to characterize the confidence that the prompt information belongs to each preset scene; Based on the scene classification result, a matching target model is selected from the multiple models to be screened.
3. The method according to claim 2, characterized in that The selecting a matching target model from the multiple models to be screened based on the scene classification result includes: Determine the quality scores of the plurality of models to be screened in each of the preset scenarios; Based on the scene classification result and the quality score, a matching target model is selected from the multiple models to be screened.
4. The method according to claim 3, characterized in that The selecting a matching target model from the multiple models to be screened based on the scene classification result and the quality score includes: Based on the scene classification results and the quality scores, determining the total quality scores of each to-be-screened model in the plurality of preset scenes respectively; The model to be screened whose total quality score meets the preset conditions is taken as the target model.
5. The method according to claim 2, characterized in that: The determining the scene classification result corresponding to the prompt information includes: The prompt information is input into a pre-trained intent recognition model to determine the confidence that the prompt information belongs to each preset scenario.
6. The method according to claim 2 or 4, characterized in that: Before determining the target style type set for the target image to be generated, the method further includes: Get the image to be processed; The step of generating the target image based on the target model and the prompt information includes: The target image is generated based on the target model, the prompt information, and the image to be processed.
7. The method according to claim 6, characterized in that The determining the scene classification result corresponding to the prompt information includes: Performing content recognition on the image to be processed to determine a content classification result corresponding to the image to be processed; Based on the content classification result and the prompt information, a scene classification result corresponding to the prompt information is determined.
8. The method according to claim 5, characterized in that The target model includes at least one model; The step of generating the target image based on the target model and the prompt information includes: Generate at least one target image based on each target model and the prompt information; The method further comprises: Acquire interaction information corresponding to the target image; The intent recognition model is adjusted based on the interaction information.
9. An image processing device, characterized in that: include: A determination module, used to determine a target style type set for a target image to be generated; wherein the target style type corresponds to a plurality of models to be screened; A screening module, configured to receive prompt information set under the target style type, and select a matching target model from the multiple models to be screened based on the prompt information; A generating module is used to generate the target image based on the target model and the prompt information.
10. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are performed.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are executed.