Image content control method, system and device, medium and program product

By generating and displaying the target tags and weights of image content, users can adjust them in the interactive interface, solving the problems of low image generation efficiency and complex operation in the prior art, achieving efficient and accurate image content control, and improving user experience.

CN119027530APending Publication Date: 2024-11-26MIGU CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411073878.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing image content generation method requires the user to input detailed text descriptions and custom weights, resulting in low efficiency in generating images and high requirements for user operation levels, making it difficult for users to intuitively control image content.

Method used

By receiving initial content, the weights of the target image, target label and target label are generated, and displayed in the interactive interface, the user can adjust the label and weight to regenerate the image, intuitively display the proportion of image elements, reduce debugging time, and improve generation efficiency and accuracy .

Benefits of technology

Users do not need a lot of time to debug, and can easily control the image generation effect, improve the efficiency and accuracy of image generation, enhance user experience and participation, and take into account the efficiency and accuracy of image generation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027530A_ABST
    Figure CN119027530A_ABST
Patent Text Reader

Abstract

The invention provides an image content control method, system and device, a medium and a program product, and the method comprises the steps: receiving initial content which comprises an initial image and / or initial characters; generating a target image, a target label and a weight of the target label according to the initial content; and if the user adjusts the weight of the target label, regenerating the target image according to the adjusted weight of the target label. The generated target label and the weight of the target label can visually display the content of the target image, the abstract image content is converted into concrete data representation, the interpretability of the image is enhanced, the target image can be readjusted according to the weight of the target label adjusted by a user, and the user experience is improved. The user can easily control the image generation effect, the image content customized by the user can be accurately generated under the condition that the image generation efficiency is improved, the image generation efficiency and accuracy are both considered, and the user experience feeling and the participation feeling are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image content generation technology, and in particular to an image content control method, system, device, medium, and program product. Background Art

[0002] With the widespread adoption of digital cameras and smartphones, images have become an essential medium for people to receive information in their daily lives. This has also stimulated market demand for large-scale image production. However, the creation of high-quality image content requires authors to have certain photography or drawing skills, as well as the ability to use editing software. This requires a certain level of user experience, and users must have a basic understanding of drawing, light and shadow relationships, and aesthetics to produce high-quality works. With the rapid development of artificial intelligence algorithms, a large number of algorithms and products have emerged in the field of generative algorithms that enable users to create content.

[0003] However, existing methods for generating image content require users to input detailed text descriptions to control the generated image content. Furthermore, specific styles or content often require relatively precise terminology. However, existing solutions require users to constantly customize the input text descriptions and corresponding weights, a process that consumes considerable time and effort. Summary of the Invention

[0004] The embodiments of the present application provide an image content control method, system, device, medium, and program product to solve the problem of how to balance the efficiency and accuracy of image generation.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image content control method, comprising:

[0007] receiving initial content, wherein the initial content includes an initial image and / or initial text;

[0008] Generating a target image, a target label, and a weight of the target label according to the initial content;

[0009] If the user adjusts the weight of the target tag, the target image is regenerated according to the adjusted weight of the target tag.

[0010] Optionally, generating a target image, a target label, and a weight of the target label according to the initial content includes:

[0011] Generating the target tag and the confidence of the target tag according to the initial content;

[0012] Generating a weight of the target tag according to the confidence of the target tag;

[0013] generating the target image according to the target label and the weight of the target label;

[0014] The target image, the target label, and the weight of the target label are displayed on an interactive interface.

[0015] Optionally, generating the weight of the target tag according to the confidence of the target tag includes:

[0016] Removing target tags whose confidences are lower than a threshold from the target tags, and obtaining remaining target tags and confidences of the remaining target tags;

[0017] A weight of the target tag is generated according to the remaining target tags and the confidence of the remaining target tags.

[0018] Optionally, also include:

[0019] If the user adjusts the target label, regenerating the target image according to the adjusted target label and the weight of the adjusted target label includes at least one of the following:

[0020] If the user deletes some target tags, regenerate the target image according to the remaining target tags and the weights corresponding to the remaining target tags;

[0021] If the user adds a new target tag, obtain the weight corresponding to the new target tag, and regenerate the target image based on all target tags after the target tag is added and the weights corresponding to all target tags;

[0022] If the user modifies part of the target label, a target image is generated according to the modified target label and the weight corresponding to the modified target label.

[0023] Optionally, also include:

[0024] If the user adjusts the target label and / or the weight of the target label based on the regenerated target image, the target image is regenerated according to the adjusted target label and / or the adjusted weight of the target label until the user requirements are met.

[0025] Optionally, generating a target image includes:

[0026] Converting the target label and the weight of the target label into a target vector;

[0027] Obtain diffusion model;

[0028] The target vector is input into the diffusion model to generate the target image.

[0029] In a second aspect, an embodiment of the present application provides an image content control system, including:

[0030] A receiving module, configured to receive initial content, wherein the initial content includes an initial image and / or initial text;

[0031] A generation module, configured to generate a target image, a target label, and a weight of the target label according to the initial content;

[0032] The adjustment module is configured to regenerate the target image according to the adjusted weight of the target label if the user adjusts the weight of the target label.

[0033] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor, a memory, and a program stored on the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the image content control method as described in the first aspect above are implemented.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the image content control method as described in the first aspect above are implemented.

[0035] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the image content control method as described in the first aspect.

[0036] In an embodiment of the present application, by receiving initial content, the initial content includes an initial image and / or initial text; based on the initial content, a target image, a target tag, and the weight of the target tag are generated; if the user adjusts the weight of the target tag, the target image is regenerated based on the adjusted weight of the target tag. In an embodiment of the present application, not only can the target image, the target tag of the target image, and the weight of the target tag be automatically generated based on the initial content, the generated target tag and the weight of the target tag can intuitively display the content of the target image, but the target image can also be readjusted based on the weight of the target tag adjusted by the user, so that the generated image content is more in line with user needs. For users, there is no need to control the weights when requesting to generate a target image. They only need to input the initial image and / or initial text, which reduces the time cost of users requesting the target image. Through the displayed target labels and the weights of the target labels, users can form a more intuitive concept of the proportions of different elements in the target image, convert abstract image content into concrete data representations, enhance the interpretability of the image, and make it more intuitive for users in subsequent adjustments. Users can also adjust the weights of the generated target labels to control the image generation results. Therefore, users do not need to spend a lot of time and debugging. Through the image control method of the embodiment of the present application, users can easily control the image generation effect, and can accurately generate user-customized image content while improving image generation efficiency, taking into account both the efficiency and accuracy of image generation, and improving user experience and sense of participation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0038] Figure 1 This is a flowchart of an image content control method provided by an embodiment of the present application;

[0039] Figure 2 is a schematic diagram of an interactive interface of an image content control method provided in an embodiment of the present application;

[0040] Figure 3 This is a flowchart of an embodiment of an image content control method provided by an embodiment of the present application;

[0041] Figure 4 A schematic structural diagram of an image content control device provided in an embodiment of the present application;

[0042] Figure 5This is a schematic diagram of the structure of an image content control system provided by an embodiment of the present application;

[0043] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] Most existing image content generation methods use different models for different image types. Each batch of input control images requires a dedicated model for training, resulting in high time costs and poor usability. Furthermore, some image content generation methods require encoding the input control images as feature vectors, requiring users to use these feature vectors to control image generation. However, these vectors typically lack intuitive semantic content, have poor interpretability, and provide a poor user experience. Other image content generation methods lack user adjustment capabilities and, aside from replacing the initial input image, cannot fine-tune the generated new image during the process.

[0046] Please refer to Figure 1 , an embodiment of the present application provides an image content control method, comprising:

[0047] Step 11: Receive initial content, which includes an initial image and / or initial text. Optionally, the initial image can be an image input by the user according to the user's needs. For example, please refer to Figure 2 The input image in can be used as the initial image; the initial text can be a description of the content and / or scene. For example, the user can enter the initial text: a fox in a flower bush in a field in a comic-style scene, with a lake in the distance.

[0048] Optionally, the initial content may also include an initial image and initial text, which the user can enter. Figure 2 The initial image shown is added with initial text, for example: reducing the proportion of the wilderness in the image.

[0049] Step 12: Generate a target image, a target label, and a weight of the target label based on the initial content;

[0050] Optional, please refer to Figure 2Taking an image as the initial content as an example, the target image, the target label of the target image, and the weight of the target label can be directly output based on the initial image. The automatically generated labels include fox, wilderness, cartoon style, flowers, and lake, with corresponding weights of 1, 1.2, 1.5, 1, and 1, respectively. Optionally, the target image, target label, and the weight of the target label can be displayed on the interactive interface.

[0051] Step 13: If the user adjusts the weight of the target tag, the target image is regenerated according to the adjusted weight of the target tag.

[0052] Optionally, users can adjust the weight of the target label directly on the interactive interface. Furthermore, after adjusting the weight, the effect of the adjustment can be displayed in real time on the interactive interface. Users can improve the image content through continuous interactive modification. Please refer to Figure 2 On the interactive interface, the target label and the weight of the target label are generated according to the initial image input by the user. Based on the generated target label and the weight of the target label, the user can continuously adjust the weight corresponding to each label to make the generated image meet the user's expectations, and the target image after adjusting the weight is displayed in real time.

[0053] In an embodiment of the present application, the initial content includes an initial image and / or initial text; based on the initial content, a target image, a target tag, and the weight of the target tag are generated; if the user adjusts the weight of the target tag, the target image is regenerated based on the adjusted weight of the target tag. In an embodiment of the present application, not only can the initial content automatically generate a target image, a target tag of the target image, and the weight of the target tag, the generated target tag and the weight of the target tag can intuitively display the content of the target image, but the target image can also be readjusted based on the weight of the target tag adjusted by the user, so that the generated image content is more in line with user needs. For users, there is no need to control the weights when requesting to generate a target image. They only need to input the initial image and / or initial text, which reduces the time cost of users requesting the target image. Through the displayed target labels and the weights of the target labels, users can form a more intuitive concept of the proportions of different elements in the target image, convert abstract image content into concrete data representations, enhance the interpretability of the image, and make it more intuitive for users in subsequent adjustments. Users can also adjust the weights of the generated target labels to control the image generation results. Therefore, users do not need to spend a lot of time and debugging. Through the image control method of the embodiment of the present application, users can easily control the image generation effect, and can accurately generate user-customized image content while improving image generation efficiency, taking into account both the efficiency and accuracy of image generation, and improving user experience and sense of participation.

[0054] It is worth noting that the embodiment of the present application also supports receiving the user's adjustment of the target label and the weight of the target label on the interactive interface at the same time, and regenerating the target image according to the adjusted target label and the weight of the target label.

[0055] In the embodiment of the present application, optionally, generating a target image, a target label, and a weight of the target label according to the initial content includes:

[0056] Generating the target tag and the confidence of the target tag according to the initial content;

[0057] Generating a weight of the target tag according to the confidence of the target tag;

[0058] generating the target image according to the target label and the weight of the target label;

[0059] The target image, the target label, and the weight of the target label are displayed on an interactive interface.

[0060] Optionally, the target label and the confidence of the target label are first directly generated through the initial content, and then the average confidence of all target labels is calculated, and the confidence of each target label is divided by the average confidence of all target labels to obtain the weight of each target label.

[0061] Optionally, the confidence of the target label can be in the range of 0 to 1.

[0062] In an embodiment of the present application, using confidence as the basis for weight can more accurately distinguish the importance of different target tags. By dividing the confidence of each target tag by the average confidence of all target tags, a more reliable target tag weight can be generated, and target tags with greater influence can be screened out. Users can more intuitively see which elements have a higher proportion, and can make adaptive modifications according to needs, so that the subsequently generated target image is more in line with user needs.

[0063] It is worth noting that the prior art can only give labels for pictures, and users need to judge the importance of each label by themselves, and how to adjust the weights also requires reference to existing experience or multiple rounds of testing and modification to obtain the desired effect. In the embodiment of the present application, the processed labels and corresponding weights can be displayed to the user through the interactive interface, and the user can adjust the weights in the interactive interface. The embodiment of the present application can also give each label a corresponding confidence level based on the above embodiment, reflecting the importance of different labels. In the embodiment of the present application, when the user adjusts the weight, the adjustment can be made directly based on the weight obtained in the above embodiment, and the effect of the adjustment can be intuitively seen through the interactive interface, giving the initiative to the user, who can make adjustments according to his or her preferences, thereby enhancing the user experience.

[0064] Furthermore, in the embodiment of the present application, optionally, generating the weight of the target tag according to the confidence of the target tag includes:

[0065] Removing target tags whose confidences are lower than a threshold from the target tags, and obtaining remaining target tags and confidences of the remaining target tags;

[0066] A weight of the target tag is generated according to the remaining target tags and the confidence of the remaining target tags.

[0067] Specifically, before calculating the weight of the target tag, the target tags with confidence levels lower than a threshold can be removed, and the weight values ​​of the remaining target tags can be calculated based on the remaining target tags and their confidence levels. By removing target tags with lower confidence levels in advance in the embodiment of the present application, target tags with lower influencing factors are eliminated, and unnecessary information interference is reduced. Moreover, by screening and retaining target tags with higher confidence levels, the waste of computing resources when the target image is subsequently regenerated can be reduced.

[0068] In the embodiment of the present application, optionally, generating the target tag and the confidence level of the target tag according to the initial content includes:

[0069] The initial content is input into the trained multimodal model for encoding processing to generate a target label corresponding to the initial content and a confidence level of the target label corresponding to the initial content. The trained multimodal model is trained based on the training content and the label corresponding to the training content. The training content includes training images and / or training text.

[0070] Specifically, the trained multimodal model is used to process the user's input. When the user inputs a picture (a picture) into the model, the model encodes the picture and processes it to generate multiple labels describing the content of the picture and the confidence of the labels. When the user inputs text into the model, the model encodes the text content and generates multiple labels and the confidence of the labels that match the text content. In an embodiment of the present application, the trained multimodal model can encode both the input images and text, generate the corresponding target labels and the confidence of the target labels corresponding to the initial content, and through multimodal model analysis, the user's intentions can be understood more accurately, more accurate information extraction and understanding can be achieved, and the user experience can be improved.

[0071] The present application further provides an embodiment of training a multimodal model. Optionally, before inputting the initial content into the trained multimodal model for encoding, the method includes:

[0072] Obtaining a label corresponding to the training content from a database of the training content and multimodal data;

[0073] Setting the confidence of the label corresponding to the training content in the multimodal data database to 1, and setting the confidence of other labels corresponding to the multimodal data database to 0;

[0074] A multimodal model is trained according to the training content, the label corresponding to the training content, and the confidence of the label corresponding to the training content to obtain a trained multimodal model.

[0075] Optionally, before using the multimodal model, it is trained using image-label data pairs and text-label data to enable it to generate corresponding labels based on images or text. Specifically, assuming that there are M outputs (corresponding to M labels) in the database of multimodal data, the value corresponding to each output is the confidence corresponding to the label; when training the multimodal model, first obtain the training content (training images or training text) and the N labels contained in the training content, use the training content as the input content of the multimodal model, determine the N labels corresponding to the training content among the M labels as N output contents, set the confidence of the label corresponding to the training content to 1; set the confidence corresponding to other labels to 0, and thus perform multimodal model training to obtain a trained multimodal model. In an embodiment of the present application, the multimodal model is trained by training images and / or training text, and the confidence of the training content and the corresponding label is set to 1, which helps to ensure that the model better understands and learns the relationship between the content and labels of the input data during the training process, improves the performance and generalization ability of the model on various tasks, and the trained multimodal model can better handle different forms of input data in actual applications, enhance interactivity, and improve user experience.

[0076] The present application also provides an embodiment for generating a target image according to a target label and a weight of the target label.

[0077] Optionally, generating a target image includes:

[0078] Converting the target label and the weight of the target label into a target vector;

[0079] Obtain diffusion model;

[0080] The target vector is input into the diffusion model to generate the target image.

[0081] Optionally, a text encoder can be used to convert the labels and corresponding weights into fixed-length vectors, which are then used to control the diffusion model for image generation. It is worth noting that if the user adjusts the weight of the target label, the target image is regenerated based on the adjusted target label weight. Specifically, the target vector obtained by encoding random noise and text is input into the diffusion model, which denoises the noise and outputs an image that matches the text description. Denoising methods include, but are not limited to, denoising diffusion probability models, noise-conditional score networks, and stochastic differential equations.

[0082] Optionally, in addition to adjusting the weight corresponding to the target label separately, the user can also adjust the target label and the weight corresponding to the target label at the same time. Specifically:

[0083] Optionally, also include:

[0084] If the user adjusts the target label, regenerating the target image according to the adjusted target label and the weight of the adjusted target label includes at least one of the following:

[0085] If the user deletes some target tags, regenerate the target image according to the remaining target tags and the weights corresponding to the remaining target tags;

[0086] If the user adds a new target tag, obtain the weight corresponding to the new target tag, and regenerate the target image based on all target tags after the target tag is added and the weights corresponding to all target tags;

[0087] If the user modifies part of the target label, a target image is generated according to the modified target label and the weight corresponding to the modified target label.

[0088] Optionally, in an embodiment of the present application, if the user deletes some labels, the weight of the deleted target label will also become "0", that is, if the user does not delete the weight corresponding to the target label at the same time when deleting the target label, when regenerating the target image, the system will exclude the influence of the weight corresponding to the target label, which is equivalent to removing the deleted label from the current target label set to form a new target label set, ensuring that only the remaining target labels that have not been deleted and their corresponding weights regenerate the target image.

[0089] If the user wants to add a new target label, the weight of the added target label also needs to be obtained. The weight of the added target label can be obtained according to the user input. If the user does not enter the weight of the new target label, a weight can be automatically generated according to the content of the currently generated target image for the user to adjust later. Adding a target label is equivalent to adding the new target label and the corresponding weight to the existing target label set to form a complete set containing all old labels and new labels, and then regenerating the target image based on all target labels and corresponding weights.

[0090] If the user wants to modify the original target label, for example, the original label is "fox" and the user wants to change it to "adult fox", the target image will be regenerated based on the user's modified target label and the corresponding weight.

[0091] It is worth noting that the above-mentioned multiple modification methods can coexist. Users can delete some target tags and then add new target tags or modify the original target tags.

[0092] Optionally, in an embodiment of the present application, the user can adjust the target label and the corresponding weight at the same time, and regenerate the target image based on the adjusted target label and the weight of the adjusted target label. This ensures that after the user modifies the label, the system can dynamically adjust and generate an image that best meets the user's new needs. During the adjustment process, the user can not only adjust based on the existing target label, but also directly adjust the target label. The user has greater initiative, and the generated image is more in line with the user's needs, further enhancing the user experience.

[0093] Optionally, also include:

[0094] If the user adjusts the target label and / or the weight of the target label based on the regenerated target image, the target image is regenerated according to the adjusted target label and / or the adjusted weight of the target label until the user requirements are met.

[0095] It should be noted that after adjusting the weights, the user can see the effect of the weight adjustment in the interactive interface and then adjust the weights again to generate a new image, thereby improving the image content through continuous interactive modification. Figure 2 The labels generated based on the initial image include fox, wilderness, cartoon style, flowers, and lake, with corresponding weights of 1, 1.2, 1.5, 1, and 1 respectively. Users can continuously adjust the weights corresponding to each label on this basis to make the generated image meet user expectations.

[0096] Please refer to Figure 3 In an embodiment of the present application, after the user inputs text or a picture, a label and confidence level are generated by the model (multimodal model), and the weights of different labels are further calculated. The image, label, and weight are displayed on the interactive interface, and the user can modify them on the interactive interface, and a result graph is generated through the diffusion model. The embodiment of the present application lowers the threshold for using the image generation model, while improving the interpretability of the generated results and supporting users to fine-tune the generated results by modifying the label weights, thereby simplifying the complexity of the generation process and making the implementation process more user-friendly, thereby enhancing the user experience.

[0097] Please refer to Figure 3 and Figure 4 In an embodiment of the present application, optionally, a device M0 for implementing the above-mentioned image content control method is further provided:

[0098] It includes a label generation module M1, which is used to process the input initial content (initial text and / or initial image) and generate labels and corresponding weights;

[0099] The label modification module M2 is used to display the target image, the target label of the target image and the weight of the target label to the user, and supports the user to modify the label weight.

[0100] The image generation module M3 is used to generate a target image based on the label and label weight.

[0101] Specifically:

[0102] Label generation module M1: used to execute the relevant logic in step 11 and step 12 of the above method, and can also realize the function of generating labels and confidence levels based on text and picture input. In the embodiment of the present application, according to actual conditions, computing modules such as Pytorch, OpenCV, transformer and their related dependent third-party libraries can be integrated. Among them, Pytorch, transformer and other modules are mainly used for model definition, model training and other steps, specifically for model definition and model training of multimodal models, and OpenCV and other modules are mainly used for picture reading and other steps, specifically for reading pictures input into the multimodal model and converting them into vector forms that can be recognized by the model.

[0103] Label Weight Processing Module M2: This module is used to execute the algorithmic logic for weight display and modification described in the above embodiments. It primarily extracts and computes label weights and displays and modifies them to the user. Depending on the specific method implementation, it may require integration with computational modules such as Pytorch and OpenCV, as well as their associated third-party libraries. The Label Weight Processing Module M2 may include an interactive interface that displays processed labels and their corresponding weights to the user, allowing the user to adjust the weights.

[0104] Image generation module M3: used to execute the calculation logic of generating the target image, realize the function of generating images according to labels and weights, and integrate OpenCV, Pytorch, diffusers and other computing modules and their related dependent third-party libraries according to the specific method implementation. Among them, Pytorch, transformer, diffusers and other modules are mainly used for model definition, model training and other steps, specifically for model definition and model training of the diffusion model, and OpenCV and other modules are mainly used for image display of the image output by the diffusion model and other steps. The diffusion model requires input of random noise and vectors obtained by text encoding. After denoising the noise, the model outputs an image that conforms to the text description. Denoising methods include but are not limited to denoising diffusion probability models, noise conditional scoring networks and stochastic differential equations.

[0105] The image content control device M0 provided in the embodiment of the present application can achieve Figure 1 The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.

[0106] Please refer to Figure 5 The embodiment of the present application further provides an image content control system 20, including:

[0107] The receiving module 21 is configured to receive initial content, wherein the initial content includes an initial image and / or initial text;

[0108] A generating module 22, configured to generate a target image, a target label, and a weight of the target label according to the initial content;

[0109] The adjustment module 23 is configured to regenerate the target image according to the adjusted weight of the target tag if the user adjusts the weight of the target tag.

[0110] Optionally, the generating module 22 includes:

[0111] A generating submodule, configured to generate the target tag and the confidence level of the target tag based on the initial content;

[0112] Generating a weight of the target tag according to the confidence of the target tag;

[0113] generating the target image according to the target label and the weight of the target label;

[0114] The target image, the target label, and the weight of the target label are displayed on an interactive interface.

[0115] Optionally, the generating module 22 further includes:

[0116] A screening module is used to remove target tags whose confidence levels are lower than a threshold value from the target tags, and obtain the remaining target tags and the confidence levels of the remaining target tags;

[0117] A weight of the target tag is generated according to the remaining target tags and the confidence of the remaining target tags.

[0118] Optionally, the image content control system 20 further includes:

[0119] A first adjustment module is configured to regenerate a target image based on the adjusted target label and the weight of the adjusted target label when the user adjusts the target label, including at least one of the following:

[0120] If the user deletes some target tags, regenerate the target image according to the remaining target tags and the weights corresponding to the remaining target tags;

[0121] If the user adds a new target tag, obtain the weight corresponding to the new target tag, and regenerate the target image based on all target tags after the target tag is added and the weights corresponding to all target tags;

[0122] If the user modifies part of the target label, a target image is generated according to the modified target label and the weight corresponding to the modified target label.

[0123] Optionally, the image content control system 20 further includes:

[0124] A second adjustment module is used to regenerate the target image according to the adjusted target label and / or the adjusted weight of the target label until the user requirements are met if the user adjusts the target label and / or the weight of the target label based on the regenerated target image.

[0125] Optionally, the generating module 22 includes:

[0126] An output module, configured to convert the target label and the weight of the target label into a target vector;

[0127] Obtain diffusion model;

[0128] The target vector is input into the diffusion model to generate the target image.

[0129] The image content control system 20 provided in the embodiment of the present application can achieve Figure 1 The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.

[0130] The present application embodiment provides an electronic device 30, see Figure 6 Show, Figure 6 The principle block diagram of the electronic device 30 in an embodiment of the present application includes a processor 31, a memory 32, and a program or instruction stored in the memory 32 and executable on the processor 31. When the program or instruction is executed by the processor, the steps in any one of the image content control methods of the present application are implemented.

[0131] An embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of an embodiment of the image content control method such as any one of the above-mentioned items are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0132] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0133] The present application also provides a computer program product including computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of any one of the embodiments of the image content control method are shown, and can achieve the same technical effect. To avoid repetition, they are not described here.

[0134] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0135] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions disclosed herein comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and maintain the security of user personal information and network security.

[0136] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0138] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for controlling image content, characterized in that: include: Receiving initial content, wherein the initial content includes an initial image and / or an initial text; Generate a target image, a target label, and a weight of the target label according to the initial content; If the user adjusts the weight of the target label, the target image is regenerated according to the adjusted weight of the target label.

2. The method according to claim 1, characterized in that The step of generating a target image, a target label and a weight of the target label according to the initial content includes: Generating the target tag and the confidence of the target tag according to the initial content; Generating a weight of the target tag according to the confidence of the target tag; Generate the target image according to the target label and the weight of the target label; The target image, the target label and the weight of the target label are displayed on an interactive interface.

3. The method according to claim 2, characterized in that Generating the weight of the target tag according to the confidence of the target tag includes: Removing target tags whose confidences are lower than a threshold from the target tags, and obtaining remaining target tags and confidences of the remaining target tags; A weight of the target tag is generated according to the remaining target tags and the confidence of the remaining target tags.

4. The method according to claim 1, characterized in that Also includes: If the user adjusts the target label, regenerating the target image according to the adjusted target label and the weight of the adjusted target label includes at least one of the following: If the user deletes some target tags, regenerate the target image according to the remaining target tags and the weights corresponding to the remaining target tags; If the user adds a new target tag, the weight corresponding to the new target tag is obtained, and the target image is regenerated according to all target tags after the target tag is added and the weights corresponding to all target tags; If the user modifies part of the target label, the target image is generated according to the modified target label and the weight corresponding to the modified target label.

5. The method according to claim 1, characterized in that Also includes: If the user adjusts the target tag and / or the weight of the target tag based on the regenerated target image, the target image is regenerated according to the adjusted target tag and / or the adjusted weight of the target tag until the user requirements are met.

6. The method according to claim 1, characterized in that Generating a target image comprises: Converting the target label and the weight of the target label into a target vector; Get the diffusion model; The target vector is input into the diffusion model to generate the target image.

7. An image content control system, characterized in that: include: A receiving module, used for receiving initial content, wherein the initial content includes an initial image and / or initial text; A generating module, used for generating a target image, a target label and a weight of the target label according to the initial content; The adjustment module is used to regenerate the target image according to the adjusted weight of the target label if the user adjusts the weight of the target label.

8. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the image content control method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the image content control method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the image content control method according to any one of claims 1 to 6.