Image processing method and device, electronic equipment and storage medium

By adjusting the image material attributes according to the user input, the problem of insufficient controllability and editability of image generation in the prior art is solved, and efficient poster design is realized, labor and time costs are reduced, and user experience is improved.

CN119027547APending Publication Date: 2024-11-26BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411304087.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

When the existing technology uses stable diffusion models and large models to generate images, the controllability and editability of the images are insufficient, making it difficult to generate complete posters, especially text layout optimization, and is not friendly to ordinary users and has a large hardware resource overhead. , resulting in high labor and time costs.

Method used

An image processing method is provided, by determining the adjustment prompt text and attribute adjustment information based on the adjustment text input by the user, and adjusting the attributes of the image material, thereby generating the processed image, reducing labor costs and time costs, and improving the user experience.

Benefits of technology

The designated modification of images is realized, which lowers the threshold for image generation, and ordinary users can also adjust exquisite images, improve user experience, and simplify the poster design process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027547A_ABST
    Figure CN119027547A_ABST
Patent Text Reader

Abstract

The invention provides an image processing method, and relates to the technical field of artificial intelligence, in particular to the technical fields of large models, image processing, virtual digital humans and the like. According to the specific implementation scheme, at least one adjustment prompt text and at least one piece of attribute adjustment information are determined according to an adjustment text corresponding to a to-be-processed image, and the to-be-processed image corresponds to at least one to-be-adjusted material; according to at least one piece of attribute adjustment information corresponding to the at least one adjustment prompt text, adjusting attribute information of at least one to-be-adjusted material corresponding to the at least one adjustment prompt text to obtain at least one adjusted material; and obtaining a processed image according to the at least one adjusted material. The invention further provides an image processing device, electronic equipment and a storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the technical fields of large models, image processing and virtual digital humans, and can be applied to image processing, poster image editing, and social media posters, marketing posters, promotional materials and operation poster production. More specifically, the present disclosure provides an image processing method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the application scenarios of large models are constantly expanding. Based on artificial intelligence technology, images can be generated based on the text entered by the user. Summary of the invention

[0003] The present disclosure provides an image processing method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, there is provided an image processing method, the method comprising: determining at least one adjustment prompt text and at least one attribute adjustment information based on an adjustment text corresponding to an image to be processed, wherein the image to be processed corresponds to at least one material to be adjusted; adjusting the attribute information of at least one material to be adjusted corresponding to the at least one adjustment prompt text based on at least one attribute adjustment information corresponding to the at least one adjustment prompt text to obtain at least one adjusted material; and obtaining a processed image based on the at least one adjusted material.

[0005] According to another aspect of the present disclosure, an image processing device is provided, which includes: a determination module, used to determine at least one adjustment prompt text and at least one attribute adjustment information based on the adjustment text corresponding to the image to be processed, wherein the image to be processed corresponds to at least one material to be adjusted; an adjustment module, used to adjust the attribute information of at least one material to be adjusted corresponding to the at least one adjustment prompt text based on the at least one attribute adjustment information corresponding to the at least one adjustment prompt text, so as to obtain at least one adjusted material; and an acquisition module, used to obtain a processed image based on the at least one adjusted material.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an exemplary system architecture to which an image processing method and apparatus can be applied according to an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram of an image processing method according to an embodiment of the present disclosure;

[0014] Figure 4A is a schematic diagram of a visual interface according to one embodiment of the present disclosure;

[0015] Figure 4B is a schematic diagram of an initial image according to an embodiment of the present disclosure;

[0016] Figure 4C is a schematic diagram of a processed image according to an embodiment of the present disclosure;

[0017] Figure 5 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure; and

[0018] Figure 6 is a block diagram of an electronic device to which an image processing method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] Images can include posters. In the operation of an enterprise, publicity is an indispensable part. Posters are an indispensable part of corporate publicity materials. However, in order to design and produce an exquisite poster that meets the requirements, one or more designers with excellent aesthetics and rich experience need to spend a lot of time to complete it, resulting in high labor costs in the poster design process, especially the labor costs of professionals, and also high time costs required for poster production.

[0021] In some embodiments, a stable diffusion model may be used to generate images, or a conversational large model may be used to generate images based on natural language input by the user.

[0022] However, when images are generated using a stable diffusion model or a large model, the controllability and editability of the images are insufficient. Although the function of generating images using artificial intelligence is already very complete, it is still difficult to control the images. In addition, images generated based on stable diffusion models and large models are also difficult to be modified by users. If you want to make specific modifications to the images, you need people with professional capabilities to perform secondary optimization.

[0023] In addition, when using a stable diffusion model to generate images or using a large model to generate images, it is difficult to generate a complete poster, especially difficult to optimize the text layout. The copy content and text layout are important components of the poster. If the layout cannot be done, it will be impossible to produce a complete poster.

[0024] In addition, in order to use the big model to generate posters, it is necessary to input foreign text (such as English text) into the big model and adjust the text according to the specifications corresponding to the big model, which is very unfriendly to ordinary users. In addition, the hardware resources required for poster generation using the big model are relatively expensive and the technology maturity is also relatively low.

[0025] Therefore, in order to efficiently generate images, the present disclosure provides an image processing method, and the system architecture applying the method will be described below.

[0026] Figure 1 is a schematic diagram of an exemplary system architecture to which an image processing method and apparatus can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0027] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptops, desktop computers, etc.

[0029] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0030] It should be noted that the image processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the image processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The image processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0031] It can be understood that the image processing device of the present disclosure is described above, and the method of the present disclosure will be described below.

[0032] Figure 2 is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0033] like Figure 2 As shown, the method 200 may include operations S210 to S230.

[0034] In operation S210, at least one adjustment prompt text and at least one attribute adjustment information are determined according to the adjustment text corresponding to the image to be processed.

[0035] In the embodiment of the present disclosure, the image to be processed may be a generated image. For example, the image to be processed may be a generated poster.

[0036] In the disclosed embodiment, the image to be processed corresponds to at least one material to be adjusted. For example, the image to be processed may be generated using at least one material. Any material in the image to be processed may be used as the material to be adjusted. Taking the image to be processed as a poster as an example, the text, background, etc. of the poster may be used as the material to be adjusted. The attributes of the material may include size, position, color, content, etc.

[0037] In the disclosed embodiment, the adjustment text may be input by the user for the generated image. For example, the adjustment text may be “please help me change the background to a blue color”.

[0038] In the disclosed embodiments, the adjustment prompt text and the attribute adjustment information can be determined in various ways. For example, the adjustment text can be segmented. Based on the result of the segmentation, the adjustment prompt text and the attribute adjustment information are determined. Taking the example of taking the first noun as the adjustment prompt text and determining the attribute adjustment information with the verb and the related words, "background" can be taken as an adjustment prompt text, and the attribute adjustment information can be determined based on "changing to blue".

[0039] In operation S220, attribute information of at least one to-be-adjusted material corresponding to at least one adjustment prompt text is adjusted according to at least one attribute adjustment information corresponding to at least one adjustment prompt text, to obtain at least one adjusted material.

[0040] In the disclosed embodiment, the adjustment prompt text may correspond to one or more property adjustment information. The adjustment prompt text may also correspond to one or more materials to be adjusted. For example, the adjustment prompt text "background" may correspond to the image background material, and also to the property adjustment information determined based on "change to blue". The color of the image background material may be adjusted to blue to obtain the adjusted image background material.

[0041] In operation S230, a processed image is obtained according to the at least one adjusted material.

[0042] In the disclosed embodiment, an image can be generated based on the adjusted material and the unadjusted material to obtain a processed image. Alternatively, the adjusted material can be used to replace the material corresponding to the adjustment prompt text to obtain a processed image.

[0043] Through the disclosed embodiments, the adjustment prompt text and attribute adjustment information are used to adjust the material, thereby achieving the specified modification of the image and reducing the labor cost and time cost required for image generation. In addition, the image can be modified by the adjustment text input by the user, which reduces the threshold for specified modification. Users who do not have professional artistic ability can also adjust beautiful images, which can effectively improve the user experience.

[0044] It can be understood that the above describes the method of adjusting the image to be processed, and the image to be processed of the present disclosure will be described below.

[0045] In some embodiments, the image to be processed can be obtained based on the initial image. For example, the initial image can be used as the image to be processed. The initial image after one or more adjustments can also be used as the image to be processed. Figure 3 Provide explanation.

[0046] Figure 3 is a schematic diagram of an image processing method according to an embodiment of the present disclosure.

[0047] In some embodiments, the above method may further include: generating an initial image according to the input text.

[0048] In the embodiment of the present disclosure, the input text may be the initial text input by the user. Figure 3 As shown, user user30 can enter an initial text. The initial text can be "I want a digital human promotion poster to promote my newly generated digital human."

[0049] In the embodiment of the present disclosure, generating an initial image according to the input text includes: determining at least one initial prompt text according to the input text. Figure 3As shown, in response to receiving the initial text input by the user, at least one initial prompt text can be determined using the large model LLM30. The large model can be a large language model (LLM). The large model can be fine-tuned using multiple sample texts and multiple preset prompt texts. The preset prompt text can be a standardized prompt text (prompt). The sample text can be a historical text input into the large model by a user, or a historical text input into the large model by multiple users with similar attributes, or a text generated based on the historical text input by the user with a high degree of similarity, and the present disclosure does not limit this. The large model can be a conversational large model such as Wenxin Yiyan. Through the embodiment of the present disclosure, using the conversational large model, a short natural language text prompt word can be used to quickly design an image (poster) based on the produced material in the material production platform. The large model is fine-tuned using the preset prompt text, and the preset prompt text can correspond to the identification text of the material, so that the fine-tuned large model can quickly determine the material corresponding to the initial prompt text or the adjusted prompt text from multiple materials.

[0050] In the embodiment of the present disclosure, multiple preset prompt texts correspond to multiple materials of the user. Figure 3 As shown, the material library mdb30 may include multiple materials. The material library mdb30 may be a material library uploaded by the user, or a material library shared by multiple users.

[0051] In the embodiment of the present disclosure, the plurality of materials include at least one image element material. The at least one image element material includes at least one of a text material, an image foreground material, an image background material, and an object material. Figure 3 As shown, the multiple materials in the material library mdb30 may include multiple image element materials. The multiple image element materials include object materials m31, image background materials m32, image foreground materials m33, text materials m34, logo materials m35 and text frame materials m36. The multiple materials may also include image template materials, virtual image materials and lighting materials.

[0052] For another example, a preset prompt text may be determined based on the identification text of the material. The identification text of the material may be the name or type name of the material. Different types of materials may have the same identification text.

[0053] In the disclosed embodiment, the initial prompt text may be determined by the large model from a plurality of preset prompt texts. The initial prompt text may be matched with different types of materials. For example, at least one initial prompt text corresponding to the above initial text may include "digital human of enterprise customer service", "professional style", "promotional poster" and "purple tone". The initial prompt text "professional style" may be matched with one or more image template materials, one or more virtual image materials, and one or more lighting materials.

[0054] It can be understood that the initial prompt text and material library of the present disclosure are described above, and some methods for determining the material to be processed will be described below.

[0055] In the embodiment of the present disclosure, generating an initial image according to the input text includes: determining at least one to-be-processed material corresponding to at least one initial prompt text from a plurality of materials. The plurality of materials may also include at least one image template material. The image template material includes at least one image element attribute information. The attributes indicated by the image element attribute information include at least one of the position, size, color, etc. of the image element material.

[0056] In an embodiment of the present disclosure, determining at least one to-be-processed material corresponding to at least one initial prompt text from a plurality of materials includes: determining an to-be-processed image template material corresponding to at least one initial prompt text from at least one image template material. Determine at least one to-be-processed image element material from at least one image element material based on the to-be-processed image template material. For example, an to-be-processed image template material corresponding to the initial prompt text "professional style" may be determined from a plurality of image template materials. The to-be-processed image template material may include a plurality of image element attribute information such as image background attribute information, object attribute information, and text attribute information. The attributes indicated by the image background attribute information may include the color of the image background. Determine at least one to-be-processed image element material from at least one image element material based on the to-be-processed image template material. Through an embodiment of the present disclosure, an image template may be determined first, so that the image element may be further efficiently determined based on the image template, thereby improving the image generation efficiency.

[0057] In the disclosed embodiment, at least one image element material is multiple. According to the image template material to be processed, determining at least one image element material to be processed from at least one image element material includes: according to the image template material to be processed, determining at least one image element material to be processed corresponding to at least one initial prompt text from multiple image element materials. For example, if the image template material to be processed includes image background attribute information, object attribute information, and text attribute information, the image background material to be processed, the object material to be processed, and the text material to be processed can be determined from the image background material, the object material, and the text material according to the initial prompt text. If the object material to be processed or the text material to be processed cannot be determined according to the initial prompt text, it can be randomly selected from the object material or the text material. For another example, for the object material, it can also be generated according to one or more initial prompt texts, virtual image materials, action materials m37, and lighting materials m38, which will be explained below.

[0058] In the disclosed embodiment, determining at least one to-be-processed material corresponding to at least one initial prompt text from a plurality of materials includes: determining an to-be-processed image material corresponding to at least one initial prompt text from at least one virtual image material. Determining an to-be-processed lighting material corresponding to at least one initial prompt text from at least one lighting material. Obtaining an object material to be processed according to the to-be-processed image material and the to-be-processed lighting material. For example, the plurality of materials may also include an action material m37, a lighting material m38, and a lens material m39. From at least one virtual image material, a to-be-processed virtual image material corresponding to the initial prompt text "Enterprise Customer Service Digital Human" may be determined. From at least one action material, at least one lens material, and at least one lighting material, an to-be-processed action material, a to-be-processed lens material, and a to-be-processed lighting material corresponding to the initial prompt text "Professional Style" may be determined. The to-be-processed lighting material includes a plurality of adjustable lighting attribute values. The plurality of adjustable lighting attribute values ​​include color temperature, hue, brightness, and the like. When generating the initial image, the default attribute value of the to-be-processed lighting material may be used. According to the image material to be processed, the action material to be processed, the lens material to be processed and the lighting material to be processed, an object material to be processed can be rendered. Through the disclosed embodiments, the virtual image material, the lighting material and the like can be used to render and generate the object material, so that the action of the virtual image and the clothing can be more matched, the model penetration can be avoided, the accuracy of image generation and adjustment can be improved, and the user experience can be improved.

[0059] Next, according to the multiple image element attribute information included in the template material to be processed, one or more image element materials to be processed are used to generate an initial image. Next, the process can return to the step of receiving user input text to adjust the image according to the adjustment text input by the user.

[0060] It can be understood that some methods of generating an initial image are described above, and some methods of adjusting an image to be processed obtained from the initial image will be described below.

[0061] In some embodiments, the above method may further include: obtaining at least one of the initial text and the adjusted text input by the user in the visual interface. Displaying the initial image in the visual interface. Figure 4A and Figure 4B Provide explanation.

[0062] Figure 4A is a schematic diagram of a visual interface according to an embodiment of the present disclosure.

[0063] like Figure 4A As shown, the visual interface i40 may include an input box ib40. The user uses the input box ib40 to input an initial text "I want a digital human poster to promote my newly generated digital human". Next, an initial image may be generated based on the initial text and displayed on the visual interface i40.

[0064] Figure 4B is a schematic diagram of an initial image according to an embodiment of the present disclosure.

[0065] like Figure 4B As shown, the visual interface i40 displays an initial image image40. The initial image image can be used as the first level image to be processed. The user can use the input box ib400 to input adjustment text. The adjustment text can be, for example, "Please help me change the background to blue."

[0066] In the embodiment of the present disclosure, determining at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text corresponding to the image to be processed includes: in response to receiving the adjustment text input by the user, using the big model to determine at least one adjustment prompt text and at least one attribute adjustment information. For example, the above-mentioned big model llm30 can be used to determine the adjustment prompt text "background" and the attribute adjustment information "change to blue series". Through the embodiment of the present disclosure, using the conversational big model, according to the input text prompt words, the style and tone of the image (such as a poster) can be changed quickly and in real time.

[0067] In an embodiment of the present disclosure, adjusting the attribute information of at least one material to be adjusted corresponding to at least one adjustment prompt text includes: in response to hitting the material to be adjusted corresponding to the adjustment prompt text from at least one material to be adjusted, displaying the identification text of the material to be adjusted corresponding to the adjustment prompt text on a visual interface. For example, the material to be processed used when generating the initial image image40 can be used as the material to be adjusted. From multiple materials to be adjusted, the material to be adjusted corresponding to the adjustment prompt text "background" can be determined. The material to be adjusted can be an image background material. The text "successfully positioned the background" can be displayed on the visual interface i40 to display the identification text "background" of the material to be adjusted. Through the embodiment of the present disclosure, displaying the identification text of the material corresponding to the adjustment prompt text on the visual interface can enable the user to obtain the standard and normative text describing the material in the image, which helps to improve the efficiency of image adjustment and further improve the user experience.

[0068] In the disclosed embodiment, adjusting the attribute information of at least one material to be adjusted corresponding to at least one adjustment prompt text according to at least one attribute adjustment information corresponding to at least one adjustment prompt text includes: adjusting the attribute information of the material to be adjusted corresponding to the prompt text according to the attribute adjustment information corresponding to the adjustment prompt text. For example, according to the attribute adjustment information "Change to blue" corresponding to the adjustment prompt text "background", the image background material can be adjusted to blue, and a blue image background can be obtained as the adjusted material.

[0069] In the embodiment of the present disclosure, obtaining a processed image according to at least one adjusted material may include: displaying the processed image in a visual interface. Figure 4C Provide explanation.

[0070] Figure 4C is a schematic diagram of a processed image according to an embodiment of the present disclosure.

[0071] like Figure 4C As shown, the processed image image41 can be displayed on the visual interface i40. The background of the processed image image41 is blue.

[0072] It can be understood that the above description uses the example of changing the color of the image background to explain the present disclosure, but the present disclosure is not limited thereto, and will be described below.

[0073] In the disclosed embodiment, the attribute information of the copy material indicates the attribute including the text content. According to the attribute adjustment information corresponding to the copy material, the text content can be adjusted. For example, the words, sentences or paragraphs in the text content can be adjusted, and the font of the words can also be adjusted. Thus, by using the conversational large model, according to the input text prompt words, the text elements, copy or font of the image (such as a poster) can be changed and replaced quickly and in real time.

[0074] In the disclosed embodiment, the attribute information of the material may indicate at least one of the position and size of the material. The size and position of the material may be adjusted according to the corresponding attribute adjustment information. Thus, using the conversational macro model, the position and size of elements in the image (e.g., a poster) may be quickly and in real time changed according to the input text prompt words.

[0075] In the disclosed embodiment, the material corresponding to the adjustment text may be a material randomly selected from a class of materials. The attribute information of the material indicates an attribute including an identifier among multiple materials of the class. According to the attribute adjustment information corresponding to the material, the identifier may be changed to reselect a material from the class of materials to achieve material replacement. The material may also be deleted. Thus, using the conversational macro model, existing elements in the poster may be replaced or deleted quickly and in real time according to the input text prompt words.

[0076] It can be understood that some modes of attribute adjustment are further described above, and the application scenarios of the present disclosure will be further described below.

[0077] In some embodiments, the above method can be used to create posters for social media. For example, using the above method and the conversational mockup, users can quickly obtain interesting and attractive posters for display on social media platforms and convenient daily sharing.

[0078] In some embodiments, the above method can be used to create corporate advertising and marketing posters. For example, using the above method and the conversational large model, relevant personnel of the company can easily create promotional posters to showcase the characteristics of the company's products and brand concept, thereby improving advertising effects and attracting more potential customers.

[0079] In some embodiments, the above method can be used to produce physical corporate promotional materials. For example, in the process of corporate promotion, it is necessary to design and produce a variety of promotional materials, such as postcards, roll-up banners, leaflets, etc. Using the above method and the conversational large model, different sizes can be adjusted based on the generated posters to obtain posters of different sizes as different promotional materials to meet the needs of various users.

[0080] In some embodiments, the above method can be used to create e-commerce operation posters. For example, in the e-commerce industry, the daily operation of a store requires a lot of design materials. These design materials include main pictures, detail pages, store homepages, daily operation activity posters, etc. Using the above method and the conversational big model, a variety of material designs can be performed to complete the feat of one person opening a store.

[0081] Through the disclosed embodiments, in the poster production process, starting from the user's perspective and following the general poster commercial production process, at each key link, by relying on a large model to input text prompts, the user is assisted in quickly obtaining the desired effect for each link.

[0082] In the disclosed embodiment, the large model may be Wenxinyiyan, which may support the user's full Chinese input and provide a more accurate understanding of Chinese semantics.

[0083] It can be understood that the method of the present disclosure is described above, and the device of the present disclosure will be described below.

[0084] Figure 5 is a schematic block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0085] like Figure 5 As shown, the apparatus 500 may include a determination module 510 , an adjustment module 520 , and an acquisition module 530 .

[0086] The determination module 510 is used to determine at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text corresponding to the image to be processed.

[0087] In the embodiment of the present disclosure, the image to be processed corresponds to at least one material to be adjusted.

[0088] The adjustment module 520 is used to adjust the attribute information of at least one to-be-adjusted material corresponding to at least one adjustment prompt text according to at least one attribute adjustment information corresponding to at least one adjustment prompt text, so as to obtain at least one adjusted material.

[0089] The obtaining module 530 is used to obtain a processed image according to at least one adjusted material.

[0090] In some embodiments, the adjustment module includes: an adjustment submodule, which is used to adjust the attribute information of the to-be-adjusted material corresponding to the adjustment prompt text according to the attribute adjustment information corresponding to the adjustment prompt text.

[0091] In some embodiments, the image to be processed is obtained based on the initial image, and the device further includes: a generation module, which is used to generate the initial image based on the input text.

[0092] In some embodiments, the generation module includes: a first determination submodule, configured to determine at least one initial prompt text according to the input text; a second determination submodule, configured to determine at least one to-be-processed material corresponding to the at least one initial prompt text from a plurality of materials; and a generation submodule, configured to generate an initial image according to the at least one to-be-processed material.

[0093] In some embodiments, the multiple materials include at least one image template material and at least one image element material, the image template material includes at least one image element attribute information, and the attributes indicated by the image element attribute information include: at least one of the position and size of the image element material. The at least one image element material includes at least one of a text material, an image foreground material, an image background material, and an object material. The second determination submodule includes: a first determination unit, which is used to determine, from at least one image template material, a to-be-processed image template material corresponding to at least one initial prompt text. A second determination unit, which is used to determine, from at least one image element material, at least one to-be-processed image element material according to the to-be-processed image template material.

[0094] In some embodiments, the at least one image element material is multiple. The second determining unit is further configured to: determine at least one image element material to be processed corresponding to the at least one initial prompt text from the multiple image element materials according to the image template material to be processed.

[0095] In some embodiments, the plurality of materials further include at least one virtual image material and at least one lighting material. The at least one material to be processed includes an object material to be processed. The second determination submodule includes: a third determination unit, configured to determine, from the at least one virtual image material, an image material to be processed corresponding to at least one initial prompt text. A fourth determination unit, configured to determine, from the at least one lighting material, a lighting material to be processed corresponding to at least one initial prompt text. An acquisition unit, configured to obtain an object material to be processed based on the image material to be processed and the lighting material to be processed.

[0096] In some embodiments, the determination module includes: a third determination submodule, configured to determine at least one adjustment prompt text and at least one attribute adjustment information using the large model in response to receiving the adjustment text input by the user.

[0097] In some embodiments, the first determination submodule includes: a fifth determination unit, configured to determine at least one initial prompt text using a large model in response to receiving input text input by a user.

[0098] In some embodiments, the large model is fine-tuned using multiple sample texts and multiple preset prompt texts. The initial prompt text and the adjusted prompt text are determined from the multiple preset prompt texts using the large model.

[0099] In some embodiments, the adjustment module includes: a first display submodule, which is used to display the identification text of the material to be adjusted corresponding to the adjustment prompt text in the visual interface in response to hitting the material to be adjusted corresponding to the adjustment prompt text from at least one material to be adjusted.

[0100] In some embodiments, the acquisition module includes: a second display submodule, used to display the processed image on the visual interface.

[0101] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0102] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0103] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0104] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0105] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0106] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a text processing method and / or a deployment method of a deep learning framework. For example, in some embodiments, the text processing method and / or the deployment method of a deep learning framework may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the text processing method and / or the method for deploying a deep learning framework described above may be performed. Alternatively, in other embodiments, computing unit 601 may be configured to execute the text processing method and / or the method for deploying a deep learning framework in any other appropriate manner (e.g., by means of firmware).

[0107] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0109] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0112] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0114] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing method, include: Determining at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text corresponding to the image to be processed, wherein the image to be processed corresponds to at least one material to be adjusted; According to at least one of the property adjustment information corresponding to at least one of the adjustment prompt texts, adjusting the property information of at least one of the to-be-adjusted materials corresponding to at least one of the adjustment prompt texts, to obtain at least one of the adjusted materials; A processed image is obtained according to at least one of the adjusted materials.

2. The method according to claim 1, in, The step of adjusting the attribute information of at least one of the to-be-adjusted materials corresponding to at least one of the adjustment prompt texts according to at least one of the attribute adjustment information corresponding to at least one of the adjustment prompt texts comprises: According to the property adjustment information corresponding to the adjustment prompt text, the property information of the to-be-adjusted material corresponding to the adjustment prompt text is adjusted.

3. The method according to claim 1, in, The image to be processed is obtained according to the initial image. The method further comprises: The initial image is generated according to the input text.

4. The method according to claim 3, in, Generating the initial image according to the input text comprises: Determining at least one initial prompt text according to the input text; Determining at least one to-be-processed material corresponding to at least one of the initial prompt texts from a plurality of materials; The initial image is generated according to at least one of the materials to be processed.

5. The method according to claim 4, in, The plurality of materials include at least one image template material and at least one image element material, the image template material includes at least one image element attribute information, and the attributes indicated by the image element attribute information include: at least one of the position and size of the image element material; At least one of the image element materials includes at least one of a text material, an image foreground material, an image background material, and an object material; The step of determining at least one to-be-processed material corresponding to at least one of the initial prompt texts from the plurality of materials comprises: Determining, from at least one of the image template materials, an image template material to be processed corresponding to at least one of the initial prompt texts; At least one image element material to be processed is determined from at least one of the image element materials according to the image template material to be processed.

6. The method according to claim 5, in, At least one of the image element materials is multiple; The determining, according to the to-be-processed image template material, at least one to-be-processed image element material from at least one of the image element materials comprises: At least one to-be-processed image element material corresponding to at least one initial prompt text is determined from a plurality of the image element materials according to the to-be-processed image template material.

7. The method according to claim 4, in, The plurality of materials further include at least one virtual image material and at least one lighting material. At least one of the materials to be processed includes an object material to be processed; The step of determining at least one to-be-processed material corresponding to at least one of the initial prompt texts from the plurality of materials comprises: Determining, from at least one of the virtual image materials, an image material to be processed corresponding to at least one of the initial prompt texts; Determining, from at least one of the illumination materials, an illumination material to be processed corresponding to at least one of the initial prompt texts; The object material to be processed is obtained according to the image material to be processed and the lighting material to be processed.

8. The method according to claim 1, in, The determining, according to the adjustment text corresponding to the image to be processed, at least one adjustment prompt text and at least one attribute adjustment information comprises: In response to receiving the adjustment text input by the user, at least one adjustment prompt text and at least one attribute adjustment information are determined using a large model.

9. The method according to claim 4, in, Determining at least one initial prompt text according to the input text comprises: In response to receiving the input text input by the user, at least one of the initial prompt texts is determined using a large model.

10. The method according to claim 8 or 9, in, The large model is obtained by fine-tuning multiple sample texts and multiple preset prompt texts. The initial prompt text and the adjusted prompt text are determined from a plurality of preset prompt texts using the large model.

11. The method according to claim 1, in, The adjusting of the attribute information of at least one of the to-be-adjusted materials corresponding to at least one of the adjustment prompt texts includes: In response to hitting the to-be-adjusted material corresponding to the adjustment prompt text from at least one of the to-be-adjusted materials, an identification text of the to-be-adjusted material corresponding to the adjustment prompt text is displayed on a visual interface.

12. The method according to claim 1, in, The step of obtaining a processed image according to at least one of the adjusted materials comprises: The processed image is displayed in a visual interface.

13. An image processing device, include: A determination module, configured to determine at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text corresponding to the image to be processed, wherein the image to be processed corresponds to at least one material to be adjusted; an adjustment module, configured to adjust the attribute information of at least one of the to-be-adjusted materials corresponding to at least one of the adjustment prompt texts according to at least one of the attribute adjustment information corresponding to at least one of the adjustment prompt texts, so as to obtain at least one of the adjusted materials; The obtaining module is used to obtain a processed image according to at least one of the adjusted materials.

14. The device according to claim 13, in, The adjustment module comprises: The adjustment submodule is used to adjust the attribute information of the to-be-adjusted material corresponding to the adjustment prompt text according to the attribute adjustment information corresponding to the adjustment prompt text.

15. The device according to claim 13, in, The image to be processed is obtained according to the initial image. The device also includes: The generating module is used to generate the initial image according to the input text.

16. The device according to claim 15, in, The generation module comprises: A first determination submodule, configured to determine at least one initial prompt text according to the input text; A second determining submodule is used to determine at least one to-be-processed material corresponding to at least one of the initial prompt texts from a plurality of materials; The generating submodule is used to generate the initial image according to at least one of the materials to be processed.

17. The device according to claim 13, in, The adjustment module comprises: The first display submodule is used for displaying the identification text of the material to be adjusted corresponding to the adjustment prompt text on a visual interface in response to hitting the material to be adjusted corresponding to the adjustment prompt text from at least one of the materials to be adjusted.

18. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

19. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.