Virtual image processing method and device, electronic equipment and storage medium
By entering text to determine the prompt text and using rendering engine and big model technology to generate high-quality virtual images, solving the problems of high threshold and unstable generation effect of existing virtual image design software, and achieving efficient generation and diversification of virtual images design.
Patent Information
- Application Number
- CN202411303984.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-11-08
AI Technical Summary
Existing virtual avatar design software requires professional skills and high thresholds, making it difficult to generate high-quality virtual avatars, especially in terms of clothing design and dynamic video generation, where there are problems of instability and directional modification difficulties.
By entering text, determine the prompt text, obtain the pending material and images, combine the rendering engine and big model technology to generate the target virtual image, lower the user threshold, and improve the rendering effect and artistic expression of the virtual image.
It realizes the generation of high-quality virtual images, reduces the difficulty of user operations, improves the artistic expression and user experience of virtual images, and supports diversified virtual image design and dynamic video generation.
Smart Images

Figure CN118918234A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the technical fields of large models, virtual digital humans, virtual reality and augmented reality, and can be applied to electronic games, advertising and marketing image design, online education and training, online social and entertainment, clothing design, smart homes and virtual assistants, etc. More specifically, the present disclosure provides a virtual image processing method, device, electronic device and storage medium. Background Art
[0002] With the development of artificial intelligence technology, the application scenarios of large models are increasing. Summary of the invention
[0003] The present invention provides a method, device, equipment and storage medium for processing a virtual image.
[0004] According to one aspect of the present disclosure, a virtual image processing method is provided, the method comprising: determining at least one first prompt text and at least one second prompt text based on input text; obtaining a material to be processed and an image to be processed respectively based on the at least one first prompt text and the at least one second prompt text, wherein the material to be processed comprises an area to be processed; adding a plurality of images to be processed to the area to be processed to obtain a target material corresponding to the material to be processed; and obtaining a target virtual image based on the target material.
[0005] According to another aspect of the present disclosure, a virtual image processing device is provided, which includes: a first determination module, used to determine at least one first prompt text and at least one second prompt text according to an input text; a first acquisition module, used to obtain a material to be processed and an image to be processed respectively according to at least one first prompt text and at least one second prompt text, wherein the material to be processed includes an area to be processed; an adding module, used to add a plurality of images to be processed to the area to be processed to obtain a target material corresponding to the material to be processed; and a second acquisition module, used to obtain a target virtual image according to the target material.
[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.
[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.
[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.
[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0011] Figure 1 is a schematic diagram of an exemplary system architecture to which a method and apparatus for processing a virtual image can be applied according to an embodiment of the present disclosure;
[0012] Figure 2 is a flow chart of a method for processing a virtual image according to an embodiment of the present disclosure;
[0013] Figure 3 is a schematic diagram of determining a prompt text according to an embodiment of the present disclosure;
[0014] Figure 4A is a schematic diagram of a method for processing a virtual image according to an embodiment of the present disclosure;
[0015] Figure 4B is a schematic diagram of an image to be processed according to an embodiment of the present disclosure;
[0016] Figure 5A is a schematic diagram of a visual interface according to one embodiment of the present disclosure;
[0017] Figure 5B is a schematic diagram of a target virtual image according to an embodiment of the present disclosure;
[0018] Figure 6 is a schematic diagram of an adjusted avatar according to an embodiment of the present disclosure;
[0019] Figure 7 is a block diagram of a virtual image processing device according to an embodiment of the present disclosure; and
[0020] Figure 8 is a block diagram of an electronic device to which a virtual image processing method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0022] The production of virtual images requires professional designers to perform modeling, binding and other operations in modeling software.
[0023] In some embodiments, some software products include a rich material library and a virtual image library, providing topological structure data, coordinate data, and multiple editors for the virtual image. Designers can stack various materials to provide realistic skin details, makeup effects, and texture effects for the virtual image. These software products are widely used in digital art creation and animation production. However, these software products require users to have more professional three-dimensional production capabilities and be familiar with various functions in the rendering engine, and the threshold is relatively high. Although these software products provide a material library, users are still required to manually set it according to their needs, which requires a certain art foundation.
[0024] In some embodiments, some software products have powerful image processing functions. Image processing functions include, for example, a virtual human fitting function. Based on this function, users can easily generate pictures of trying on clothes, which provides a good user experience for clothing e-commerce and bloggers, greatly simplifies the product display process and improves efficiency. However, these software products only support image generation, and the generation effect is unstable. It is difficult to support directional modification or adjustment, and the generated content is difficult to be repeatedly called, and it is also difficult to generate continuous dynamic videos.
[0025] Therefore, in order to generate a high-quality virtual image, the present disclosure provides a virtual image processing method, and the system architecture of the method will be described below.
[0026] Figure 1 is a schematic diagram of an exemplary system architecture to which a method and apparatus for processing a virtual image can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.
[0027] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.
[0028] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptops, desktop computers, etc.
[0029] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.
[0030] It should be noted that the virtual image processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the virtual image processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The virtual image processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the virtual image processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.
[0031] It can be understood that the above describes the system architecture of the present disclosure, and the following will describe the method of the present disclosure.
[0032] Figure 2 is a flowchart of a method for processing a virtual image according to an embodiment of the present disclosure.
[0033] like Figure 2 As shown, the method 200 may include operations S210 to S240.
[0034] In operation S210, at least one first prompt text and at least one second prompt text are determined according to the input text.
[0035] In the disclosed embodiment, the input text may be text input by a user. For example, the input text may be "Help me match a set of clothes suitable for outdoor activities in spring, with a girlish feel".
[0036] In the disclosed embodiment, according to the input text, the first prompt text and the second prompt text can be determined in various ways. The first prompt text can be, for example, text related to clothing. The second prompt text can be text related to texture or pattern. The input text can be split to obtain multiple word segments. The first prompt text and the second prompt text are determined from multiple word segments. For example, according to the word segment "outdoor", the first prompt text "sports outdoor" can be determined. Based on the first prompt text, it can be determined that the clothing material is a sweater or a jacket. According to the word segment "girl", the second prompt text "cute" can be determined. The material corresponding to the second prompt text can be a cartoon texture material or a fruit texture material. The texture material can be a vector graphic.
[0037] In operation S220, a to-be-processed material and an to-be-processed image are obtained according to at least one first prompt text and at least one second prompt text.
[0038] For example, according to the first prompt text "sports and outdoor", an initial clothing material can be randomly determined from a plurality of initial clothing materials such as a sweater material or a jacket material, and rendered based on the initial clothing material to obtain a material to be processed.
[0039] In the embodiment of the present disclosure, the material to be processed includes a region to be processed. For example, any layer of the material to be processed can be used as the region to be processed.
[0040] For example, according to the second prompt text "cute", an initial texture material can be randomly determined from a plurality of initial texture materials such as cartoon texture materials or fruit texture materials, and a combination is performed based on the initial texture material to generate an image to be processed.
[0041] In operation S230, a plurality of to-be-processed images are added to the to-be-processed region to obtain a target material corresponding to the to-be-processed material.
[0042] For example, multiple images to be processed may be tiled to the area to be processed to obtain the target material. The area to be processed may have a boundary. If one or more images to be processed exceed the boundary of the area to be processed, the image area exceeding the boundary may be deleted.
[0043] In operation S240, a target avatar is obtained according to the target material.
[0044] For example, a target virtual image may be rendered based on the target material and the portrait material.
[0045] Through the embodiment of the present disclosure, the material to be processed is obtained according to the first prompt text, and the image determined according to the second prompt text is added to the material to be processed, so that patterns or textures corresponding to the text can be added to the virtual image, thereby improving the rendering effect and artistic expression of the virtual image, lowering the threshold for users to obtain virtual images, and improving user experience.
[0046] It can be understood that the above describes the method of the present invention, and some ways of determining the first prompt text and the second prompt text of the present invention will be described below.
[0047] In some embodiments, in some implementations of the above operation S210, determining at least one first prompt text and at least one second prompt text according to the input text may include: determining at least one first prompt text, at least one first attribute information corresponding to the at least one first prompt text, at least one second prompt text, and at least one second attribute information corresponding to the at least one second prompt text according to the input text. For example, the first prompt text, the second prompt text, the first attribute information, and the second attribute information may be determined using a lightweight model. Figure 3 Provide explanation.
[0048] Figure 3 It is a schematic diagram of determining prompt text according to an embodiment of the present disclosure.
[0049] like Figure 3 As shown, the input text input30 can be "Help me match an outfit suitable for outdoor spring, with a girly feel. The color matching should not be too tacky, and there should not be too many accessories, simple and elegant."
[0050] The lightweight model mdoel30 can disassemble the input text input30. For example, according to the identifier of the preset task and multiple preset prompt texts corresponding to the preset task, the lightweight model model30 can disassemble the input text input30. The preset tasks may include scene determination tasks, style type determination tasks, trend color determination tasks, and accessories determination tasks. The preset prompt texts corresponding to the scene determination tasks may include "sports and outdoor", "indoor", etc. The preset prompt texts corresponding to the style type determination tasks may include "cute", "vicissitudes", etc. The identifier of the trend color determination task may be "trendy color matching". The preset prompt texts corresponding to the accessories determination task may include "no accessories", "with accessories", etc.
[0051] In an embodiment of the present disclosure, the input text is disassembled to obtain multiple initial disassembly results. The multiple initial disassembly results may include "outdoors", "girls", "don't wear too vulgar colors", and "don't wear too many accessories". The initial disassembly results are adjusted to remove words or phrases that are semantically irrelevant or have low relevance to obtain disassembly results to be processed. Multiple disassembly results to be processed may include "outdoors", "girls", "good colors", and "no accessories". The disassembly result to be processed "outdoors" can be matched with the preset prompt text "sports and outdoor". The disassembly result to be processed "girls" can be matched with the preset prompt text "cute". The disassembly result to be processed "good colors" can be matched with the label "trendy colors". The disassembly result to be processed "don't wear accessories" can be matched with the preset prompt text "no accessories".
[0052] In the embodiment of the present disclosure, the preset prompt text that matches the pending disassembly result can be determined as the initial prompt text. For example, the preset prompt texts "sports and outdoor", "cute", and "no accessories" can be used as initial prompt texts respectively. The pending disassembly result "good color matching" hits the identification of the trend color matching determination task. For the trend color matching determination task, the preset prompt text "random call" can be determined as the initial prompt text. Through the embodiment of the present disclosure, randomness is introduced, and a variety of prompt texts can be generated, and then different materials can be determined to generate a unique virtual image.
[0053] In the disclosed embodiment, in response to determining that the initial prompt text corresponds to the preset material, the initial prompt text is used as the first prompt text. For example, the preset prompt text "sports outdoor" corresponds to a preset material such as a sweater or a jacket, and the preset prompt text "sports outdoor" can be used as the first prompt text.
[0054] In the disclosed embodiment, at least one first attribute information corresponding to the first prompt text can be determined according to at least one first prompt text. For example, according to a preset corresponding relationship, multiple attribute information info31 can be determined according to at least one first prompt text and at least one initial prompt text. The multiple attribute information includes first attribute information and initial attribute information. For the scene determination task, the first attribute information corresponding to the first prompt text "sports outdoor" can include "outdoor sports, short hair or tied hair, sweater or jacket". For the style type determination task, the initial attribute information corresponding to the initial prompt text "cute" can include "girlish cute sweet style, pink as the base tone, randomly calling cartoon or fruit patterns". For the trend color determination task, the initial attribute information corresponding to the initial prompt text "random call" can include "randomly calling dopamine color or macaron color". For the accessories determination task, the initial attribute information corresponding to the initial prompt text "no accessories" can include "give priority to removing accessories, or adding a small amount of accessories". It can be understood that the first attribute information can be the first attribute text. The initial attribute information can be the initial attribute text. The text can be accurately and efficiently converted into task data.
[0055] In the disclosed embodiment, the second prompt text can be determined according to the initial prompt text corresponding to the style type determination task. For example, as described above, according to the preset prompt text "cute", the initial attribute information corresponding to the style type determination task is determined as "girlish, cute and sweet style, pink as the base tone, randomly calling cartoon or fruit patterns". Based on the initial attribute information, a cartoon pattern or a fruit pattern can be determined to generate a pattern. The initial prompt text "cute" can be used as the second prompt text.
[0056] In the embodiment of the present disclosure, the second attribute information can be determined according to the second prompt text. For example, the second attribute information may include "select various fruits such as strawberries". It can be understood that the second attribute information can be a second attribute text. It can be understood that the second attribute information is mainly used to generate patterns, and the style type determines that the initial attribute information corresponding to the task can be related to the overall style of the virtual image. Through the embodiment of the present disclosure, the prompt text is determined according to the user's input text, and the attribute information corresponding to the prompt text is determined, which realizes the expansion of the user input and can generate a virtual image with richer details and more realistic. Based on the attribute information, virtual image generation can be performed efficiently.
[0057] It can be understood that the above describes the prompt text and attribute information of the present disclosure, and some methods for determining the material to be processed and the pattern to be processed will be described below.
[0058] Figure 4A is a schematic diagram of a virtual image processing method according to an embodiment of the present disclosure.
[0059] like Figure 4A As shown, user user40 may provide the input text input30 to lightweight model model40. Lightweight model model40 may determine the plurality of attribute information info30.
[0060] In some embodiments, in some implementations of the above operation S220, obtaining the material to be processed and the image to be processed respectively according to at least one first prompt text and at least one second prompt text includes: obtaining the material to be processed according to at least one first prompt text and obtaining the image to be processed according to at least one second prompt text.
[0061] In an embodiment of the present disclosure, obtaining the material to be processed according to at least one first prompt text may include: determining at least one first task to be executed according to at least one first attribute information corresponding to at least one first prompt text. For example, according to at least one first attribute information corresponding to at least one first prompt text, at least one first task to be executed is determined using the first large model llm41. The first large model llm41 may be a large language model (e.g., Wen Xin Yi Yan). At least one first task to be executed may correspond to the above-mentioned scenario determination task. The first task data of the first task to be executed is determined by the first attribute information.
[0062] In the disclosed embodiment, obtaining the image to be processed according to at least one second prompt text includes: determining at least one second task to be performed according to at least one second attribute information corresponding to the at least one second prompt text. For example, the above-mentioned second prompt text and the above-mentioned second attribute information can also be provided to the first large model. The first large model llm41 can determine at least one second task to be performed according to at least one second attribute information. The second task data of the second task to be performed can be determined by the second attribute information, and can also be determined according to the input field of the second large model.
[0063] In the disclosed embodiment, the second large model may be a visual large model. The visual large model may be a midjounery large model or a Wenxin·CV large model. The input fields of the visual large model include fields related to image size, fields related to image generation mode, flag parameter (tile) fields, and fields related to prompt text. The value of the field related to image size may, for example, indicate that the image is a 1:1 image. The value of the flag parameter field is used to indicate the generation of continuous images. The value of the field related to the prompt text may correspond to the second attribute information. The value of the input field may be the second task data determined by the first large model. It can be understood that the value of the field related to image size, the value of the field related to image generation mode, and the value of the flag parameter (tile) field may be preset values. The preset value may be used as the second task data.
[0064] In the embodiment of the present disclosure, at least one initial task to be performed can be determined based on at least one initial attribute information corresponding to at least one initial prompt text. For example, based on at least one initial attribute information corresponding to at least one initial prompt text, at least one initial task to be performed is determined using the first large model llm41. The at least one initial task to be performed corresponds to the above-mentioned style type determination task, trend color determination task, and accessories determination task. The initial task data of the initial task to be performed is determined by the initial attribute information. Through the embodiment of the present disclosure, the large language model can efficiently determine the tasks to be performed, and can give full play to the generation and understanding capabilities of the large model.
[0065] In the disclosed embodiment, the first large model may provide the first task to be executed and the initial task to be executed to the program end prog40 integrated with the rendering engine e40, and may also provide the second task to be executed to the second large model lvm42.
[0066] In the disclosed embodiment, according to at least one first prompt text, obtaining the material to be processed includes: executing at least one first task to be executed to obtain the material to be processed. For example, the first task data may indicate an initial clothing material. The initial clothing material may be, for example, a sweater material. The program end prog40 is used to determine one or more first instructions corresponding to the first task to be executed. Next, the rendering engine e40 executes one or more instructions to adjust the attributes of the initial clothing material indicated by the first task data according to the first attribute information to obtain the sweater material to be processed.
[0067] In the disclosed embodiment, at least one initial task to be executed can be executed to obtain an adjusted initial material. For example, the initial task data can indicate an initial material. The program terminal prog40 is used to determine one or more initial instructions corresponding to the initial task to be executed. The rendering engine is used to execute one or more initial instructions to adjust the properties of the initial material indicated by the initial task data according to the initial attribute information to obtain the adjusted initial material.
[0068] In the embodiment of the present disclosure, obtaining the image to be processed according to at least one second prompt text includes: executing at least one second task to be executed to obtain the image to be processed. For example, the image to be processed image40 can be obtained according to the task data of the second task to be executed using the second large model. When the image size is 1:1, the image to be processed image40 can be a four-sided continuous map. Figure 4B Provide explanation.
[0069] Figure 4B is a schematic diagram of an image to be processed according to an embodiment of the present disclosure.
[0070] like Figure 4B As shown, the image to be processed image40 may include a plurality of “strawberries”.
[0071] Through the embodiments of the present disclosure, conditional generation is achieved by effectively utilizing prompt text and corresponding attribute information, and a virtual image, pattern, and texture corresponding to the input text can be generated, which can efficiently meet the user's intentions and needs.
[0072] It can be understood that the above describes the method of determining the material to be processed and the image to be processed, and the following describes the method of determining the target material.
[0073] In some embodiments, in some implementations of the above operation S230, adding multiple images to be processed to the area to be processed to obtain the target material corresponding to the material to be processed includes: determining the first attribute adjustment information of each of the multiple images to be processed and the target position information of each of the multiple images to be processed according to the coordinate data of the material to be processed. For example, the material to be processed may be a sweater material to be processed. The target position information may indicate a sub-area to be processed in the area to be processed. The first attribute adjustment information may be determined based on the coordinate data (UV) of the image to be processed and the material to be processed in three-dimensional space.
[0074] In some embodiments, in some implementations of the above operation S230, adding multiple images to be processed to the area to be processed to obtain a target material corresponding to the material to be processed includes: adjusting the multiple images to be processed respectively according to the first attribute adjustment information of each of the multiple images to be processed to obtain multiple adjusted images. For example, based on the first attribute adjustment information, at least one of the overall size and the size of each side of the image to be processed can be adjusted to obtain the adjusted image.
[0075] In some embodiments, in some implementations of the above operation S230, adding multiple images to be processed to the area to be processed to obtain the target material corresponding to the material to be processed includes: adding multiple adjusted images to multiple sub-areas to be processed respectively according to the target position information of each of the multiple images to be processed to obtain the target material. For example, multiple adjusted images are added to multiple sub-areas to be processed on the sweater material to be processed to obtain the target material. Through the embodiments of the present disclosure, the image generated by the second prompt text can be adjusted so as to generate a virtual image with the material determined by the first prompt text, thereby improving the visual effect of the virtual image's clothing, making the virtual image more exquisite, and fully improving the user experience.
[0076] In some embodiments, in some implementations of the above operation S240, a target virtual image may be obtained according to the target material and the adjusted initial material.
[0077] It can be understood that the method of the present invention is described above, and the above-mentioned first large model will be further described below.
[0078] In some embodiments, the first large model is obtained by fine-tuning multiple sample texts and multiple preset prompt texts. The sample text can be a historical text input into the large model by a user, or a historical text input into the large model by multiple users with similar attributes, or a text generated based on the historical text input by the user with high similarity, and the present disclosure does not limit this. The large model can be a conversational large model such as Wenxin Yiyan. The first prompt text, the second prompt text, and the initial prompt text can also be determined from multiple preset prompt texts using the first large model. Through the embodiment of the present disclosure, using the conversational large model, a short natural language text prompt word can be used to quickly generate a virtual image based on the produced material in the material production platform. The large model is fine-tuned using the preset prompt text, so that the large model can be more suitable for the generation of digital people, especially digital people with different creations. The preset prompt text can correspond to the identification text and attribute field of the material in the material library db40, so that the fine-tuned large model can quickly determine the material corresponding to the first prompt text, the second prompt text, and the initial prompt text from multiple materials. The attribute field can include a color attribute field, an action attribute field, and a scaling and rotation attribute field. The large model can determine the identification and attribute fields of the material. The rendering engine e40 can obtain the data of the material from the material library db40 based on the identification of the material to display the material in the visual interface.
[0079] It can be understood that the above describes some ways of obtaining the target virtual image in the present disclosure. In the embodiment of the present disclosure, the virtual image can be displayed using a visual interface, which will be described below.
[0080] Figure 5A is a schematic diagram of a visual interface according to an embodiment of the present disclosure.
[0081] like Figure 5A As shown, the visual interface i50 may include an input box ib50. The user may use the input box ib50 to input the input text input30. The visual interface i50 may display a reference virtual image vp50. Next, the above operations S210 to S240 may be performed to obtain a target virtual image. Next, the target virtual image may be displayed on the visual interface.
[0082] Figure 5B is a schematic diagram of a target virtual image according to an embodiment of the present disclosure.
[0083] like Figure 5BAs shown, the visual interface i50 can display the target virtual image vp51. The clothing of the target virtual image vp51 includes an image with adjusted size. The image can be called a pattern. Through the embodiment of the present disclosure, using the visual interactive interface, the user can intuitively interact with the large model, and can also use the large model to generate or adjust the virtual image based on simple dragging, clicking or text input.
[0084] It can be understood that the above describes the method of obtaining a virtual image in the present disclosure, and the following describes some methods of adjusting the virtual image.
[0085] In some embodiments, the method further includes: determining at least one adjustment prompt text and at least one second attribute adjustment information according to the adjustment text corresponding to the target virtual image. In response to receiving the adjustment text input by the user, at least one adjustment prompt text and at least one second attribute adjustment information can be determined using the first large model. The first large model can be the first large model llm41 mentioned above. The adjustment prompt text can be determined from multiple preset prompt texts. For example, for the target virtual image vp51 mentioned above, the adjustment text input by the user can be "I don't like this pink, change to another color, and I don't like the color of the skirt either." As mentioned above, according to the initial attribute information corresponding to the style type determination task "girlish cute sweet style, pink as the base tone, randomly calling cartoon or fruit patterns", the style of the target virtual image is pink. Thus, the adjustment prompt text "style" and "skirt" can be determined. The second attribute adjustment information corresponding to the adjustment prompt text "style" can be changing pink. The attribute adjustment information corresponding to the adjustment prompt text "skirt" can be changing color.
[0086] In some embodiments, the method further includes: adjusting the attribute information of at least one to-be-adjusted material corresponding to the at least one adjustment prompt text according to at least one second attribute adjustment information corresponding to the at least one adjustment prompt text, to obtain at least one adjusted material. For example, the overall style of the avatar may be replaced with a different color. The color of the skirt may also be changed to another color.
[0087] In some embodiments, the method further includes: obtaining an adjusted virtual image based on at least one adjusted material. For example, a virtual image may be regenerated based on the adjusted material to obtain an adjusted virtual image. Next, the adjusted virtual image is displayed on a visual interface. Figure 6 Provide explanation.
[0088] Figure 6 is a schematic diagram of an adjusted virtual image according to an embodiment of the present disclosure.
[0089] like Figure 6As shown, the visual interface i60 can display the adjusted virtual image vp62. The adjusted virtual image vp62 can be obtained by adjusting the target virtual image vp51. The visual interface i60 can also include an input box ib60. Through the embodiment of the present disclosure, the virtual image can be quickly adjusted to achieve efficient iteration, greatly improve the processing efficiency of the virtual image, and quickly obtain a high-quality virtual image.
[0090] It can be understood that the above further describes the method of the present disclosure, and the application scenario of the present disclosure will be described below.
[0091] In some embodiments, character design is a crucial part in application scenarios such as games and movies. Based on the above method, digital humans can be generated with the assistance of a large model to obtain digital humans with different shapes. Users can create unique and attractive virtual images as characters more efficiently. Users only need to provide input text representing some basic design requirements and creative inspirations, and the large model can quickly generate multiple character design schemes that meet the requirements. This not only greatly shortens the design cycle, but also provides users with more creative options, which helps to create more vivid and realistic game and film and television characters and enhance the audience's visual experience.
[0092] In some embodiments, in advertising and marketing scenarios, an attractive image can often significantly increase brand awareness and product sales. Based on the above method, a large model can be used to assist in the generation of digital humans to obtain digital humans with different shapes. Users can easily design unique virtual images based on brand positioning and target audiences. The virtual image can be a cute cartoon character, a fashionable model, or a future warrior with a sense of technology. Based on the method disclosed in the present invention, the design concept can be quickly realized, thereby enhancing the attractiveness and communication effect of the advertisement.
[0093] In some embodiments, with the rapid development of virtual reality and augmented reality technologies, the application of digital human modeling design in these fields is becoming increasingly widespread. In virtual reality games, users can play various unique roles and experience different virtual worlds. In augmented reality applications, digital human images can be used as virtual representatives of users to interact with users. Using the above method, digital human generation can be assisted by large models. Designers can create more realistic and personalized digital human images for virtual images or augmented reality applications to enhance the user's immersion and interactive experience. These images can not only be customized according to the user's preferences, but also realize real-time action response and expression recognition in the virtual environment, bringing users a more real and natural interactive experience.
[0094] In some embodiments, the above method is used to help provide more vivid and interesting teaching content for online education platforms. The above method can generate a digital teacher or teaching assistant image with affinity to enhance students' learning interest and participation. These digital teachers can guide students in the course, explain complex concepts, and even interact with students to improve teaching effectiveness.
[0095] In some embodiments, on social networks and entertainment platforms, using the above method, users can present themselves with unique images. Using the above method, users can easily create and customize their own personalized virtual images. These images can not only be used as avatars and personal profiles on social platforms, but also in virtual chat rooms, online games and other scenarios to increase users' sense of participation and belonging.
[0096] In some embodiments, in fashion and clothing design scenarios, using the above method, users can quickly test and display new design concepts. Users can use the above method to create a virtual image as a model to try on and display new clothing designs. This can not only speed up the design iteration process, but also more intuitively display the design effect, thereby better communicating with customers or teams.
[0097] In some embodiments, in a smart home scenario, using the above method, a personalized virtual image can be created as an intelligent assistant. These intelligent assistants can interact with users in natural language and perform various tasks, such as checking the weather, playing music, setting reminders, etc. By designing a unique virtual assistant image, the user experience can be enhanced, making the smart home system more humane and interesting.
[0098] It can be understood that the above describes the application scenario of the present disclosure, and the device of the present disclosure will be described below.
[0099] Figure 7 is a block diagram of a virtual image processing device according to an embodiment of the present disclosure.
[0100] like Figure 7 As shown, the apparatus 700 may include a first determining module 710 , a first obtaining module 720 , an adding module 730 , and a second obtaining module 740 .
[0101] The first determination module 710 is used to determine at least one first prompt text and at least one second prompt text according to the input text.
[0102] The first obtaining module 720 is used to obtain the material to be processed and the image to be processed according to at least one first prompt text and at least one second prompt text. The material to be processed includes a region to be processed.
[0103] The adding module 730 is used to add a plurality of to-be-processed images to the to-be-processed area to obtain a target material corresponding to the to-be-processed material.
[0104] The second obtaining module 740 is used to obtain a target virtual image according to the target material.
[0105] In some embodiments, the adding module includes: a first determining submodule, which is used to determine the first attribute adjustment information of each of the multiple images to be processed and the target position information of each of the multiple images to be processed according to the coordinate data of the material to be processed. The target position information is used to indicate the sub-region to be processed in the region to be processed. The adjusting submodule is used to adjust the multiple images to be processed according to the first attribute adjustment information of each of the multiple images to be processed to obtain multiple adjusted images. The adding submodule is used to add the multiple adjusted images to the multiple sub-regions to be processed according to the target position information of each of the multiple images to be processed to obtain the target material.
[0106] In some embodiments, the device further includes: a second determination module, configured to determine at least one adjustment prompt text and at least one second attribute adjustment information according to the adjustment text corresponding to the target virtual image. An adjustment module, configured to adjust the attribute information of at least one to-be-adjusted material corresponding to the at least one adjustment prompt text according to the at least one second attribute adjustment information corresponding to the at least one adjustment prompt text, to obtain at least one adjusted material. A third acquisition module, configured to obtain an adjusted virtual image according to the at least one adjusted material.
[0107] In some embodiments, the first determination module includes: a second determination submodule, used to determine at least one first prompt text, at least one first attribute information corresponding to the at least one first prompt text, at least one second prompt text and at least one second attribute information corresponding to the at least one second prompt text based on the input text.
[0108] In some embodiments, the first obtaining module includes: a first obtaining submodule, used to obtain the material to be processed according to at least one first prompt text; and a second obtaining submodule, used to obtain the image to be processed according to at least one second prompt text.
[0109] In some embodiments, the first obtaining submodule includes: a first determining unit, configured to determine at least one first task to be performed according to at least one first attribute information corresponding to at least one first prompt text; and a first executing unit, configured to execute at least one first task to be performed to obtain the material to be processed.
[0110] In some embodiments, the second obtaining submodule includes: a second determining unit, configured to determine at least one second task to be performed according to at least one second attribute information corresponding to at least one second prompt text; and a second executing unit, configured to execute at least one second task to be performed to obtain an image to be processed.
[0111] In some embodiments, the first determination unit includes: a first determination subunit, which is used to determine at least one first task to be performed using a first large model based on at least one first attribute information corresponding to at least one first prompt text.
[0112] In some embodiments, the second determination unit includes: a second determination subunit, which is used to determine at least one second task to be performed using the first large model according to at least one second attribute information corresponding to at least one second prompt text.
[0113] In some embodiments, the second determination module includes: a third determination submodule, configured to determine at least one adjustment prompt text and at least one second attribute adjustment information using the first large model in response to receiving the adjustment text input by the user.
[0114] In some embodiments, the first large model is obtained by fine-tuning multiple sample texts and multiple preset prompt texts. The first prompt text, the second prompt text and the adjusted prompt text are determined from multiple preset prompt texts using the first large model.
[0115] In some embodiments, the second execution unit includes: using the second large model to execute at least one second task to be executed to obtain an image to be processed. The second large model is a visual large model.
[0116] In some embodiments, the second acquisition module includes: a first display submodule, used to display the target virtual image on the visual interface.
[0117] In some embodiments, the third obtaining module includes: a second display submodule, used to display the adjusted virtual image in the visual interface.
[0118] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0119] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0120] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0121] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0122] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0123] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the virtual image processing method. For example, in some embodiments, the virtual image processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the virtual image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the virtual image processing method in any other appropriate manner (eg, by means of firmware).
[0124] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0125] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0126] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.
[0129] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0131] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for processing a virtual image, comprising: Determine at least one first prompt text and at least one second prompt text according to the input text; According to at least one of the first prompt texts and at least one of the second prompt texts, respectively obtaining a material to be processed and an image to be processed, wherein the material to be processed includes an area to be processed; Adding a plurality of the to-be-processed images to the to-be-processed area to obtain a target material corresponding to the to-be-processed material; A target virtual image is obtained according to the target material.
2. The method according to claim 1, wherein: Determining at least one first prompt text and at least one second prompt text according to the input text includes: Based on the input text, determine at least one of the first prompt texts, at least one first attribute information corresponding to at least one of the first prompt texts, at least one of the second prompt texts, and at least one second attribute information corresponding to at least one of the second prompt texts.
3. The method according to claim 1, wherein: The obtaining of a to-be-processed material and an to-be-processed image according to at least one of the first prompt texts and at least one of the second prompt texts comprises: Obtaining the material to be processed according to at least one of the first prompt texts; The image to be processed is obtained according to at least one of the second prompt texts.
4. The method according to claim 3, wherein: The obtaining the to-be-processed material according to at least one of the first prompt texts includes: Determining at least one first task to be performed according to at least one first attribute information corresponding to at least one first prompt text; Execute at least one of the first tasks to be executed to obtain the material to be processed.
5. The method according to claim 3, wherein: The obtaining the image to be processed according to at least one of the second prompt texts comprises: Determining at least one second task to be performed according to at least one second attribute information corresponding to at least one second prompt text; Execute at least one of the second tasks to be executed to obtain the image to be processed.
6. The method according to claim 1, wherein: The step of adding the plurality of images to be processed to the area to be processed to obtain a target material corresponding to the material to be processed comprises: Determine, according to the coordinate data of the material to be processed, first attribute adjustment information of each of the plurality of images to be processed and target position information of each of the plurality of images to be processed, wherein the target position information is used to indicate a sub-region to be processed in the region to be processed; Adjust the plurality of images to be processed respectively according to the respective first attribute adjustment information of the plurality of images to be processed to obtain a plurality of adjusted images; According to the target position information of each of the plurality of images to be processed, the plurality of adjusted images are added to the plurality of sub-regions to be processed respectively to obtain the target material.
7. The method according to claim 1, further comprising: Determining at least one adjustment prompt text and at least one second attribute adjustment information according to the adjustment text corresponding to the target virtual image; According to at least one of the second property adjustment information corresponding to at least one of the adjustment prompt texts, adjusting the property information of at least one to-be-adjusted material corresponding to at least one of the adjustment prompt texts, to obtain at least one adjusted material; An adjusted virtual image is obtained according to at least one of the adjusted materials.
8. The method according to claim 4, wherein: The determining at least one first task to be performed according to at least one first attribute information corresponding to at least one first prompt text comprises: At least one of the first tasks to be performed is determined using a first large model according to at least one first attribute information corresponding to at least one of the first prompt texts.
9. The method according to claim 5, wherein: The determining at least one second task to be performed according to at least one second attribute information corresponding to at least one second prompt text comprises: At least one of the second tasks to be performed is determined using the first large model according to at least one second attribute information corresponding to at least one of the second prompt texts.
10. The method according to claim 7, wherein: The determining, according to the adjustment text corresponding to the target virtual image, at least one adjustment prompt text and at least one second attribute adjustment information comprises: In response to receiving the adjustment text input by the user, at least one adjustment prompt text and at least one second attribute adjustment information are determined using the first large model.
11. The method according to any one of claims 8 to 10, wherein: The first model is obtained by fine-tuning using multiple sample texts and multiple preset prompt texts. The first prompt text, the second prompt text and the adjustment prompt text are determined from a plurality of preset prompt texts using the first large model.
12. The method according to claim 5, wherein: The performing at least one of the second tasks to be performed to obtain the image to be processed comprises: At least one of the second tasks to be performed is performed using a second large model to obtain the image to be processed, wherein the second large model is a large visual model.
13. The method according to claim 1, wherein: The step of obtaining a target virtual image according to the target material includes: The target virtual image is displayed on a visual interface.
14. The method according to claim 7, wherein: The step of obtaining an adjusted virtual image according to at least one of the adjusted materials comprises: The adjusted virtual image is displayed on a visual interface.
15. A virtual image processing device, comprising: A first determination module, used to determine at least one first prompt text and at least one second prompt text according to the input text; A first obtaining module, configured to obtain a material to be processed and an image to be processed respectively according to at least one of the first prompt texts and at least one of the second prompt texts, wherein the material to be processed includes an area to be processed; An adding module, used for adding a plurality of the to-be-processed images to the to-be-processed area to obtain a target material corresponding to the to-be-processed material; The second obtaining module is used to obtain a target virtual image according to the target material.
16. The device according to claim 15, wherein: The adding module comprises: A first determination submodule, configured to determine first attribute adjustment information of each of the plurality of images to be processed and target position information of each of the plurality of images to be processed according to the coordinate data of the material to be processed, wherein the target position information is used to indicate a sub-region to be processed in the region to be processed; An adjustment submodule, configured to adjust the plurality of images to be processed respectively according to the respective first attribute adjustment information of the plurality of images to be processed, so as to obtain a plurality of adjusted images; The adding submodule is used to add the plurality of adjusted images to the plurality of sub-areas to be processed respectively according to the target position information of each of the plurality of images to be processed, so as to obtain the target material.
17. The apparatus according to claim 15, further comprising: A second determination module, configured to determine at least one adjustment prompt text and at least one second attribute adjustment information according to the adjustment text corresponding to the target virtual image; an adjustment module, configured to adjust the attribute information of at least one to-be-adjusted material corresponding to at least one of the adjustment prompt texts according to at least one of the second attribute adjustment information corresponding to at least one of the adjustment prompt texts, so as to obtain at least one adjusted material; The third obtaining module is used to obtain an adjusted virtual image according to at least one of the adjusted materials.
18. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 14.
19. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 14.
20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 14.
Citation Information
Cited By
Content generation method and device, medium, electronic equipment and program product
CN121304842A
