Virtual image processing method and device, electronic equipment and storage medium

Abstract

The invention provides a virtual image processing method, and relates to the technical field of artificial intelligence, in particular to the technical fields of large models, virtual digital humans, virtual reality, augmented reality and the like. According to the specific implementation scheme, at least one first prompt text and at least one first to-be-executed task corresponding to the at least one first prompt text are determined according to an input text, the first to-be-executed task corresponds to at least one second to-be-executed task, and the second to-be-executed task corresponds to a second prompt text; according to at least one to-be-processed prompt text, the initial virtual image is obtained, and the to-be-processed prompt text is obtained by fusing the first prompt text and at least one second prompt text corresponding to the first prompt text. The invention further provides a virtual image processing device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the fields of large models, virtual digital humans, virtual reality and augmented reality, and can be applied to electronic games, computer graphics (CG) promotional videos, digital human live broadcasts, etc. More specifically, the present disclosure provides a virtual image processing method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the application scenarios of large models are increasing. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for processing a virtual image.

[0004] According to one aspect of the present disclosure, a method for processing a virtual image is provided, the method comprising: determining, based on input text, at least one first prompt text and at least one first task to be executed corresponding to the at least one first prompt text, wherein the first task to be executed corresponds to at least one second task to be executed, and the second task to be executed corresponds to the second prompt text; obtaining an initial virtual image based on the at least one prompt text to be processed, wherein the prompt text to be processed is obtained by fusing the first prompt text with the at least one second prompt text corresponding to the first prompt text.

[0005] According to another aspect of the present disclosure, a virtual image processing device is provided, which includes: a determination module, used to determine at least one first prompt text and at least one first task to be executed corresponding to the at least one first prompt text based on input text, wherein the first task to be executed corresponds to at least one second task to be executed, and the second task to be executed corresponds to the second prompt text; a first acquisition module, used to obtain an initial virtual image based on at least one prompt text to be processed, wherein the prompt text to be processed is obtained by fusing the first prompt text with at least one second prompt text corresponding to the first prompt text.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0011] Figure 1 is a schematic diagram of an exemplary system architecture to which a method and apparatus for processing a virtual image can be applied according to an embodiment of the present disclosure;

[0012] Figure 2 is a flow chart of a method for processing a virtual image according to an embodiment of the present disclosure;

[0013] Figure 3 is a schematic flow chart of determining prompt text to be processed according to an embodiment of the present disclosure;

[0014] Figure 4A is a schematic diagram of a visual interface according to one embodiment of the present disclosure;

[0015] Figure 4B is a schematic diagram of an initial virtual image according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram of adjusting a virtual image according to an embodiment of the present disclosure;

[0017] Figure 6 is a schematic diagram of an adjusted avatar according to an embodiment of the present disclosure;

[0018] Figure 7 is a partial schematic diagram of an adjusted virtual image according to an embodiment of the present disclosure;

[0019] Figure 8 is a schematic block diagram of a virtual image processing device according to an embodiment of the present disclosure; and

[0020] Fig. 9 is a block diagram of an electronic device to which a virtual image processing method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] In application scenarios such as games, digital human industry, and CG animation, the production of virtual images requires professional designers to perform modeling, binding, and other operations in modeling software.

[0023] However, the production cost of virtual images is high. In order to create an exquisite virtual character, the producer needs to have a solid foundation in character modeling and model binding capabilities. At the same time, it is difficult for the producer to produce a virtual image that fully meets expectations in one go. After the production is completed, it takes a long time to modify. It takes a lot of time for a virtual image to meet production expectations. For related companies, in order to produce exquisite virtual images, it also requires a lot of manpower costs.

[0024] In addition, different producers have different criteria for judging virtual images. During the production process, the virtual image may need to be constantly modified to meet the judgment criteria of most people. That is, in order to produce the virtual image, the producer has rich production experience to reduce the number of modifications.

[0025] In addition, it is difficult for ordinary modelers, ordinary people, small and medium-sized teams and other people engaged in the production of virtual images to quickly and efficiently obtain satisfactory virtual images. For enterprises or teams, the contradiction between the development cycle required by the project and the actual development cycle is difficult to resolve. In application scenarios such as games, digital humans, and CG animations, the technical development of virtual image production is limited.

[0026] In some embodiments, artificial intelligence technologies such as generative adversarial networks (GANs) and diffusion models can be used to enable users to generate a three-dimensional virtual image head by talking to a large model or uploading an image, so as to meet the needs of producers who have low requirements for portrait quality and low art foundation. In other embodiments, virtual images can also be generated based on artificial intelligence technologies such as large language models (LLMs), visual models, and three-dimensional image generation. For example, based on the text input by the user, text parsing, visual mapping, two-dimensional facial parsing, three-dimensional generation, parametric representation, etc. can be performed.

[0027] However, the effect of the virtual image generated based on artificial intelligence technology depends largely on the training data used to generate the virtual image. If the training data is not diversified or biased, the generated virtual image may also have problems. In addition, generating high-quality virtual images requires higher hardware computing power. If the computing power of the user's hardware device is insufficient, it may lead to slower generation of virtual images and poorer results. If users are not familiar with 3D modeling or artificial intelligence technology, it is difficult for them to use artificial intelligence technology to generate virtual images. In addition, when generating videos, it is difficult for the virtual images in the video to maintain consistency, resulting in a poor user experience.

[0028] In addition, based on artificial intelligence technology, three-dimensional virtual images can be generated based on text or images. However, in some cases, users may need more sophisticated and customized virtual images. When using artificial intelligence technology, users need to upload their own images or provide personal information, which may lead to data privacy and security issues. In some scenarios, the effect of the generated virtual image may not be good, and the match between the generated results and the user input is low, which needs further optimization and improvement.

[0029] Therefore, in order to efficiently generate a high-quality virtual image, the present disclosure provides a virtual image processing method, and the system architecture of the method will be described below.

[0030] Figure 1 is a schematic diagram of an exemplary system architecture to which a method and apparatus for processing a virtual image can be applied according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0031] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0032] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptops, desktop computers, etc.

[0033] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0034] It should be noted that the virtual image processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the virtual image processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The virtual image processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the virtual image processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0035] It can be understood that the above describes the system architecture of the present disclosure, and the following will describe the method of the present disclosure.

[0036] Figure 2 is a flowchart of a method for processing a virtual image according to an embodiment of the present disclosure.

[0037] like Figure 2 As shown, the method 200 may include operations S210 to S220.

[0038] In operation S210, at least one first prompt text and at least one first to-be-performed task corresponding to the at least one first prompt text are determined according to the input text.

[0039] In the disclosed embodiment, the input text may be text input by a user. For example, the input text may be “I want a beauty blogger look”.

[0040] In the disclosed embodiment, the first prompt text may be determined in various ways according to the input text. The input text may be split to obtain multiple segmented words. The first prompt text may be determined from the multiple segmented words. For example, the segmented word "beauty blogger" whose part of speech is a noun may be used as the first prompt text.

[0041] In the disclosed embodiment, the first prompt text may correspond to at least one first task to be performed. For example, the first prompt text "Beauty blogger" may correspond to a face shape transformation task. The face shape transformation task may be a first task to be performed. It is understood that the correspondence between the prompt text and the task to be performed may be preset.

[0042] In the disclosed embodiment, the first task to be performed corresponds to at least one second task to be performed. For example, the face shape modification task also corresponds to the eye modification task, the nose modification task, the lip modification task, and the ear modification task. The eye modification task, the nose modification task, the lip modification task, and the ear modification task can be respectively used as a second task to be performed.

[0043] In the disclosed embodiment, the second task to be performed may correspond to the second prompt text. For example, the second prompt text may be determined from the input text or may be preset. In the case where the input text is "I want to look like a beauty blogger" and the second task to be performed is an eye transformation task, the second prompt text may be the preset "double eyelids".

[0044] In operation S220, an initial virtual image is obtained according to at least one prompt text to be processed.

[0045] In the disclosed embodiment, the prompt text to be processed is obtained by fusing the first prompt text with at least one second prompt text corresponding to the first prompt text. For example, the first prompt text "beauty blogger" corresponds to the face shape transformation task. The face shape transformation task corresponds to the eye transformation task. The eye transformation task corresponds to the second prompt text "double eyelids". The second prompt text "double eyelids" corresponds to the first prompt text "beauty blogger". The first prompt text and the second prompt text can be spliced ​​as the prompt text to be processed.

[0046] In the disclosed embodiment, one or more materials may be determined based on the prompt text to be processed. For example, materials corresponding to "beauty blogger" and "double eyelids" may be determined. A virtual image may be obtained based on these materials.

[0047] Through the disclosed embodiment, the first prompt text and the first task to be executed are determined according to the user's input text. The second task to be executed corresponding to the first task to be executed is determined, and a beautiful virtual image can be efficiently generated. The requirements for user input are reduced. The user can generate a high-quality virtual image that conforms to the text description by inputting simple natural language text, which can effectively improve the user experience and lower the threshold for generating virtual images.

[0048] It can be understood that the method of the present disclosure is described above, and the prompt text of the present disclosure will be further described below.

[0049] In some embodiments, in some implementations of the above operation S210, at least one first prompt text and at least one first task to be performed may be determined using a large model according to the input text. The large model may be a large language model (LLM). The large language model may be various conversational artificial intelligence models such as Wenxinyiyan.

[0050] In some embodiments, the big model can be obtained by fine-tuning multiple sample texts and multiple preset prompt texts. The first prompt text can be determined from multiple preset prompt texts using the big model. The sample text can be a historical text input into the big model by a user, or it can be a historical text input into the big model by multiple users with similar attributes, or it can be a text generated based on the historical text input by the user with a high degree of similarity, and the present disclosure does not limit this. The big model can be a conversational big model such as Wen Xin Yi Yan. Through the embodiments of the present disclosure, using the conversational big model, short natural language text prompt words can be used to quickly generate a virtual image based on the produced materials in the material production platform. The big model is fine-tuned using the preset prompt text, and the preset prompt text can correspond to the identification text of the material, so that the fine-tuned big model can quickly determine the material corresponding to the first prompt text from multiple materials.

[0051] Figure 3 It is a schematic flowchart of determining prompt text to be processed according to an embodiment of the present disclosure.

[0052] like Figure 3 As shown, according to the input text input20 input by the user, operation S311 may be performed.

[0053] In operation S311, the first prompt text is determined. For example, the input text input30 may be "I want to look like a beauty blogger, with long red hair, fashionable makeup, and fashionable clothes". According to the input text input30, multiple first prompt texts may be determined. The multiple first prompt texts include: "beauty blogger", "long red hair", "fashionable makeup", and "fashionable clothes".

[0054] like Figure 3 As shown, next, the large model llm30 can be used to determine the first task to be performed corresponding to the first prompt text from multiple preset tasks. For example, the multiple preset tasks may include a face shape transformation task, a hairstyle transformation task, a makeup transformation task, a clothing matching task, and an accessories matching task. According to the multiple first prompt texts mentioned above, it can be determined that the multiple first tasks to be performed are: a face shape transformation task, a hairstyle transformation task, a makeup transformation task, and a clothing matching task.

[0055] The large model llm30 can also be used to determine the correspondence between the first prompt text and the first task to be executed. For example, multiple correspondences include "face shape transformation: beauty blogger", "hairstyle transformation: long red hair", "makeup transformation: trendy and fashionable makeup", and "clothing matching: fashionable clothes". Next, for the first task to be executed, determine the pending task corresponding to the first task to be executed.

[0056] In operation S312 , it is determined whether the first task to be performed corresponds to the second task to be performed.

[0057] For example, taking the face shape transformation task as an example, the face shape transformation task can correspond to the face shape style determination task, the eye adjustment task, the nose adjustment task, the mouth adjustment task, the eyebrow adjustment task and the ear adjustment task. These tasks can be used as multiple second tasks to be performed. It can be determined that the face shape transformation task corresponds to multiple second tasks to be performed.

[0058] In operation S313, a second prompt text is determined.

[0059] For example, the face style determination task, the eye adjustment task, the nose adjustment task, the mouth adjustment task, the eyebrow adjustment task, and the ear adjustment task may correspond to preset prompt texts respectively. These preset prompt texts may be referred to as default prompt texts. Multiple preset prompt texts corresponding to multiple second tasks to be performed may be used as multiple second prompt texts. The preset prompt text corresponding to the eye adjustment task may be "double eyelids".

[0060] like Figure 3 As shown, the large model llm30 can be used to determine the prompt text p30 to be processed. For example, multiple second prompt texts can be spliced ​​with the first prompt text. During the splicing process, the second prompt text "double eyelids" corresponding to the eye adjustment task can be spliced ​​with the first prompt text "beauty blogger" corresponding to the face shape transformation task.

[0061] Next, the above operations S312 to S313 may be repeated for one or more tasks except the face shape transformation task among the multiple first tasks to be performed, so as to obtain multiple prompt texts to be processed corresponding to the multiple first tasks to be performed.

[0062] It can be understood that the above text describes the first task to be performed and the second task to be performed in conjunction with the face shape transformation task. However, the present disclosure is not limited to this, and the hairstyle transformation task, makeup transformation task, clothing matching task and accessories matching task also correspond to one or more tasks respectively. For example, the hairstyle transformation task corresponds to the hairstyle size determination task, the hair color determination task, etc. The makeup transformation task can correspond to the facial makeup determination task, the eye makeup determination task, the lip makeup determination task, etc. The clothing matching task can correspond to the clothing determination task, the pattern determination task, etc. The accessories matching task can correspond to the earring determination task, the glasses determination task, the necklace determination task, the ring determination task, the headwear determination task, etc. Through the embodiments of the present disclosure, the material properties of the virtual image can be quickly determined according to the input text, and the hairstyle and clothing that meet the requirements can also be quickly determined.

[0063] It can be understood that the above describes the method of determining the prompt text to be processed, and the following describes some methods of obtaining the initial virtual image.

[0064] Figure 4A is a schematic diagram of a visual interface according to an embodiment of the present disclosure.

[0065] like Figure 4A As shown, the visual interface i40 may include an input box ib40. The visual interface i40 may also display a preset virtual image vp40.

[0066] The user can use the input box ib40 to input the text "create a traditional tea culture promotion spokesperson whose image should show the charm of Chinese style and classical style". Next, multiple prompt texts to be processed can be determined to generate an initial virtual image, which will be explained below.

[0067] In some embodiments, in some implementations of the above operation S220, obtaining an initial virtual image according to at least one pending prompt text includes: determining at least one pending task according to at least one pending prompt text. The pending task may correspond to a first pending task and at least one second pending task. For example, taking a clothing matching task as an example, a pending task may correspond to a clothing matching task, a clothing determination task, and a pattern determination task. The clothing matching task may be used as the first pending task. The clothing determination task and the pattern determination task may be used as the second pending task. It can be understood that the pending prompt text corresponding to the pending task may be obtained by splicing the first prompt text "Chinese style classical" and the second prompt text "cheongsam".

[0068] In the disclosed embodiment, the identification text of the task to be processed can be displayed in the visual interface. Figure 4AAs shown, the identification text of the task to be processed, such as "face shape transformation", "clothing matching", etc., can be displayed on the visual interface. It can be understood that the identification text of the task to be processed can be the identification text of the corresponding first task to be executed.

[0069] In an embodiment of the present disclosure, determining at least one task to be processed according to at least one prompt text to be processed includes: determining at least one material to be processed and at least one target attribute information corresponding to the at least one material to be processed according to the at least one prompt text to be processed. Determine at least one task to be processed according to the at least one material to be processed and at least one target attribute information corresponding to the at least one material to be processed. For example, according to "cheongsam" in the prompt text to be processed, a cheongsam material can be determined as the material to be processed from multiple clothing materials. The attributes of the clothing material may include logos, colors, materials, etc. A logo corresponds to a clothing material. If multiple logos are changed, the clothing material will also be replaced. According to the cheongsam material and the default attribute information of the cheongsam material, a task to be processed can be determined. The task to be processed can be provided to the rendering engine.

[0070] It can be understood that the above uses clothing quality as an example to illustrate the materials of the present disclosure. However, the present disclosure is not limited thereto, and the multiple materials may also include portrait materials, hair materials, and accessories materials. The attributes of the materials may include logos, colors, materials, etc. The attributes of the materials may be adjusted.

[0071] In some embodiments, in some implementations of the above operation S220, obtaining the initial virtual image according to at least one pending prompt text further includes: processing at least one pending task to obtain the initial virtual image. For example, a rendering engine may be used to process at least one pending task to obtain the initial virtual image. It is understood that the pending task may indicate one or more materials or may indicate the attributes of one or more materials. The rendering engine may render the materials according to the relevant attributes to obtain the initial virtual image. Figure 4B Provide explanation.

[0072] Figure 4B is a schematic diagram of an initial virtual image according to an embodiment of the present disclosure.

[0073] like Figure 4B As shown, the visual interface i40 can display an initial virtual image vp41. The clothing of the initial virtual image vp41 can be a cheongsam.

[0074] It can be understood that the above describes some ways of obtaining a virtual image in the present disclosure, and the following describes some ways of adjusting the virtual image.

[0075] In some embodiments, the adjusted virtual image is obtained according to the adjusted text and the initial virtual image. Figure 5 Provide explanation.

[0076] Figure 5 is a schematic diagram of adjusting a virtual image according to an embodiment of the present disclosure.

[0077] like Figure 5 As shown, user user50 can input text. The large model llm50 can receive the text and determine one or more tasks based on the text. The rendering engine e50 can execute the one or more tasks. One or more first prompt texts, second prompt texts, prompt texts to be processed, and one or more materials to be processed corresponding to the prompt texts to be processed used by the large model llm50 can be stored as historical data in the storage unit store50.

[0078] In the embodiment of the present disclosure, obtaining an adjusted virtual image according to the adjustment text and the initial virtual image may include: obtaining an adjusted virtual image after at least one adjustment round according to the adjustment text and the initial virtual image of at least one adjustment round. The adjustment of the initial adjustment round will be described below.

[0079] In the disclosed embodiment, obtaining the adjusted virtual image according to the adjustment text and the initial virtual image includes: determining at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text. According to the adjustment text of the initial adjustment round, at least one adjustment prompt text and at least one attribute adjustment information can be determined using a large model. The large model can be the above-mentioned large model llm30. The adjustment prompt text can be determined from a plurality of preset prompt texts. For example, for the above-mentioned initial virtual image vp41, in the initial adjustment round, the adjustment text input by the user can be "change the hair color, the clothes color is lighter, and the lipstick color is darker". Thus, the adjustment prompt texts "hair", "clothing" and "lipstick" of the initial adjustment round can be determined. The attribute adjustment information corresponding to the adjustment prompt text "hair" can be a change of color. The attribute adjustment information corresponding to the adjustment prompt text "clothing" can be a lightening of the color. The attribute adjustment information corresponding to the adjustment prompt text "lipstick" can be a darkening of the color.

[0080] In the disclosed embodiment, obtaining the adjusted virtual image according to the adjustment text and the initial virtual image may also include: in response to determining that the adjustment prompt text hits the historical prompt text, determining the material adjustment task according to at least one material to be adjusted corresponding to the hit historical prompt text and the attribute adjustment information corresponding to the adjustment prompt text. The historical prompt text is obtained based on at least one of the prompt text to be processed, the first prompt text, and the second prompt text. For example, the historical prompt text stored in the storage unit store50 may include the above-mentioned first prompt text "Chinese style classical" and the second prompt text "cheongsam". The adjustment prompt text "clothing" may hit the second prompt text "cheongsam". The large model llm50 may determine a material adjustment task by using the attribute adjustment information color lightening and the cheongsam material corresponding to the second prompt text "cheongsam". The material adjustment task may be provided to the rendering engine e50.

[0081] In the disclosed embodiment, obtaining the adjusted virtual image according to the adjustment text and the initial virtual image may also include: obtaining the adjusted virtual image according to the initial virtual image and the material adjustment task. For example, for the above-mentioned initial virtual image vp41, the rendering engine e50 may execute the material adjustment task to lighten the color of the cheongsam material of the initial virtual image vp41 to obtain the adjusted material. It can be understood that, based on the adjustment prompt text "hair" and the adjustment prompt text "lipstick", the corresponding materials to be adjusted can also be determined from the multiple materials used to generate the initial virtual image vp41. The two tasks can be determined respectively using the attribute adjustment information "change color" and the attribute adjustment information "darken color". After the rendering engine executes these two tasks, the adjusted material can be obtained. Based on these adjusted materials, the adjusted virtual image of the initial adjustment round can be obtained.

[0082] Through the disclosed embodiments, when adjusting the virtual image, the historical prompt text and the corresponding material are used, and the material to be adjusted can be quickly determined from the historically used materials, so as to reduce the hardware resources required for adjusting the virtual image. In the case of using a large model to determine the task, the number of tokens is also reduced, the response speed of the large model is improved, and it is helpful to improve the user experience.

[0083] Through the embodiments of the present disclosure, various materials for generating a virtual image can be fine-tuned, and various attributes of the virtual image (such as material and color) can be quickly and in real time adjusted according to the text input by the user, so as to achieve changes in the facial makeup, eye makeup, and lip makeup effects of the virtual image. Attributes such as the color and glossiness of the hairstyle can also be adjusted. Attributes such as the color, size, color and material of the logo on the clothing can also be adjusted. Attributes such as the transparency, color, texture, etc. of accessories (such as lenses) can also be adjusted.

[0084] In the embodiment of the present disclosure, obtaining the adjusted virtual image according to the adjustment text and the initial virtual image may also include: displaying the adjusted virtual image on the visual interface. Figure 6 Provide further explanation.

[0085] Figure 6 is a schematic diagram of an adjusted virtual image according to an embodiment of the present disclosure.

[0086] like Figure 6 As shown, the visual interface i60 can display the adjusted virtual image vp62. The adjusted virtual image vp62 can be obtained by adjusting the initial virtual image vp41. Thus, the adjustment of the initial adjustment round is completed.

[0087] Next, according to the user's input, the next adjustment round can be adjusted, or the process can be ended and the adjusted virtual image can be stored. The following takes multiple adjustment rounds as an example to further illustrate the present disclosure.

[0088] In the disclosed embodiment, obtaining the adjusted virtual image after at least one adjustment round according to the adjustment text and the initial virtual image of at least one adjustment round includes: obtaining the adjusted virtual image of the target adjustment round according to the adjustment text of the target adjustment round in multiple adjustment rounds and the adjusted virtual image of the previous adjustment round of the target adjustment round. For example, after displaying the adjusted virtual image vp62, the user can input the text "The hair color is not good-looking, let's go back to the previous version". Thus, the second adjustment round can be entered, that is, the round after the above-mentioned initial adjustment round. The text "The hair color is not good-looking, let's go back to the previous version" can be used as the adjustment text of the second adjustment round. According to the adjustment text, the adjustment prompt text "hair" and the corresponding attribute adjustment information "roll back to the previous version" can be determined.

[0089] In the disclosed embodiment, the historical prompt text of the target adjustment round is obtained based on at least one of the prompt text to be processed, the first prompt text, the second prompt text and at least one of the adjustment prompt texts of the previous adjustment round. For example, in the historical data stored in the above-mentioned storage unit store50, the adjustment prompt texts "hair", "clothing" and "lipstick" of the initial adjustment round can be stored in the storage unit store50 as historical prompt texts. In the second adjustment round, the adjustment prompt text "hair" can hit the historical prompt text "hair". Using the attribute adjustment information "roll back to the previous version" and the material corresponding to the historical prompt text "hair", a material adjustment task can be determined. The rendering engine executes the task and adjusts the adjusted virtual image of the initial adjustment round, and the adjusted virtual image of the second adjustment round can be obtained as the adjusted virtual image of the target adjustment round. The adjusted virtual image can be displayed next. The following will be combined with Figure 7 Provide explanation.

[0090] Figure 7 is a partial schematic diagram of an adjusted virtual image according to an embodiment of the present disclosure.

[0091] like Figure 7 As shown, the scaling ratio of the visual interface i70 can be adjusted to display the head of the adjusted virtual image vp73. Figure 7 As shown, the hair color of the adjusted avatar vp73 is consistent with the hair color of the initial avatar vp41. Through the disclosed embodiment, the user can modify the hairstyle, makeup, clothing, configuration and other materials of the avatar using natural language, and can store the materials used when generating the avatar, supporting historical backtracking functions such as returning to the previous version or the original version.

[0092] In some embodiments, the method may further include: fusing the virtual image with the action driving data. For example, the adjusted virtual image may be fused with the action driving data of a preset expression to drive the face to display the expression.

[0093] In some embodiments, the method further includes: storing a file corresponding to the virtual image. For example, the file may be in various formats. Through the disclosed embodiment, the initial virtual image and the adjusted virtual image after any adjustment round may be saved as a file in a desired format so that other users can modify it, and it may also be compatible with most rendering engines.

[0094] It can be understood that the method of the present disclosure is described above, and the application scenario of the present disclosure will be further described below.

[0095] In some embodiments, in an electronic game scene, based on the above method, a user can quickly and simultaneously produce multiple virtual human images that meet the requirements by inputting natural language text. A virtual image that meets the requirements can be determined from multiple images for further adjustment and production. The generation time of the virtual image only takes a few minutes, which greatly reduces the time for virtual image production and greatly improves development efficiency. During the development process, if the appearance, clothing, makeup, hairstyle, accessories, etc. of the virtual image need to be modified, the corresponding file can be imported into the rendering engine, and the adjustment text can be entered to use the method of the present invention to quickly modify the virtual image, and even no professional personnel are required, and no art foundation or professional knowledge is required to achieve virtual image modification only through dialogue.

[0096] In some embodiments, in various styles of CG cartoons, promotional videos, or virtual character green screen scenes, based on the method provided by the present disclosure, a virtual image that matches the text can be quickly generated by inputting natural language text.

[0097] In some embodiments, in various virtual digital human live broadcasting scenes, based on the method provided by the present disclosure, multiple virtual digital human images can be quickly and batch-produced by inputting natural language text, and a virtual digital human image library can be established based on the generated results. Next, digital human images of various styles can be replaced in the scene to meet different shooting or live broadcasting requirements, and finally various virtual digital human projects can be quickly completed.

[0098] It can be understood that the method of the present disclosure is described above, and the device of the present disclosure will be described below.

[0099] Figure 8 is a schematic block diagram of a virtual image processing device according to an embodiment of the present disclosure.

[0100] like Figure 8 As shown, the apparatus 800 may include a determination module 810 and a first obtaining module 820 .

[0101] The determination module 810 is used to determine at least one first prompt text and at least one first task to be executed corresponding to the at least one first prompt text according to the input text. The first task to be executed corresponds to at least one second task to be executed, and the second task to be executed corresponds to the second prompt text;

[0102] The first obtaining module 820 is used to obtain an initial virtual image according to at least one prompt text to be processed. The prompt text to be processed is obtained by fusing the first prompt text with at least one second prompt text corresponding to the first prompt text.

[0103] In some embodiments, the first obtaining module includes: a first determining submodule, configured to determine at least one pending task according to at least one pending prompt text. The pending task corresponds to the first pending task and at least one second pending task. A processing submodule, configured to process at least one pending task to obtain an initial virtual image.

[0104] In some embodiments, the first determination submodule includes: a first determination unit, configured to determine at least one material to be processed and at least one target attribute information corresponding to the at least one material to be processed according to at least one prompt text to be processed; and a second determination unit, configured to determine at least one task to be processed according to at least one material to be processed and at least one target attribute information corresponding to the at least one material to be processed.

[0105] In some embodiments, the apparatus 800 further includes: a second obtaining module, configured to obtain an adjusted virtual image according to the adjustment text and the initial virtual image.

[0106] In some embodiments, the second acquisition module includes: a second determination submodule, which is used to determine at least one adjustment prompt text and at least one attribute adjustment information based on the adjustment text. A third determination submodule, which is used to determine the material adjustment task in response to determining that the adjustment prompt text hits the historical prompt text, based on at least one to-be-adjusted material corresponding to the hit historical prompt text and the attribute adjustment information corresponding to the adjustment prompt text. The historical prompt text is obtained based on at least one of the to-be-processed prompt text, the first prompt text, and the second prompt text. The first acquisition submodule is used to obtain the adjusted virtual image based on the initial virtual image and the material adjustment task.

[0107] In some embodiments, the second obtaining module includes: a second obtaining sub-module, configured to obtain the adjusted virtual image after at least one adjustment round according to the adjustment text of at least one adjustment round and the initial virtual image.

[0108] In some embodiments, there are multiple adjustment rounds. The second obtaining submodule includes: a first obtaining unit, configured to obtain an adjusted virtual image of the target adjustment round according to the adjustment text of the target adjustment round in the multiple adjustment rounds and an adjusted virtual image of a previous adjustment round of the target adjustment round.

[0109] In some embodiments, the second obtaining submodule includes: a second obtaining unit, configured to obtain an adjusted virtual image of a starting adjustment round according to the initial virtual image and the adjustment text of a starting adjustment round among multiple adjustment rounds.

[0110] In some embodiments, the historical prompt text of the target adjustment round is obtained based on at least one of the prompt text to be processed, the first prompt text, the second prompt text, and the adjustment prompt text of at least one previous adjustment round.

[0111] In some embodiments, the determination module includes: a fourth determination submodule, configured to determine at least one first prompt text and at least one first task to be performed based on the input text using a large model.

[0112] In some embodiments, the second determination submodule includes: a third determination unit, configured to determine at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text using a large model.

[0113] In some embodiments, the large model is obtained by fine-tuning multiple sample texts and multiple preset prompt texts. The first prompt text and the adjusted prompt text are determined from multiple preset prompt texts using the large model.

[0114] In some embodiments, the first obtaining module includes: a display submodule, which is used to display the initial virtual image on the visual interface.

[0115] In some embodiments, the processing submodule includes: a display unit, which is used to display the identification text of the task to be processed on the visual interface.

[0116] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0118] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0119] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0120] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0121] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the virtual image processing method. For example, in some embodiments, the virtual image processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the virtual image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the virtual image processing method in any other appropriate manner (eg, by means of firmware).

[0122] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0127] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0128] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0129] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for processing a virtual image, comprising: Determine, according to the input text, at least one first prompt text and at least one first task to be performed corresponding to the at least one first prompt text, wherein the first task to be performed corresponds to at least one second task to be performed, and the second task to be performed corresponds to the second prompt text; An initial virtual image is obtained according to at least one prompt text to be processed, wherein the prompt text to be processed is obtained by fusing the first prompt text with at least one of the second prompt texts corresponding to the first prompt text.

2. The method according to claim 1, wherein: The obtaining of the initial virtual image according to at least one prompt text to be processed comprises: Determine at least one task to be processed according to at least one of the prompt texts to be processed, wherein the task to be processed corresponds to the first task to be executed and at least one of the second tasks to be executed; Process at least one of the pending tasks to obtain the initial virtual image.

3. The method according to claim 2, wherein: Determining at least one task to be processed according to at least one prompt text to be processed includes: Determining at least one material to be processed and at least one target attribute information corresponding to at least one material to be processed according to at least one of the prompt texts to be processed; At least one of the tasks to be processed is determined according to at least one of the materials to be processed and at least one of the target attribute information corresponding to at least one of the materials to be processed.

4. The method according to claim 1, further comprising: According to the adjustment text and the initial virtual image, an adjusted virtual image is obtained.

5. The method according to claim 4, wherein: The step of obtaining an adjusted virtual image according to the adjustment text and the initial virtual image comprises: Determining at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text; In response to determining that the adjustment prompt text hits a historical prompt text, determining a material adjustment task according to at least one of the to-be-adjusted material corresponding to the hit historical prompt text and the attribute adjustment information corresponding to the adjustment prompt text, wherein the historical prompt text is obtained according to at least one of the to-be-processed prompt text, the first prompt text, and the second prompt text; An adjusted virtual image is obtained according to the initial virtual image and the material adjustment task.

6. The method according to claim 4, wherein: The step of obtaining an adjusted virtual image according to the adjustment text and the initial virtual image comprises: According to the adjustment text of at least one adjustment round and the initial virtual image, a virtual image adjusted through at least one adjustment round is obtained.

7. The method according to claim 6, wherein: There are multiple adjustment rounds. The step of obtaining the virtual image adjusted through at least one adjustment round according to the adjustment text of at least one adjustment round and the initial virtual image comprises: The adjusted virtual image of the target adjustment round is obtained according to the adjustment text of the target adjustment round among the multiple adjustment rounds and the adjusted virtual image of the previous adjustment round of the target adjustment round.

8. The method according to claim 6, wherein: The step of obtaining the adjusted virtual image after at least one adjustment round according to the adjusted text of at least one adjustment round and the initial virtual image comprises: According to the initial virtual image and the adjustment text of the initial adjustment round among the multiple adjustment rounds, an adjusted virtual image of the initial adjustment round is obtained.

9. The method according to claim 7, wherein: The historical prompt text of the target adjustment round is obtained according to at least one of the prompt text to be processed, the first prompt text, the second prompt text and the adjustment prompt text of at least one previous adjustment round.

10. The method according to claim 1, wherein: The determining, according to the input text, at least one first prompt text and at least one first task to be performed corresponding to the at least one first prompt text comprises: According to the input text, at least one of the first prompt texts and at least one of the first tasks to be performed are determined using a large model.

11. The method according to claim 5, wherein: The determining, according to the adjustment text, at least one adjustment prompt text and at least one attribute adjustment information comprises: According to the adjustment text, at least one of the adjustment prompt texts and at least one of the attribute adjustment information are determined using a large model.

12. The method according to claim 10 or 11, wherein: The large model is obtained by fine-tuning multiple sample texts and multiple preset prompt texts. The first prompt text and the adjustment prompt text are determined from a plurality of preset prompt texts using the large model.

13. The method according to claim 1, wherein: The obtaining of the initial virtual image according to at least one prompt text to be processed comprises: The initial virtual image is displayed on a visual interface.

14. The method according to claim 2, wherein: The processing of at least one of the pending tasks to obtain the initial virtual image comprises: The identification text of the task to be processed is displayed on the visual interface.

15. A virtual image processing device, comprising: A determination module, configured to determine, according to an input text, at least one first prompt text and at least one first task to be executed corresponding to at least one first prompt text, wherein the first task to be executed corresponds to at least one second task to be executed, and the second task to be executed corresponds to a second prompt text; The first obtaining module is used to obtain an initial virtual image according to at least one prompt text to be processed, wherein the prompt text to be processed is obtained by fusing the first prompt text with at least one of the second prompt texts corresponding to the first prompt text.

16. The apparatus according to claim 15, further comprising: The second obtaining module is used to obtain the adjusted virtual image according to the adjustment text and the initial virtual image.

17. The device according to claim 16, wherein: The second obtaining module comprises: A second determination submodule, configured to determine at least one adjustment prompt text and at least one attribute adjustment information according to the adjustment text; A third determination submodule is configured to, in response to determining that the adjustment prompt text hits a historical prompt text, determine a material adjustment task according to at least one of the to-be-adjusted material corresponding to the hit historical prompt text and the attribute adjustment information corresponding to the adjustment prompt text, wherein the historical prompt text is obtained according to at least one of the to-be-processed prompt text, the first prompt text, and the second prompt text; The first obtaining submodule is used to obtain an adjusted virtual image according to the initial virtual image and the material adjustment task.

18. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 14.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 14.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 14.

Images