Role model generation method, electronic device, and computer-readable storage medium
Patent Information
- Application Number
- CN202510497317.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2026-10-23
AI Technical Summary
[0005]本申请实施例提供了一种角色模型生成方法、电子设备及计算机可读存储介质,以至少解决相关技术中生成二维角色,存在制作成本高、制作效率低且多样性差的技术问题
[0015]根据本申请实施例的另一方面,还提供了一种计算机程序产品,包括计算机程序,计算机程序在被处理器执行时实现本申请实施例中任意一项的角色模型生成方法。
Smart Images

Figure CN122966800A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of digital human technology, artificial intelligence technology, and large model technology. Specifically, it relates to a method for generating character models, an electronic device, and a computer-readable storage medium. Background Technology
[0002] In recent years, with the rapid development of digital human technology, the realism and interactivity of three-dimensional (3D) characters have been significantly improved. In contrast, the creation and application of interactive 2D cartoon characters are relatively lagging behind, especially in terms of automation, personalization and production efficiency.
[0003] Currently, Live2D technology is used to create animations for 2D characters by simulating 3D motion, thus eliminating the need to build complete 3D models. However, traditional Live2D models (i.e., 2D character models) require complex manual layer division and mesh binding during production, which is costly, time-consuming, and makes it difficult to generate a large number of character images with different styles in a short period of time.
[0004] There is currently no effective solution to the above problems. Summary of the Invention
[0005] This application provides a character model generation method, an electronic device, and a computer-readable storage medium to at least solve the technical problems of high production cost, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0006] According to one aspect of the embodiments of this application, a method for generating a character model is provided, comprising: acquiring input text, wherein the input text is used to record descriptive information of a two-dimensional character model to be generated; extracting descriptive terms of multiple target body parts from the input text, wherein the descriptive terms are used to determine the part types of multiple part templates; generating a template combination map based on the multiple part templates corresponding to the descriptive terms; determining part attribute information of the multiple part templates according to the template combination map and the input text, wherein the part attribute information is used to determine the pixel positions and pixel sizes of the multiple part templates; and transferring the part attribute information to a model texture map of the two-dimensional character model to be generated to generate a two-dimensional character model.
[0007] According to one aspect of the embodiments of this application, a method for generating a character model is provided, comprising: acquiring input text, wherein the input text is used to record descriptive information of a virtual customer service digit person to be generated; extracting descriptive terms of multiple target body parts from the input text, wherein the descriptive terms are used to determine the part type; generating a template combination map based on multiple part templates corresponding to the descriptive terms; determining part attribute information of multiple part templates according to the template combination map and the input text, wherein the part attribute information is used to determine the pixel position and pixel size of the multiple part templates; and transferring the part attribute information to the model texture of the virtual customer service digit person to be generated to generate the virtual customer service digit person.
[0008] According to one aspect of the embodiments of this application, a character model generation method is provided, comprising: obtaining a character model generation request through a first application programming interface, wherein the request data carried in the character model generation request includes: input text, which is used to record descriptive information of the two-dimensional character model to be generated; and returning a character model generation response through a second application programming interface, wherein the response data carried in the character model generation response includes: a two-dimensional character model, wherein the two-dimensional character model is generated according to any one of the character model generation methods in the embodiments of this application.
[0009] According to one aspect of the embodiments of this application, a character model generation method is provided, comprising: obtaining a currently input character model generation dialogue request, wherein the request data carried in the character model generation dialogue request includes: input text, which is used to record descriptive information of the two-dimensional character model to be generated; responding to the character model generation dialogue request, returning a character model generation dialogue response, wherein the information carried in the character model generation dialogue response includes: a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application; and rendering the two-dimensional character model in a graphical user interface.
[0010] According to one aspect of the embodiments of this application, a character model generation method is provided, comprising: displaying input text on the operation interface in response to an input instruction applied to the operation interface; and rendering a two-dimensional character model on the operation interface in response to a processing instruction applied to the operation interface; wherein the input text is used to record descriptive information of the two-dimensional character model to be generated, and the two-dimensional character model is generated according to any one of the character model generation methods in the embodiments of this application.
[0011] According to another aspect of the embodiments of this application, a character model generation system is also provided, including: a client for sending input text, wherein the input text is used to record descriptive information of the two-dimensional character model to be generated;
[0012] The server connects to the client and is used to generate a two-dimensional character model according to any one of the character model generation methods in the embodiments of this application; the client is also used to render the two-dimensional character model.
[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor connected to the memory via a bus for running the program, wherein the program executes any of the role model generation methods in the embodiments of this application.
[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to execute any of the role model generation methods in the embodiments of this application.
[0015] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the role model generation method of any one of the embodiments of this application.
[0016] In this embodiment, by acquiring input text, which records the descriptive information of the two-dimensional character model to be generated, and then extracting descriptive terms of multiple target body parts from the input text, wherein the descriptive terms are used to determine the part types of multiple part templates, then generating a template combination map based on the multiple part templates corresponding to the descriptive terms, and then determining the part attribute information of multiple part templates based on the template combination map and the input text, wherein the part attribute information is used to determine the pixel position and pixel size of multiple part templates, and finally transferring the part attribute information to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model. This achieves the goal of automatically generating diverse two-dimensional character models according to text intent, thus enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects, effectively reducing production costs, improving production efficiency and quality, and thus solving the technical problems of high production costs, low production efficiency and poor diversity in the generation of two-dimensional characters in related technologies.
[0017] It is worth noting that the general description above and the detailed description that follow are merely for illustrative purposes and do not constitute a limitation on this application. Attached Figure Description
[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0019] Figure 1 This is a schematic diagram illustrating an application scenario of a character model generation method according to an embodiment of this application;
[0020] Figure 2 This is a flowchart of a character model generation method according to an embodiment of this application;
[0021] Figure 3 This is a schematic diagram illustrating the generation of a two-dimensional character model according to an embodiment of this application;
[0022] Figure 4 This is a flowchart of a character model generation method according to an embodiment of this application;
[0023] Figure 5 This is a flowchart of a character model generation method according to an embodiment of this application;
[0024] Figure 6 This is a flowchart of a character model generation method according to an embodiment of this application;
[0025] Figure 7 This is a flowchart of a character model generation method according to an embodiment of this application;
[0026] Figure 8 This is a schematic diagram of the structure of a character model generation system according to an embodiment of this application;
[0027] Figure 9 This is a schematic diagram of the structure of a character model generation device according to an embodiment of this application;
[0028] Figure 10 This is a schematic diagram of another character model generation device according to an embodiment of this application;
[0029] Figure 11 This is a schematic diagram of another character model generation device according to an embodiment of this application;
[0030] Figure 12 This is a schematic diagram of another character model generation device according to an embodiment of this application;
[0031] Figure 13 This is a schematic diagram of another character model generation device according to an embodiment of this application;
[0032] Figure 14 This is a structural block diagram of a computing device according to an embodiment of this application;
[0033] Figure 15 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0034] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.
[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0036] The technical solution provided in this application is mainly implemented using large-scale model technology. Here, "large-scale model" refers to a deep learning model with a massive number of parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of parameters. Large-scale models can also be called foundation models. They are pre-trained using large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.
[0037] It should be noted that, in practical applications, large models can be fine-tuned using a small number of samples to adapt them to different tasks. For example, large models can be widely used in Natural Language Processing (NLP), computer vision, and speech processing. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. Therefore, the main application scenarios for large models include, but are not limited to, digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design. In this embodiment, the generation of a 2D cartoon character using a large model is used as an example for explanation.
[0038] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:
[0039] Live2D: A technique for creating and rendering two-dimensional (2D) character animations. Live2D allows developers to transform static 2D images into dynamic character models with expressions and movements.
[0040] Artificial Intelligence Generated Content (AIGC) refers to content of various types, such as text, images, audio, and video, that is automatically generated using artificial intelligence technology.
[0041] Stable Diffusion XL (SDXL): This refers to a text-to-image generation model based on the Diffusion Model technique, used to generate high-quality images from text descriptions by progressively removing noise to create realistic pictures.
[0042] Large Language Model (LLM): refers to a model containing billions or even hundreds of billions of parameters that is capable of understanding and generating complex language tasks in human language processing.
[0043] Supervised Fine-Tuning (SFT) refers to the process of further training a pre-trained model using labeled supervised data to better adapt the model to a specific task or dataset.
[0044] ControlNet: A technique for enhancing and extending the control capabilities of generative models, allowing users to more precisely control the details and structure of generated content. By introducing additional control signals (such as edge maps, pose maps, depth maps, etc.), it generates higher-quality results that better meet expectations.
[0045] Model textures are images or textures applied to the surface of a 3D model to define its color, detail, and appearance. In Live2D technology, model textures are images that contain details of a character's appearance. These images are designed to match the model's mesh structure, allowing the character to display rich expressions and movements in animation while maintaining the consistency and realism of its appearance.
[0046] Fine-tuning (FT) is a technical term in machine learning and deep learning, particularly prevalent in the context of pre-trained models. It typically refers to further training a pre-trained model (such as a large language model (LLM) or an image generation model) on a new, specific, and smaller dataset to adapt it to the needs of a particular task. This process involves making small adjustments to the model's parameters, enabling it to better understand and process domain-specific or data-specific information, thereby improving its performance on those datasets.
[0047] Original artwork: In the fields of game development and animation production, it refers to the original design drawings created by artists before characters, scenes, etc. are presented in the final animation or game.
[0048] Mask image: A commonly used concept in image processing and computer vision, a mask image is a binary image (usually a black and white image) used to indicate which parts of an image should be processed or preserved, and which parts should be ignored.
[0049] 2D cartoon style is an important art form in digital character creation, especially popular with younger audiences. While advancements in digital human technology have spurred extensive research into realistic digital humans and 3D characters, interactive 2D cartoon characters have received relatively less attention. Unlike 3D cartoon characters, which require complex construction and resource-intensive rendering, the widely used 2D cartoon character format Live2D offers a more efficient alternative. Live2D allows for the animation of 2D characters in a manner that simulates 3D motion, without the need to build complete 3D models. Furthermore, Live2D uses lightweight Hypertext Markup Language 5 (HTML5) for rendering, improving accessibility and efficiency. However, the detailed layering and mesh binding in Live2D model creation remains a time-consuming and laborious task, and the completed Live2D models are difficult to reuse for different appearances.
[0050] Currently, 3D interactive digital human solutions have been proposed, but the hyper-realistic 3D digital human images are handmade, resulting in high costs and long lead times. Furthermore, although this solution incorporates Artificial Intelligence Generated Content (AIGC) technology, it is primarily used to generate scenes, camera movements, animations, and sounds, and has not been applied to the generation of 2D cartoon characters, leading to a lack of efficiency and personalization in the creation of 2D cartoon characters.
[0051] In addition, 2D realistic digital humans and 3D digital humans were proposed. The 2D realistic digital humans are trained by recording model videos, lacking user customization capabilities and limiting the diversity and innovation of the characters. While the 3D digital humans support text-based generation, they actually match text to a pre-defined material library, resulting in diversity limited by the size and content of the material library, failing to fully demonstrate the advantages of large models in diverse generation.
[0052] The methods for generating 2D cartoon characters in related technologies have the following drawbacks.
[0053] Defect 1: The creation of traditional Live2D models requires manual and meticulous layer division and mesh binding. This process is not only tedious but also requires a high level of professional skills, resulting in a long overall production cycle and high labor costs.
[0054] Defect 2: Traditional Live2D models are usually fixed to one or a few images, making it difficult to quickly change or customize them. This limits the reusability and flexibility of the models and cannot meet users' needs for diverse images.
[0055] To address the aforementioned deficiencies, no effective solution has been proposed prior to this application.
[0056] According to an embodiment of this application, a method for generating a character model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0057] Considering the large number of model parameters in large models and the limited computing resources of mobile terminals, the method provided in this application embodiment can be applied to, for example, Figure 1 The application scenarios shown are not limited to these. In, for example... Figure 1 In the application scenario shown, the large model is deployed on server 10. Server 10 can connect to one or more client devices 20 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Client devices 20 can interact with users through a graphical user interface to access the large model, thereby implementing the method provided in this embodiment.
[0058] In this embodiment, the system consisting of a client device and a server can perform the following steps: the client device performs steps such as acquiring input text, where the input text is used to record descriptive information of the two-dimensional character model to be generated, and sending the input text to the server; the server performs steps such as extracting descriptive terms of multiple target body parts from the input text, where the descriptive terms are used to determine the part types of multiple part templates; then generating a template combination image based on the multiple part templates corresponding to the descriptive terms; then determining the part attribute information of multiple part templates based on the template combination image and the input text, where the part attribute information is used to determine the pixel position and pixel size of multiple part templates; finally, transferring the part attribute information to the model texture of the two-dimensional character model to be generated, generating the two-dimensional character model, and returning the two-dimensional character model to the client device; it should be noted that, if the operating resources of the client device can meet the deployment and operation conditions of a large model, this embodiment can be performed on the client device.
[0059] It should be noted that with the rapid development of high-performance computing units, the methods provided in this application embodiment can also be applied to model-in-machine systems in other application scenarios. In one optional embodiment, the model-in-machine system has multiple built-in models, and users can select one model to adjust as needed to obtain their own model. The high-performance computing unit built into the model-in-machine system can then directly call the adjusted model to execute the methods provided in this application embodiment. In another optional embodiment, the large model-in-machine system has a pre-trained model built-in, and the high-performance computing unit built into the model-in-machine system can then directly call that model to execute the methods provided in this application embodiment.
[0060] Furthermore, when users need to train their own models, they can upload their own datasets via the client. These datasets are then sent to the server, allowing the server to adjust the pre-trained model using the dataset to obtain the user's customized model, which can then be deployed to the production environment. To facilitate users' model adjustment needs, the server provides complete adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies. This allows the adjusted model to better adapt to different application domains and achieve a high degree of customization.
[0061] Under the aforementioned operating environment, this application provides the following: Figure 2 The character model generation method shown. Figure 2 This is a flowchart of a character model generation method according to an embodiment of this application. Figure 2 As shown, the method may include the following steps:
[0062] Step S21: Obtain input text, wherein the input text is used to record the descriptive information of the two-dimensional character model to be generated;
[0063] Step S22: Extract descriptive terms for multiple target body parts from the input text, wherein the descriptive terms are used to determine the part type of multiple part templates;
[0064] Step S23: Generate a template combination diagram based on multiple component templates corresponding to descriptive terms;
[0065] Step S24: Based on the template combination diagram and the input text, determine the component attribute information of multiple component templates, wherein the component attribute information is used to determine the pixel position and pixel size of multiple component templates;
[0066] Step S25: Transfer the component attribute information to the model texture of the two-dimensional character model to be generated, and generate the two-dimensional character model.
[0067] In this embodiment, the two-dimensional character model can be understood as a traditional 2D model or a Live2D model, and is not limited thereto. Live2D model is a special 2D character representation technology. It can simulate the effects of 3D animation by using a series of image layers and mesh structures, while maintaining the aesthetics and style of 2D art. The following explanation uses a two-dimensional character model as an example of a Live2D model.
[0068] The input text can be understood as a text description provided by the user or the system, used to record descriptive information about the two-dimensional character model to be generated. For example, it may include detailed information such as the characteristics, appearance, and style of the two-dimensional character model to be generated, which is not limited here. For example, the input text may describe the character's gender, age, hairstyle, clothing, expression, and other characteristics, such as "a young man wearing ancient warrior armor, with long blond hair and blue eyes," which is not limited here.
[0069] The two-dimensional character model to be generated can be understood as a Live2D model that meets user needs. Furthermore, the two-dimensional character model to be generated is a Live2D model that meets user needs and is diverse.
[0070] By acquiring the input text used to record the descriptive information of the two-dimensional character model to be generated, we can obtain the user's specific needs, capture the user's expectations for the character model, and thus provide clear guidance for subsequent generation steps.
[0071] After obtaining the input text, descriptive terms for multiple target body parts are extracted from the input text. These descriptive terms are used to determine the part type of the multiple part templates, that is, to describe the specific characteristics of each body part, such as "long blond hair," "blue eyes," "ancient warrior armor," etc., without any restrictions.
[0072] A component template can be understood as a preset image template for the design of various components (such as head, body, limbs, clothing, accessories, etc.) of a 2D character model. Component templates can include the basic shape, key control points, and mesh layout of body parts to facilitate subsequent image generation and animation production. For example, multiple preset image templates can be prepared for each body part in advance, which is not limited here.
[0073] By extracting descriptive terms related to each body part from the input text, the system can automatically identify and classify various elements in the user's needs, ensuring that the attributes of the generated character model are consistent with the input text description.
[0074] After obtaining the descriptive terms, a template combination image is generated based on the multiple component templates corresponding to the descriptive terms. The template combination image can be understood as a composite image generated by superimposing and combining multiple component templates that match the descriptive terms, which is a preview image of the overall image, thus providing a basic framework for the subsequent generation of character models.
[0075] By combining multiple component templates corresponding to descriptive terms to obtain a template combination diagram, a preliminary character image that conforms to the user's description can be quickly constructed, providing a basic framework for subsequent image refinement and editing, thereby improving generation efficiency.
[0076] After generating the template combination image, the component attribute information of multiple component templates is determined based on the template combination image and the input text. The component attribute information is used to determine the pixel position and pixel size of the multiple component templates. In other words, the component attribute information describes the exact pixel position and size that each component template should occupy in the 2D character model to be generated, thus ensuring that the subsequently generated image can accurately fit onto the mesh of the 2D character model to be generated.
[0077] For example, component attribute information may include detailed parameters of each component template (such as hair, eyes, clothing, etc.), such as the exact pixel coordinates where the hair component image will be placed and the actual size of the hair component image on the model texture. Furthermore, it may include occlusion handling, such as how to handle occlusion areas when multiple components overlap (e.g., hair and hat overlapping), which is not limited here.
[0078] Guided by template combination diagrams and input text, the precise pixel position and size of each component template can be determined. It's important to note that aligning the individual component templates with textures is a challenging aspect of the layer splitting process when creating Live2D models. Traditional segmentation methods cannot achieve pixel-level precision in segmenting individual components. This application, however, determines the pixel position and size of multiple component templates through component attribute information, ensuring that the generated image perfectly aligns with the Live2D model's mesh structure. This avoids mismatches or distortions between the image and the Live2D model, improving the aesthetics of the 2D character model and the smoothness of the animation.
[0079] After determining the component attribute information, this information is transferred to the model texture of the 2D character model to be generated, thus creating the 2D character model. The model texture can be understood as the specific image region corresponding to each body component of the Live2D model within the model structure, used to determine the appearance of the Live2D model.
[0080] Migrating component attribute information to the model textures of a 2D character model can be understood as mapping the generated image onto the textures of the Live2D model. Model textures are texture maps in a Live2D model used to fit the appearance of each body part. To maintain the continuity and correctness of the model's animation, the generated image needs to be perfectly aligned with the mesh structure of the model texture. Therefore, through the above migration process, the generated image content can be "pasted" onto the corresponding model texture, thus perfectly aligning the component attribute information with the model's mesh structure.
[0081] As can be seen, by transferring component attribute information to the model textures of the 2D character model, visual effects can be tightly integrated with the dynamic control of the character model, thereby generating a complete 2D character model with animation potential. This not only achieves a personalized appearance for the character but also ensures the dynamic performance of the 2D character model, enabling the final generated Live2D character to exhibit smooth animation effects while maintaining the uniqueness of the user-described design. This automated process greatly simplifies the time-consuming and labor-intensive manual texturing work in traditional Live2D model creation, reducing production costs and improving creative efficiency and quality.
[0082] In this embodiment, by acquiring user-input text and guiding the generation of 2D character models based on the input text, diverse 2D character models that meet user needs can be generated. Furthermore, by pre-preparing multiple component templates for body parts, a foundation for creating diverse and personalized character models is provided. Simultaneously, by determining the component attribute information of each component template and transferring this attribute information to the model texture of the 2D character model, it is ensured that the generated image perfectly aligns with the mesh structure of the 2D character model, and the dynamic performance of the 2D character model is guaranteed. In addition, the aforementioned automated process effectively reduces production costs and improves production efficiency and quality.
[0083] The role model generation method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving the generation of two-dimensional role models in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, it can generate two-dimensional role models related to e-commerce services, education services, legal services, and medical services, etc., and is not limited here.
[0084] By employing the embodiments of this application, input text is obtained, which is used to record the descriptive information of the two-dimensional character model to be generated. Then, descriptive terms of multiple target body parts are extracted from the input text, whereby the descriptive terms are used to determine the part types of multiple part templates. Subsequently, a template combination image is generated based on the multiple part templates corresponding to the descriptive terms. Then, based on the template combination image and the input text, the part attribute information of multiple part templates is determined, whereby the part attribute information is used to determine the pixel position and pixel size of multiple part templates. Finally, the part attribute information is transferred to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model. This achieves the goal of automatically generating diverse two-dimensional character models according to text intent, thus enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thereby solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0085] In an optional embodiment, step S22 involves extracting descriptive terms for multiple target body parts from the input text, including the following method steps:
[0086] Step S221: The fine-tuned language model is used to parse the input text into part types to obtain descriptive terms for multiple target body parts.
[0087] In this embodiment, when extracting descriptive terms for multiple target body parts from input text, a fine-tuned language model can be used to parse the input text into part types, thereby obtaining descriptive terms for multiple target body parts. That is, this application can use a fine-tuned (FT) large model to parse the input text into part types. It is understood that conventional large models cannot directly parse suitable and fixed descriptive terms from complex and varied input text. Therefore, this application uses a fine-tuned language model to parse the input text into part types by fine-tuning the large model. The fine-tuned language model can better understand and process language information related to Live2D character parts, accurately identify keywords describing different body parts from the user's input text, and provide precise guidance for the subsequent generation of images for each part, ensuring that the generated results highly match the user's needs.
[0088] A fine-tuned language model is used to parse the input text into component types. This process involves concept parsing and semantic understanding in Natural Language Processing (NLP). It can be understood as using a fine-tuned language model to parse the input text to identify and classify the described objects or concepts. In this embodiment, the user's input text describes various features of the Live2D character to be generated, including features of different body parts such as hair, eyes, and clothing. Through the fine-tuned language model, NLP techniques are used to accurately extract terms related to specific body parts—i.e., descriptive terms—from the input text.
[0089] For example, body parts can be randomly combined with common words to generate descriptive text, resulting in a large number of text-part pairs. These generated text-part pairs can then be used to fine-tune a large language model; this is not limited to specific examples. Furthermore, domain-specific knowledge can be incorporated during the training of the large model. By adding this knowledge to the training data, the large model can better understand the names and descriptions of body parts, as well as the logical relationships between them, thereby improving the accuracy of descriptive term extraction. Multimodal learning can also be performed, combining textual and visual information for training. By utilizing the body part features corresponding to images and text, the large model's ability to understand descriptive terms can be strengthened, helping it learn the association between descriptive terms and actual visual features, thus improving the accuracy of term parsing.
[0090] In one optional embodiment, the character model generation method further includes the following method steps:
[0091] Step S26: Pre-build a set of component templates, wherein the set of component templates includes: a maximum value template corresponding to multiple candidate body components, and the maximum value template is used to represent the template of the maximum size and the most complete detail version corresponding to any candidate body component.
[0092] In this embodiment, a set of component templates can be pre-built. This set includes: maximum value templates corresponding to multiple candidate body components. Candidate body components can include all body parts of a Live2D character whose appearance might be altered or decorated by the user, including but not limited to hair, tops, skirts, shoes, etc. Each candidate body component has a corresponding maximum value template to accommodate various changes.
[0093] A maximum value template can be understood as a template pre-designed for the largest size and most complete details of each possible body part (such as hair, clothing, accessories, etc.). The maximum value template contains all possible design elements for that type of body part, ensuring compatibility with images of various sizes and shapes, so that the generated image content can be accurately mapped onto the model texture without missing important details due to the template being too small.
[0094] For example, the maximum hair template is designed to be large enough to accommodate a variety of hairstyles, from long to short hair, ensuring that any generated hair image can be successfully embedded into the corresponding position of the Live2D model without any mismatch between the image and the template.
[0095] As can be seen, by pre-constructing a set of part templates, including maximum value templates corresponding to multiple candidate body parts, the generated image can be quickly and accurately located to the corresponding parts of the Live2D model, avoiding complex image segmentation and alignment work, and saving time and computing resources. At the same time, the design of the maximum value templates takes into account compatibility with different image content sizes and shapes, ensuring that no matter how complex or unique the generated image content is, it can be seamlessly integrated into the Live2D model without content loss or deformation due to template limitations. Furthermore, since multiple templates can be pre-prepared for each candidate body part (e.g., multiple styles of hair), combined with the large model generation capability, a large number of combinations can be created, greatly enriching the appearance of Live2D characters and satisfying users' pursuit of character diversity and personalization.
[0096] In an optional embodiment, step S26, pre-building the component template set includes:
[0097] Step S261: Decompose the existing two-dimensional character model into parts to obtain the decomposition results;
[0098] Step S262: Use preset type materials to obtain a set of component templates for the splitting results.
[0099] In this embodiment of the application, when pre-constructing the component template set, the existing two-dimensional character model can be split into components to obtain the splitting result. The existing two-dimensional character model can be understood as a two-dimensional character model pre-created using Live2D technology, consisting of multiple layers and a mesh. Each layer represents a different body part or clothing of the character, and the mesh is used to control the deformation and animation effects of the layers.
[0100] The existing 2D character model is decomposed into components, which can be understood as breaking down a complete 2D character model into smaller, independent graphic elements, such as front hair, middle hair, back hair, top, sleeves, pants, skirt, and shoes, etc. There are no restrictions here. This ensures that each component has a clear definition and boundary, and helps to independently control the generation of each body part or clothing in the future. This increases the flexibility and diversity in the generation process, while avoiding the computational complexity of generating the whole model and improving the generation efficiency.
[0101] After obtaining the splitting results, preset type materials are used to obtain a set of component templates. The preset type materials can be understood as a series of standard shape, size and position graphic templates prepared in advance for each body part, covering a variety of possible forms and styles of body parts. In other words, the preset type materials are the maximum amount of materials corresponding to each body part, which facilitates compatibility with small materials and can also improve the diversity of patterns and thus improve the diversity of results.
[0102] In one optional embodiment, the component control grid regions corresponding to multiple component templates are located within the component control grid regions of the splitting result.
[0103] In this embodiment, it can be understood that the control mesh can be understood as a set of control points and connecting lines defined on the surface of the character model, used to guide the deformation and animation of the character model. In Live2D technology, each part is associated with a control mesh, that is, a pre-defined mesh area is set around each body part. This mesh area contains a series of control points for later animation production and control. This control mesh area defines the range of deformation of the part in the animation and controls the dynamic effect of the part. The part control mesh area can be understood as a specific area in the model used to control the movement of a certain part.
[0104] The component control mesh area of the split result can be understood as the original control mesh area corresponding to each independent component after the Live2D model is split into components. It is used to determine the range of motion and deformation of the independent component in the animation. The component control mesh area of the split result can be understood as follows: the model is split into multiple independent layers according to different parts of the body. Each layer is used to represent one or more body parts. After splitting, each layer (i.e., component) has its corresponding control mesh area.
[0105] This application designs the control grid area of multiple component templates within the component control grid area of the split result, so that the generated image can accurately fit the mesh structure of the original Live2D model, without exceeding the deformation range of the original Live2D model, ensuring the compatibility of the generated content with the original Live2D model, supporting backward compatibility, and ensuring the naturalness and accuracy of the animation effect.
[0106] Furthermore, since the control grid area of the component template is limited to the component control grid area of the split result, it ensures that the original dynamic performance can be maintained during animation driving regardless of the generated image content, avoiding the problem of animation discontinuity or distortion that may be caused by exceeding the grid area.
[0107] In an optional embodiment, step S23 involves generating a template combination diagram based on multiple component templates corresponding to descriptive terms, including the following method steps:
[0108] Step S231: Obtain multiple component templates corresponding to descriptive terms from the component template set;
[0109] Step S232: Combine multiple component templates to generate a template combination diagram.
[0110] In this embodiment of the application, when generating a template combination diagram based on multiple component templates corresponding to descriptive terms, multiple component templates corresponding to descriptive terms can be obtained from the component template set, and then the multiple component templates can be combined to generate a template combination diagram.
[0111] In this context, template combination can be understood as piecing together multiple different component templates to form a complete image template that covers all described parts.
[0112] This can be understood as retrieving and selecting multiple corresponding component templates from a component template set based on descriptive terms extracted from the input text. For example, if the user description is "a girl with long blonde hair," then the component template for "front hair" will be selected from the component template set, and this template should match the description of "long blonde hair."
[0113] The selected component templates are then combined to form a template assembly diagram containing the corresponding parts for all descriptive terms. It is understood that the template assembly process may involve adjusting and arranging the size, position, and shape of different templates to ensure that the combined image covers the entire character model and that there are no conflicts between the parts, ensuring consistency. For example, the template for the "long yellow hair" bangs is combined with the template for the "light green V-neck short-sleeved shirt" and other templates; this is not a limitation.
[0114] It can be seen that by selecting templates that match the descriptive terms from the component template set and combining the templates to generate template combination diagrams, not only is the accuracy and visual quality of the generated results improved, but also the technical effects of improving generation efficiency and enhancing the diversity of Live2D characters are achieved.
[0115] In an optional embodiment, in step S24, the component attribute information of multiple component templates is determined based on the template combination diagram and the input text, including the following method steps:
[0116] Step S241: Convert the template combination image into an edge detection image;
[0117] Step S242: Based on the template combination image, edge detection image and input text, determine the component attribute information of multiple component templates.
[0118] In this embodiment, when determining the component attribute information of multiple component templates based on the template combination image and input text, the template combination image can be converted into an edge detection image. This can be understood as processing the template combination image through an edge detection algorithm to extract the contour information in the template combination image, thereby highlighting the boundaries and contour information of each component in the template combination image. Converting the template combination image into an edge detection image helps to accurately guide the generation of image content and ensures a precise match between the generated content and the template. For example, the edge detection image can accurately mark the boundaries of different parts such as the bangs, top, and skirt, ensuring that the generated image content falls accurately within these boundaries. This is not a limitation.
[0119] Then, based on the template composite image, edge detection image, and input text, the component attribute information of multiple component templates is determined. This can be understood as using the boundary information provided by the edge detection image, the component positions and shapes on the template composite image, and combining this with descriptive terms from the input text to precisely define the component attributes such as color, texture, and shape for each component. This allows for more accurate matching and generation of Live2D character components described by the user, improving the consistency between the generated content and user needs. Simultaneously, it automates the determination of component attribute information, reducing the need for manual editing and improving generation efficiency.
[0120] In an optional embodiment, in step S242, based on the template combination image, the edge detection image, and the input text, the component attribute information of multiple component templates is determined, including the following method steps:
[0121] Step S2421: Under preset control conditions, an initial original image is generated based on the edge detection image and the input text. The preset control conditions are used to maintain the original image style of the initial original image during the image generation process.
[0122] Step S2422: Perform local detail editing on the initial original artwork to obtain the target original artwork;
[0123] Step S2423: Extract component attribute information of multiple component templates from the target original artwork based on the template combination diagram.
[0124] In this embodiment, when determining the component attribute information of multiple component templates based on the template combination image, edge detection image, and input text, an initial original image can be generated based on the edge detection image and input text under preset control conditions. The preset control conditions are used to maintain the original image style of the initial original image during image generation. For example, the component attribute information of multiple component templates can be determined using a ControlNet. By introducing additional control signals (e.g., edge maps, pose maps, depth maps, etc.), ControlNet can more accurately control the structure and details of the image during generation, thereby maintaining the original image style of the initial original image and enhancing the controllability of the generated character model. This is not limited to specific examples.
[0125] The initial concept art can be understood as an image generated based on edge detection images and input text descriptions, containing the preliminary appearance of the Live2D character.
[0126] Under preset control conditions, an initial original artwork is generated based on edge detection images and input text. This can be understood as using boundary information provided by the edge detection images and descriptive words in the input text to generate an initial original artwork while maintaining consistency with the Live2D character style. It is understandable that automatically generating images based on text descriptions may produce results that deviate from the desired style. Therefore, this application introduces preset control conditions. By setting stylistic boundaries and guidelines for the model, the direction of image generation is constrained, ensuring that the generated content conforms to specific artistic styles and aesthetic standards.
[0127] After generating the initial original image, local detail editing is performed on the initial original image to obtain the target original image. This can be understood as fine-tuning a specific area on the generated initial original image to enhance image quality or meet personalized needs, thereby obtaining a target original image that meets all generation requirements.
[0128] For example, local detail editing may include fine-tuning colors, adding texture details, correcting small errors in the image, etc., and is not limited here.
[0129] As can be seen, by editing the initial artwork in detail, the visual effect of the initial image is improved, transforming it into a target artwork that more closely matches the user's expectations, thus making the generated Live2D character look more refined. At the same time, this detail editing allows for the fulfillment of users' personalized needs, such as specific hairstyles or clothing details, increasing the diversity of the characters.
[0130] After obtaining the target original artwork, the component attribute information of multiple component templates is extracted from the target original artwork based on the template combination diagram. This can be understood as using the component positions in the template combination diagram and the boundary information of the edge detection image to ensure that the attributes of each component are accurately separated and identified from the target original artwork, such as the colors and patterns of different parts, etc., which are not limited here.
[0131] As can be seen, the combined use of template composite images and edge detection images enables accurate location and extraction of component attribute information, improving the accuracy and efficiency of generation. Simultaneously, by extracting attribute information from the original target artwork, the edited image content can be fully utilized, ensuring that the final generated Live2D character perfectly matches the user's requirements.
[0132] In an optional embodiment, in step S2422, local detail editing is performed on the initial original image to obtain the target original image, including the following method steps:
[0133] Step S24221: Receive edit text through the image editing page, wherein the edit text is used to describe the area to be edited in the initial original image;
[0134] Step S24222: Perform local detail editing on the initial original artwork based on the edited text to obtain the target original artwork.
[0135] In this embodiment of the application, when performing local detail editing on the initial original image to obtain the target original image, edit text can be received through the image editing page. The image editing page can be understood as a user interface that allows users to intuitively manipulate and edit the generated image. For example, on the image editing page, users can specify which parts of the image need to be modified or adjusted through text input or other interactive methods; this is not limited to specific examples.
[0136] Edit text can be understood as a text description entered by the user, used to describe the area to be edited in the initial concept art, indicating the specific characteristics of the area to be edited in the initial concept art. For example, if the user wants to modify the pattern on the character's top, the edit text might contain instructions such as "change the top pattern to cherry blossoms," which is not limited here.
[0137] After receiving the edit text, the initial original artwork is partially edited based on the edit text to obtain the target original artwork. This can be understood as making partial detail edits on a specified area of the initial original artwork based on the edit text provided by the user, in order to meet the user's more specific and personalized creative needs.
[0138] It can be seen that targeted editing of the initial artwork based on user-inputted text enables precise editing of the generated content and highly customized image generation, while also improving user satisfaction with the final Live2D character.
[0139] In an optional embodiment, in step S2423, component attribute information of multiple component templates is extracted from the target original drawing based on the template combination diagram, including the following method steps:
[0140] Step S24231: Obtain the masking images corresponding to multiple component templates based on the template combination diagram;
[0141] Step S24232: Using the mask image, extract the component attribute information of multiple component templates from the target original artwork.
[0142] In this embodiment, when extracting component attribute information of multiple component templates from the target original artwork based on the template combination diagram, a mask image corresponding to the multiple component templates can be obtained based on the template combination diagram. The mask image can be understood as a binary image used to mark the position of a specific component in the target original artwork. For example, white pixels in the mask image can represent component areas, and black pixels can represent background areas; this is not limited here. In this application, the mask image is used to accurately define the outline and extent of each body component (such as hair, clothing, skirt, etc.), ensuring that the position and shape of each component are accurate when the generated image content is transferred to the Live2D model texture.
[0143] Generating mask images for multiple component templates based on the template combination diagram can be understood as analyzing the template combination diagram, extracting the boundary information of each component, and converting it into a mask image. The mask image clearly identifies the position and range of each component in the target original artwork, providing accurate guidance for subsequent attribute information extraction.
[0144] Next, using the masking image, component attribute information for multiple component templates is extracted from the target original artwork. This can be understood as using the generated masking image as a guide to accurately extract the component attribute information of each component template from the target original artwork. For example, during extraction, each marked area in the target original artwork can be analyzed to identify and record the color, texture, pattern, and other features of that area, ensuring that this information is consistent with the attributes specified by the user through text descriptions and local detail editing. There are no restrictions on this.
[0145] It can be seen that by generating a mask image and using the mask image to extract component attribute information, not only can the characteristic information of each component in the target original painting be accurately grasped, but the process of customizing Live2D character characteristics is also greatly simplified, and the accuracy and efficiency of information extraction are improved.
[0146] In one optional embodiment, the character model generation method further includes the following method steps:
[0147] Step S27: Fill the occluded area of the target original image with pixels to obtain the filling result;
[0148] Step S28: Perform image optimization on the filling result to obtain the region attribute information of the occluded area, wherein the region attribute information is used to determine the pixel position and pixel size of the occluded area.
[0149] In this embodiment of the application, the occluded area of the target original painting will also be filled with pixels to obtain the filling result. The occluded area can be understood as the area in the target original painting that is not captured due to the overlap of some parts or the body posture. These occluded areas may need to be dynamically displayed in the Live2D model animation, so pixel filling is required.
[0150] Region pixel fill can be understood as a technique that automatically fills in occluded areas to ensure the integrity of Live2D models and the natural presentation of animation effects.
[0151] To fill in the occluded areas identified in the original target image, it is understood that region pixel filling usually uses deep learning models or other image processing algorithms to automatically infer and fill in the missing pixels of the occluded areas based on the surrounding environment and existing pixel information, generating a visually continuous and natural filling result.
[0152] In this embodiment, pixel information from smaller, peripheral areas with similar visual characteristics in the original target image can be used to fill the blank areas of the occluded area, thereby helping to maintain the visual consistency and continuity between the filled area and the surrounding environment, especially in terms of edge details and color transitions.
[0153] After obtaining the filling result, image optimization is performed to obtain the regional attribute information of the occluded area. This regional attribute information can be understood as a detailed description of the occluded area in the filling result, used to determine the pixel position and pixel size of the occluded area. This includes, for example, pixel position, pixel size, color, texture, etc., which are not limited here.
[0154] Understandably, after completing the initial pixel filling of the occluded area of the target image, in order to further improve the quality, naturalness and overall consistency of the occluded area, it is necessary to perform subsequent processing on the occluded area, that is, to optimize the image of the filling result so that the filled or repaired image is as close as possible to the state of the original target image.
[0155] For example, image optimization of the fill result includes, but is not limited to: detail enhancement (increasing the details of the fill result to make it more harmonious with the surrounding environment, such as adding textures, lighting effects, etc.), boundary blending (making the edges of the fill result seamlessly connected with the boundaries of the target image to avoid obvious dividing lines and make the image look more coherent), color correction (adjusting the color of the fill result to ensure that it matches the color of the target image and maintains visual harmony), etc., which are not limited here.
[0156] In this embodiment, when optimizing the filled image, a graph-based image filling method can be used. Graph-based image filling is an image generation technique based on Generative Adversarial Networks (GANs) or diffusion models, which can generate new images with similar style and content based on one or more guiding images. When dealing with occluded areas, the graph-based image filling method can generate missing details of the occluded area based on surrounding image information, such as clothing patterns, decorations, and skin texture, making the occluded area look more realistic and match the overall style.
[0157] For example, in addition to the image-to-image filling method, image inpainting, detail enhancement, and color correction techniques can also be used, without limitation here.
[0158] It's important to note that aligning the templates and textures of each component is a challenging aspect of the layer splitting process when creating Live2D models. Traditional methods using image-based filling techniques often produce unstable results when dealing with large occluded areas, failing to consistently restore missing details. This application, however, first fills the entire area with small pixel areas before using image-based controlled generation. This initial repair using small pixel filling techniques, followed by precise control of generation using image-based techniques, allows for stable handling of large occluded areas. It avoids the unstable or unnatural filling effects that can result from a single method. Furthermore, it not only fills missing areas but also enriches details, making the Live2D model more realistic and lifelike.
[0159] By further optimizing the image of the filling result, the occluded area is matched with other parts of the target original image. Then, the regional attribute information of the occluded area is obtained from the optimization result, so that the accurate position and size of the occluded area in the Live2D model, as well as the corresponding visual features, can be accurately determined. This allows the occluded area of the Live2D model to be precisely controlled in the animation, avoiding unnatural deformation or displacement and optimizing the animation effect.
[0160] In an optional embodiment, in step S25, the component attribute information is transferred to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model, including the following method steps:
[0161] Step S251: Transfer the component attribute information and region attribute information to the model texture of the two-dimensional character model to be generated, and generate the two-dimensional character model.
[0162] In this embodiment, when transferring component attribute information to the model texture of the two-dimensional character model to generate the two-dimensional character model, both component attribute information and region attribute information can be transferred to the model texture of the two-dimensional character model to generate the two-dimensional character model. This can be understood as mapping component attribute information generated based on user text to the corresponding model texture area, ensuring that the visual presentation of each body part matches the user's description. Simultaneously, based on the processing results of occluded areas, region attribute information is merged into the model texture, filling in the blanks of previously occluded areas, ensuring the continuity and integrity of the character model during dynamic display.
[0163] As can be seen, integrating component and region attribute information into the model textures ensures that the generated Live2D models maintain consistency in style and detail, improving their visual appeal and user experience. Furthermore, automatically migrating component and region attribute information to the model textures significantly improves the efficiency of character model generation and reduces production costs compared to manually creating Live2D models.
[0164] Figure 3 This is a schematic diagram illustrating the generation of a 2D character model according to an embodiment of this application. When generating a 2D character model, this application first decomposes the existing Live2D model into parts. That is, it merges small, fragmented parts to reduce the number of parts, thereby reducing the generation complexity. Furthermore, each part can use the maximum amount of material, facilitating compatibility with small materials, thus increasing the diversity of the design and consequently the diversity of the final result.
[0165] Next, the corresponding components are parsed from the user's input text, and then combined to control the next step of generation. Since large models cannot directly parse suitable and fixed component descriptive words from complex and varied input text, this application uses existing components randomly combined with common words to generate descriptive text, thereby generating a large number of text-component pairs. The generated text-component pairs are then used to fine-tune the large model, thereby enabling the rapid parsing of accurate component categories from complex input text with an accuracy exceeding 90%.
[0166] Next, it's important to note that unlike realistic digital humans which prioritize facial recognizability, 2D cartoon characters have fewer facial variations (e.g., mostly large eyes, small noses, and small mouths), with hair, clothing, and accessories being more distinctive. Furthermore, the resolution and image quality of 2D cartoon character artwork are generally high, making the choice of model for controlling artwork generation crucial. This application compared current mainstream text-to-image models, evaluating accuracy, generation quality, text relevance, and text-pattern generation effects. Ultimately, the Stable Diffusion XL (SDXL) model was prioritized. SDXL offers strong controllability, accurate edge generation, and the ability to generate 1024 resolution images. It also responds well to long text descriptions and possesses the capability to generate text-patterns.
[0167] Meanwhile, to ensure compatibility with the original model's drivers, the generated regions must be within the control mesh areas of each component, supporting backward compatibility. The larger the original component region of the model, the more component types it supports. Therefore, this application pre-designs maximum size and most detailed version maximum value templates for each component. For example... Figure 3 The female cartoon character model is currently divided into the following component types: 5 types of back hair, 3 types of middle hair, 3 types of front hair, 5 types of tops, 6 types of sleeves, 5 types of pants, 5 types of skirts, and 6 types of shoes. Through the combination of the above components and the control capabilities of the base model, this application achieves superior control over the generated original artwork without affecting the driving effect of the original model.
[0168] Then, an image editing page is introduced, allowing users to freely paint to indicate specific locations, and also to use editable text to describe the painted areas, making detailed modifications to the initially generated original image to meet user needs and generate the final original image.
[0169] Finally, after finalizing the original artwork, a layer-by-layer splitting process, similar to that used in manually creating Live2D models, is required to perfectly place each part back into the texture map. There are two challenges here. First, traditional segmentation methods cannot achieve pixel-level precision in segmenting each component. Second, even with accurate component data, occluded and dynamically visible areas still need to be filled in. However, the image-based filling method is highly unstable when dealing with large occluded areas. Therefore, to address the first problem, this application directly uses the shape of the generated components as a mask to extract pixels from the original artwork. To address the second problem, this application first fills the entire area with small pixel areas and then uses the image-based control to generate the texture.
[0170] like Figure 3As shown, taking the input text A as "A girl with long blonde hair, blue eyes, wearing a light green V-neck short-sleeved shirt with a small pattern on the chest, a soft blue plaid skirt, and black ankle boots, exuding girlish playfulness and vitality," a finely tuned large model is used to parse the component types of the input text A, obtaining descriptive terms for multiple target body components. Then, a template combination image B is generated based on the multiple component templates corresponding to the descriptive terms. Next, the template combination image B is converted into an edge detection image C. Then, the edge detection image C is combined with the input text A and a stable diffusion model (texture-generated image model) with a control network is used to generate the initial concept art D. The initial concept art D is then edited with local details, for example, by inputting the edited text E "a flower crown on her head," resulting in the adjusted target concept art F. Finally, based on the pixel size and position of each component in the template combination image B, each component is migrated from the target concept art F to the texture map, while occlusion filling and rendering are performed simultaneously, ultimately obtaining a two-dimensional character model.
[0171] For example, this application employs text-based image generation technology and selects the SDXL open-source model architecture, which can generate images with higher resolution and different aspect ratios. During the transfer from the generated image to the target model texture, considering that image segmentation and other methods cannot obtain pixel-level precision segmentation results for each component, and that Live2D models have extremely high requirements for the accuracy of pixel position and size of each component, direct transfer to the texture is not possible. Therefore, this application pre-creates maximum value templates for each component. By parsing the component type from the text and retrieving the corresponding template combination, it converts the result into an edge detection image, and then uses a control network to constrain and generate images. This allows for the accurate pixel position and size of each component, ultimately enabling accurate transfer to the model texture. Furthermore, multiple versions of each component can be created, and the number of multiple component combinations multiplies. Combined with large model generation capabilities, this allows for the generation of a variety of Live2D models that meet user needs.
[0172] Understandably, a complex Live2D character typically consists of multiple layers, including the body, clothing, hair, etc. Each layer is divided into a polygonal mesh, which is used to control the various layers of the 2D character. This layer combination method allows for smooth deformation by controlling the position of mesh points. To precisely control these Live2D components, this application parses the user's text description and then uses text-based image processing to generate images corresponding to each component. Further editing and modification of local details, completion and repair of occluded areas, and then migration to the model texture map result in a completely new Live2D model. It can be seen that this application automatically generates new and aesthetically pleasing Live2D models, providing an automated solution for traditional Live2D model creation, greatly improving Live2D production efficiency, reducing production costs, and endowing existing Live2D models with endless versatility.
[0173] It is easy to understand that the beneficial effects of the character model generation method provided in this application include the following points.
[0174] Beneficial effects (1) The character model generation method proposed in this application is a Live2D automated generation method that can create a new two-dimensional cartoon character in a very short time without manual binding.
[0175] Beneficial effect (2): This application accurately extracts descriptive terms for each body part from complex text by fine-tuning the large model, ensuring that the generated results are closely aligned with the user's input.
[0176] Beneficial effect (3): This application utilizes the ability of text-to-image models to create a variety of driveable two-dimensional cartoon characters, greatly enhancing the diversity of Live2D.
[0177] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.
[0178] Furthermore, it should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0180] According to embodiments of this application, the following are also provided: Figure 4 This illustrates a method for generating character models. Figure 4 This is a flowchart of a character model generation method according to an embodiment of this application, such as... Figure 4 As shown, the method includes:
[0181] Step S41: Obtain input text, wherein the input text is used to record the description information of the virtual customer service avatar to be generated;
[0182] Step S42: Extract descriptive terms for multiple target body parts from the input text, wherein the descriptive terms are used to determine the part type;
[0183] Step S43: Generate a template combination diagram based on multiple component templates corresponding to descriptive terms;
[0184] Step S44: Based on the template combination diagram and the input text, determine the component attribute information of multiple component templates, wherein the component attribute information is used to determine the pixel position and pixel size of multiple component templates;
[0185] Step S45: Transfer the component attribute information to the model texture of the virtual customer service dummy to be generated, and generate the virtual customer service dummy.
[0186] The role model generation method proposed in this application embodiment can be applied to the scenario of generating virtual customer service digital humans. Virtual customer service digital humans can be understood as virtual images created using digital technology, used for customer service, information provision and other scenarios, and can simulate human appearance and behavior.
[0187] The input text can be understood as descriptive text provided by the user, including descriptions of the characteristics, appearance, clothing, decorations, etc. that the user wants the virtual customer service avatar to have, and there are no restrictions here.
[0188] For a detailed description, please refer to the description of the foregoing embodiments, which will not be repeated here.
[0189] The role model generation method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving the generation of two-dimensional role models in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, it can generate two-dimensional role models related to e-commerce services, education services, legal services, and medical services, etc., and is not limited here.
[0190] By employing the embodiments of this application, input text is obtained, which is used to record the descriptive information of the virtual customer service digit human to be generated. Then, descriptive terms of multiple target body parts are extracted from the input text, whereby the descriptive terms are used to determine the part types. Subsequently, a template combination diagram is generated based on multiple part templates corresponding to the descriptive terms. Next, based on the template combination diagram and the input text, the part attribute information of multiple part templates is determined, whereby the part attribute information is used to determine the pixel positions and pixel sizes of multiple part templates. Finally, the part attribute information is transferred to the model texture of the virtual customer service digit human to be generated, thereby generating the virtual customer service digit human. This achieves the goal of automatically generating diverse two-dimensional character models based on text intent, thus enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thereby solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0191] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.
[0192] According to embodiments of this application, the following are also provided: Figure 5 This illustrates a method for generating character models. Figure 5 This is a flowchart of a character model generation method according to an embodiment of this application, such as... Figure 5 As shown, the method includes:
[0193] Step S51: Obtain a character model generation request through the first application programming interface. The request data carried in the character model generation request includes: input text, which is used to record the description information of the two-dimensional character model to be generated.
[0194] Step S52: Return a character model generation response through the second application programming interface. The response data carried in the character model generation response includes a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application.
[0195] In this embodiment, the first application programming interface (API) and the second API can be the same API or different APIs. In an optional embodiment, the interface parameters in the first and second APIs may include, but are not limited to: global API identifier, API signature key, API timestamp, API request identifier, and system call credential identifier. The first API can use GET or POST as the API request method to obtain the file processing request. The second API can use JSON format to return the file processing response.
[0196] A character model generation request can be understood as a request sent by a user or another system through an Application Programming Interface (API) to generate a two-dimensional character model.
[0197] The character model generation response can be understood as a response returned by the API after processing the received character model generation request, which contains a two-dimensional character model generated according to the user's requirements.
[0198] For a detailed description, please refer to the description of the foregoing embodiments, which will not be repeated here.
[0199] The role model generation method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving the generation of two-dimensional role models in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, it can generate two-dimensional role models related to e-commerce services, education services, legal services, and medical services, etc., and is not limited here.
[0200] By employing the embodiments of this application, a character model generation request is obtained through a first application programming interface (API). The request data carried in the character model generation request includes input text, which is used to record the description information of the two-dimensional character model to be generated. Then, a character model generation response is returned through a second API. The response data carried in the character model generation response includes a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application. This achieves the goal of automatically generating diverse two-dimensional character models based on text intent, thereby enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thus solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0201] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.
[0202] According to embodiments of this application, the following are also provided: Figure 6 This illustrates a method for generating character models. Figure 6 This is a flowchart of a character model generation method according to an embodiment of this application, such as... Figure 6 As shown, the method includes:
[0203] Step S61: Obtain the current input character model generation dialogue request, wherein the request data carried in the character model generation dialogue request includes: input text, which is used to record the description information of the two-dimensional character model to be generated;
[0204] Step S62: In response to the character model generating dialogue request, return the character model generating dialogue response, wherein the information carried in the character model generating dialogue response includes: a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application;
[0205] Step S63: Render the two-dimensional character model within the graphical user interface.
[0206] In this embodiment of the application, the role model generation dialogue request can be understood as a request initiated by the user through a graphical user interface (GUI) or application programming interface (API) to generate a two-dimensional role model.
[0207] The character model generates a dialogue response, which can be understood as a response to the character model's request to generate a dialogue, and includes the generated two-dimensional character model information.
[0208] A graphical user interface (GUI) can be understood as a human-computer interface that provides users with a way to interact with a computer. It typically includes graphical elements, buttons, windows, etc., and is used to intuitively display information and receive user input.
[0209] For a detailed description, please refer to the description of the foregoing embodiments, which will not be repeated here.
[0210] The role model generation method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving the generation of two-dimensional role models in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, it can generate two-dimensional role models related to e-commerce services, education services, legal services, and medical services, etc., and is not limited here.
[0211] By employing the embodiments of this application, a dialogue request is generated by acquiring the currently input character model. The request data carried in the character model generation dialogue request includes: input text, which is used to record the description information of the two-dimensional character model to be generated. Then, in response to the character model generation dialogue request, a character model generation dialogue response is returned. The information carried in the character model generation dialogue response includes: a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application. Finally, the two-dimensional character model is rendered in the graphical user interface. This achieves the goal of automatically generating diverse two-dimensional character models based on text intent, thereby enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thus solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0212] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.
[0213] According to embodiments of this application, the following are also provided: Figure 7 This illustrates a method for generating character models. Figure 7 This is a flowchart of a character model generation method according to an embodiment of this application, such as... Figure 7 As shown, the method includes:
[0214] Step S71: In response to the input command applied to the operation interface, display the input text on the operation interface;
[0215] Step S72: In response to the processing instructions applied to the operation interface, render a two-dimensional character model on the operation interface; wherein, the input text is used to record the description information of the two-dimensional character model to be generated, and the two-dimensional character model is generated according to any one of the character model generation methods in the embodiments of this application.
[0216] In this embodiment of the application, the operation interface can be understood as the interface through which the user interacts with the system, allowing the user to input instructions or data to control the system to perform specific tasks.
[0217] Input commands can be understood as instructions issued by the user on the operating interface, which are intended to provide the system with text descriptions or other data for generating two-dimensional character models.
[0218] Processing instructions can be understood as commands issued by the user on the operating interface, instructing the system to generate and display a two-dimensional character model based on the input text.
[0219] For a detailed description, please refer to the description of the foregoing embodiments, which will not be repeated here.
[0220] The role model generation method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving the generation of two-dimensional role models in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, it can generate two-dimensional role models related to e-commerce services, education services, legal services, and medical services, etc., and is not limited here.
[0221] By adopting the embodiments of this application, the input text is displayed on the operation interface in response to the input command applied to the operation interface, and then a two-dimensional character model is rendered on the operation interface in response to the processing command applied to the operation interface. The input text is used to record the descriptive information of the two-dimensional character model to be generated. The two-dimensional character model is generated according to any one of the character model generation methods in the embodiments of this application. This achieves the purpose of automatically generating diverse two-dimensional character models according to the text intent, thereby enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thus solving the technical problems of high production costs, low production efficiency and poor diversity in the generation of two-dimensional characters in related technologies.
[0222] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.
[0223] According to embodiments of this application, the following are also provided: Figure 8 This is a character model generation system. Figure 8 This is a schematic diagram of the structure of a character model generation system according to an embodiment of this application, such as... Figure 8 As shown, the system includes:
[0224] The client is used to send input text, which records the descriptive information of the two-dimensional character model to be generated;
[0225] The server connects to the client and is used to generate a two-dimensional character model according to any one of the character model generation methods in the embodiments of this application.
[0226] The client is also used to render two-dimensional character models.
[0227] In this embodiment, the character model generation system is used to execute the character model generation method proposed in this application. For details, please refer to the description of the foregoing embodiments, which will not be repeated here.
[0228] The role model generation system provided in this application embodiment can be applied, but is not limited to, to application scenarios involving the generation of two-dimensional role models in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, it can generate two-dimensional role models related to e-commerce services, education services, legal services, and medical services, etc., and is not limited here.
[0229] By employing the embodiments of this application, a character model generation system is used to automatically generate diverse two-dimensional character models based on textual intent. This results in the final generated two-dimensional character models maintaining personalization and diversity while exhibiting smooth animation effects, effectively reducing production costs and improving production efficiency and quality. Consequently, it solves the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0230] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.
[0231] According to an embodiment of this application, an apparatus embodiment for implementing the above-described character model generation method is also provided. Figure 9 This is a schematic diagram of a character model generation device according to an embodiment of this application, such as... Figure 9 As shown, the device includes:
[0232] The first acquisition module 901 is used to acquire input text, wherein the input text is used to record the description information of the two-dimensional character model to be generated;
[0233] The first extraction module 902 is used to extract descriptive terms of multiple target body parts from the input text, wherein the descriptive terms are used to determine the part type of multiple part templates;
[0234] The first generation module 903 is used to generate a template combination diagram based on multiple component templates corresponding to descriptive terms;
[0235] The first determining module 904 is used to determine the component attribute information of multiple component templates based on the template combination diagram and the input text, wherein the component attribute information is used to determine the pixel position and pixel size of the multiple component templates;
[0236] The first migration module 905 is used to migrate the component attribute information to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model.
[0237] Optionally, the first extraction module 902 is further configured to: use a fine-tuned language model to parse the input text into part types, and obtain descriptive terms for multiple target body parts.
[0238] Optionally, the device further includes: a building module for pre-building a set of component templates, wherein the set of component templates includes: a maximum value template corresponding to multiple candidate body parts, the maximum value template being used to represent a template for the maximum size and most detailed version of any candidate body part.
[0239] Optionally, pre-building a set of component templates includes: splitting an existing 2D character model into components to obtain the splitting results; and using preset type materials for the splitting results to obtain a set of component templates.
[0240] Optionally, the component control grid regions corresponding to multiple component templates are located within the component control grid regions of the splitting result.
[0241] Optionally, the first generation module 903 is further configured to: obtain multiple component templates corresponding to descriptive terms from the component template set; combine the multiple component templates to generate a template combination diagram.
[0242] Optionally, the first determining module 904 is further configured to: convert the template combination image into an edge detection image; and determine the component attribute information of multiple component templates based on the template combination image, the edge detection image, and the input text.
[0243] Optionally, the first determining module 904 is further configured to: generate an initial original image based on the edge detection image and the input text under preset control conditions, wherein the preset control conditions are used to maintain the original image style of the initial original image during the image generation process; perform local detail editing on the initial original image to obtain the target original image; and extract component attribute information of multiple component templates from the target original image according to the template combination diagram.
[0244] Optionally, the first determining module 904 is further configured to: receive editing text through an image editing page, wherein the editing text is used to describe the area to be edited in the initial original image; and perform local detail editing on the initial original image based on the editing text to obtain the target original image.
[0245] Optionally, the first determining module 904 is further configured to: obtain mask images corresponding to multiple component templates based on the template combination diagram; and extract component attribute information of multiple component templates from the target original artwork using the mask images.
[0246] Optionally, the device further includes: an optimization module for filling the occluded area of the target original image with pixels to obtain a filling result; and for optimizing the filling result to obtain the region attribute information of the occluded area, wherein the region attribute information is used to determine the pixel position and pixel size of the occluded area.
[0247] Optionally, the first migration module 905 is further configured to: migrate the component attribute information and region attribute information to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model.
[0248] By employing the embodiments of this application, input text is obtained, which is used to record the descriptive information of the two-dimensional character model to be generated. Then, descriptive terms of multiple target body parts are extracted from the input text, whereby the descriptive terms are used to determine the part types of multiple part templates. Subsequently, a template combination image is generated based on the multiple part templates corresponding to the descriptive terms. Then, based on the template combination image and the input text, the part attribute information of multiple part templates is determined, whereby the part attribute information is used to determine the pixel position and pixel size of multiple part templates. Finally, the part attribute information is transferred to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model. This achieves the goal of automatically generating diverse two-dimensional character models according to text intent, thus enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thereby solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0249] It should be noted that the first acquisition module 901, the first extraction module 902, the first generation module 903, the first determination module 904, and the first migration module 905 mentioned above correspond to steps S21 to S25 in the embodiments. The five modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiments.
[0250] According to an embodiment of this application, another apparatus embodiment for implementing the above-described character model generation method is also provided. Figure 10 This is a schematic diagram of another character model generation device according to an embodiment of this application, such as... Figure 10 As shown, the device includes:
[0251] The second acquisition module 1001 is used to acquire input text, wherein the input text is used to record the description information of the virtual customer service digital human to be generated;
[0252] The second extraction module 1002 is used to extract descriptive terms of multiple target body parts from the input text, wherein the descriptive terms are used to determine the part type;
[0253] The second generation module 1003 is used to generate a template combination diagram based on multiple component templates corresponding to descriptive terms;
[0254] The second determining module 1004 is used to determine the component attribute information of multiple component templates based on the template combination diagram and the input text, wherein the component attribute information is used to determine the pixel position and pixel size of the multiple component templates;
[0255] The second migration module 1005 is used to migrate component attribute information to the model texture of the virtual customer service dummy to be generated, thereby generating the virtual customer service dummy.
[0256] By employing the embodiments of this application, input text is obtained, which is used to record the descriptive information of the virtual customer service digit human to be generated. Then, descriptive terms of multiple target body parts are extracted from the input text, whereby the descriptive terms are used to determine the part types. Subsequently, a template combination diagram is generated based on multiple part templates corresponding to the descriptive terms. Next, based on the template combination diagram and the input text, the part attribute information of multiple part templates is determined, whereby the part attribute information is used to determine the pixel positions and pixel sizes of multiple part templates. Finally, the part attribute information is transferred to the model texture of the virtual customer service digit human to be generated, thereby generating the virtual customer service digit human. This achieves the goal of automatically generating diverse two-dimensional character models based on text intent, thus enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thereby solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0257] It should be noted that the second acquisition module 1001, the second extraction module 1002, the second generation module 1003, the second determination module 1004, and the second migration module 1005 mentioned above correspond to steps S41 to S45 in the embodiments. The five modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiments.
[0258] According to an embodiment of this application, another apparatus embodiment for implementing the above-described character model generation method is also provided. Figure 11 This is a schematic diagram of another character model generation device according to an embodiment of this application, such as... Figure 11 As shown, the device includes:
[0259] The third acquisition module 1101 is used to acquire a character model generation request through the first application programming interface. The request data carried in the character model generation request includes: input text, which is used to record the description information of the two-dimensional character model to be generated.
[0260] The first return module 1102 is used to return a character model generation response through a second application programming interface. The response data carried in the character model generation response includes a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application.
[0261] By employing the embodiments of this application, a character model generation request is obtained through a first application programming interface (API). The request data carried in the character model generation request includes input text, which is used to record the description information of the two-dimensional character model to be generated. Then, a character model generation response is returned through a second API. The response data carried in the character model generation response includes a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application. This achieves the goal of automatically generating diverse two-dimensional character models based on text intent, thereby enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thus solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0262] It should be noted that the third acquisition module 1101 and the first return module 1102 mentioned above correspond to steps S51 and S52 in the embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also run in the server 10 provided in the embodiments.
[0263] According to an embodiment of this application, another apparatus embodiment for implementing the above-described character model generation method is also provided. Figure 12 This is a schematic diagram of another character model generation device according to an embodiment of this application, such as... Figure 12 As shown, the device includes:
[0264] The fourth acquisition module 1201 is used to acquire the currently input character model generation dialogue request, wherein the request data carried in the character model generation dialogue request includes: input text, which is used to record the description information of the two-dimensional character model to be generated;
[0265] The second return module 1202 is used to respond to the character model generating dialogue request and return a character model generating dialogue response, wherein the information carried in the character model generating dialogue response includes: a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application;
[0266] The first rendering module 1203 is used to render two-dimensional character models within a graphical user interface.
[0267] By employing the embodiments of this application, a dialogue request is generated by acquiring the currently input character model. The request data carried in the character model generation dialogue request includes: input text, which is used to record the description information of the two-dimensional character model to be generated. Then, in response to the character model generation dialogue request, a character model generation dialogue response is returned. The information carried in the character model generation dialogue response includes: a two-dimensional character model, which is generated according to any one of the character model generation methods in the embodiments of this application. Finally, the two-dimensional character model is rendered in the graphical user interface. This achieves the goal of automatically generating diverse two-dimensional character models based on text intent, thereby enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thus solving the technical problems of high production costs, low production efficiency, and poor diversity in the generation of two-dimensional characters in related technologies.
[0268] It should be noted that the fourth acquisition module 1201, the second return module 1202, and the first rendering module 1203 mentioned above correspond to steps S61 to S63 in the embodiments. The three modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiments.
[0269] According to an embodiment of this application, another apparatus embodiment for implementing the above-described character model generation method is also provided. Figure 13 This is a schematic diagram of another character model generation device according to an embodiment of this application, such as... Figure 13 As shown, the device includes:
[0270] Display module 1301 is used to respond to input commands applied to the operation interface and display the input text on the operation interface;
[0271] The second rendering module 1302 is used to respond to processing instructions applied to the operation interface and render a two-dimensional character model on the operation interface; wherein, the input text is used to record the description information of the two-dimensional character model to be generated, and the two-dimensional character model is generated according to any one of the character model generation methods in the embodiments of this application.
[0272] By adopting the embodiments of this application, the input text is displayed on the operation interface in response to the input command applied to the operation interface, and then a two-dimensional character model is rendered on the operation interface in response to the processing command applied to the operation interface. The input text is used to record the descriptive information of the two-dimensional character model to be generated. The two-dimensional character model is generated according to any one of the character model generation methods in the embodiments of this application. This achieves the purpose of automatically generating diverse two-dimensional character models according to the text intent, thereby enabling the final generated two-dimensional character model to maintain personalization and diversity while exhibiting smooth animation effects. This effectively reduces production costs and improves production efficiency and quality, thus solving the technical problems of high production costs, low production efficiency and poor diversity in the generation of two-dimensional characters in related technologies.
[0273] It should be noted that the display module 1301 and the second rendering module 1302 mentioned above correspond to steps S71 and S72 in the embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also run in the server 10 provided in the embodiments.
[0274] It should be noted that the preferred implementation schemes involved in the above embodiments of this application are the same as the schemes, application scenarios and implementation processes provided in the embodiments, but are not limited to the schemes provided in the embodiments.
[0275] Embodiments of this application may provide a computing device. Figure 14 This is a structural block diagram of a computing device according to an embodiment of this application. Figure 14 As shown, the computing device A may include: one or more ( Figure 14 (Only one is shown) processor 1402, memory 1404, memory controller, and peripheral interface, wherein the peripheral interface may connect to radio frequency module, audio module, display screen, etc., which are not limited here.
[0276] The aforementioned computing device A can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, PCs (Personal Computers), all-in-one model machines, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.
[0277] Specifically, computing device A can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, computing device A can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, computing device A also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, computing device A can also create applications based on models, providing API calling capabilities, allowing models to be called into created applications through API interfaces, and providing application management tools for application management and monitoring.
[0278] Furthermore, the computing device A can also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.
[0279] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0280] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.
[0281] Those skilled in the art will understand that, Figure 14 The structure shown is for illustrative purposes only; computing device A can also be a smartphone, tablet, PDA, mobile internet device (MID), PAD, or other terminal device. Figure 14This does not limit the structure of the aforementioned computing device. For example, computing device A may also include components that are more advanced than those described above. Figure 14 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same as the Figure 14 The different configurations shown.
[0282] Embodiments of this application may provide an electronic device. Figure 15 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 15 As shown, the electronic device may include: an input / output device 152; a memory 154; and a processor 156, wherein the processor 156 is connected to the input / output device 152 and the memory 154 via a bus 158.
[0283] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0284] The processor can invoke an executable program stored in memory via a transmission device to execute the methods described in any of the above embodiments. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing hardware related to the terminal device. This program can be stored in a computer-readable storage medium, which may include: a flash drive, read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.
[0285] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the aforementioned computer-readable storage medium can be used to store the program code executed by the character model generation method provided in the above embodiments.
[0286] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.
[0287] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements any of the above-described character model generation methods.
[0288] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0289] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0290] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0291] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0292] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0293] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A method for generating character models, characterized in that, include: Obtain input text, wherein the input text is used to record the descriptive information of the two-dimensional character model to be generated; Descriptive terms for multiple target body parts are extracted from the input text, wherein the descriptive terms are used to determine the part type of multiple part templates; A template combination diagram is generated based on the multiple component templates corresponding to the descriptive terms; Based on the template combination diagram and the input text, the component attribute information of the multiple component templates is determined, wherein the component attribute information is used to determine the pixel position and pixel size of the multiple component templates; The component attribute information is transferred to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model.
2. The character model generation method according to claim 1, characterized in that, The descriptive terms for extracting the plurality of target body parts from the input text include: The input text is parsed using a fine-tuned language model to obtain the descriptive terms for the multiple target body parts.
3. The character model generation method according to claim 1 or 2, characterized in that, The character model generation method also includes: A pre-built set of component templates is provided, wherein the set of component templates includes: a maximum value template corresponding to multiple candidate body components, wherein the maximum value template is used to represent the template of the maximum size and the most complete detail version corresponding to any candidate body component.
4. The character model generation method according to claim 3, characterized in that, The pre-built component template set includes: The existing 2D character model is decomposed into parts to obtain the decomposition results; The component template set is obtained by using preset type materials for the splitting results.
5. The character model generation method according to claim 4, characterized in that, The component control grid regions corresponding to the multiple component templates are located within the component control grid regions of the splitting result.
6. The character model generation method according to claim 3, characterized in that, Generating the template combination diagram based on the multiple component templates corresponding to the descriptive terms includes: Obtain the plurality of component templates corresponding to the descriptive terms from the component template set; The multiple component templates are combined to generate the template combination diagram.
7. The character model generation method according to claim 1 or 2, characterized in that, Based on the template combination diagram and the input text, the component attribute information of the multiple component templates is determined as follows: The template combination image is converted into an edge detection image; Based on the template combination image, the edge detection image, and the input text, the component attribute information of the multiple component templates is determined.
8. The character model generation method according to claim 7, characterized in that, Based on the template combination image, the edge detection image, and the input text, the component attribute information of the multiple component templates is determined as follows: Under preset control conditions, an initial original image is generated based on the edge detection image and the input text, wherein the preset control conditions are used to maintain the original image style of the initial original image during the image generation process; The initial original artwork is partially edited to obtain the target original artwork; Based on the template combination diagram, extract the component attribute information of the multiple component templates from the target original artwork.
9. The character model generation method according to claim 8, characterized in that, The target original artwork is obtained by performing local detail editing on the initial original artwork, including: The image editing page receives edit text, which describes the area to be edited in the initial original image. The target original artwork is obtained by performing local detail editing on the initial original artwork based on the edited text.
10. The character model generation method according to claim 8, characterized in that, The component attribute information of the multiple component templates extracted from the target original artwork based on the template combination diagram includes: Obtain the masking images corresponding to the multiple component templates based on the template combination diagram; Using the mask image, the component attribute information of the multiple component templates is extracted from the target original image.
11. The character model generation method according to claim 8, characterized in that, The character model generation method also includes: The occluded area of the target original image is filled with pixels to obtain the filling result; The filling result is optimized to obtain the region attribute information of the occluded area, wherein the region attribute information is used to determine the pixel position and pixel size of the occluded area.
12. The character model generation method according to claim 11, characterized in that, The process of transferring the component attribute information to the model texture of the two-dimensional character model to be generated, and generating the two-dimensional character model, includes: The component attribute information and the region attribute information are transferred to the model texture of the two-dimensional character model to be generated, thereby generating the two-dimensional character model.
13. A method for generating character models, characterized in that, include: Obtain input text, wherein the input text is used to record the description information of the virtual customer service avatar to be generated; Descriptive terms for multiple target body parts are extracted from the input text, wherein the descriptive terms are used to determine the part type; A template combination diagram is generated based on multiple component templates corresponding to the descriptive terms; Based on the template combination diagram and the input text, the component attribute information of the multiple component templates is determined, wherein the component attribute information is used to determine the pixel position and pixel size of the multiple component templates; The component attribute information is transferred to the model texture of the virtual customer service avatar to be generated, thereby generating the virtual customer service avatar.
14. A method for generating character models, characterized in that, include: A character model generation request is obtained through a first application programming interface, wherein the request data carried in the character model generation request includes: input text, which is used to record the descriptive information of the two-dimensional character model to be generated; The second application programming interface returns a character model generation response, wherein the response data carried in the character model generation response includes: a two-dimensional character model, which is generated according to the character model generation method of any one of claims 1 to 12.
15. A method for generating character models, characterized in that, include: Obtain the currently input character model to generate a dialogue request, wherein the request data carried in the character model to generate a dialogue request includes: input text, which is used to record the description information of the two-dimensional character model to be generated; In response to the character model generating dialogue request, a character model generating dialogue response is returned, wherein the information carried in the character model generating dialogue response includes: a two-dimensional character model, which is generated according to the character model generation method according to any one of claims 1 to 12; Render the two-dimensional character model within the graphical user interface.
16. A method for generating character models, characterized in that, include: In response to input commands applied to the user interface, the input text is displayed on the user interface. In response to processing instructions applied to the operation interface, a two-dimensional character model is rendered on the operation interface. The input text is used to record the description information of the two-dimensional character model to be generated, and the two-dimensional character model is generated according to the character model generation method of any one of claims 1 to 12.
17. A character model generation system, characterized in that, include: The client is used to send input text, which is used to record the descriptive information of the two-dimensional character model to be generated; A server, connected to the client, is configured to generate a two-dimensional character model according to the character model generation method described in any one of claims 1 to 12; The client is also used to render the two-dimensional character model.
18. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program executes the character model generation method according to any one of claims 1 to 16 when it runs.
19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the character model generation method according to any one of claims 1 to 16.
20. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the character model generation method according to any one of claims 1 to 16.
