Method, device and medium for generating a three-dimensional character model of a virtual character

Abstract

The present disclosure relates to a method and device for generating a three-dimensional character model of a virtual character and a medium, the method comprising: obtaining a two-dimensional character image of the virtual character; generating, according to the two-dimensional character image, a three-dimensional posture reference model representing a spatial posture of the virtual character and a plurality of independent three-dimensional components constituting a body of the virtual character; wherein the plurality of three-dimensional components correspond to different body parts of the body of the virtual character respectively; and performing spatial alignment and assembly on the plurality of three-dimensional components according to the three-dimensional posture reference model to generate the three-dimensional character model of the virtual character.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to a method, device, and medium for generating a three-dimensional character model of a virtual character. Background Technology

[0002] In recent years, the gaming industry has developed rapidly, and the generation of 3D virtual character models, as a core part of content production, directly affects product development efficiency and visual quality. Traditional 3D virtual character model production relies on professional artists manually modeling the entire process, which is not only time-consuming and labor-intensive but also costly.

[0003] With the development of artificial intelligence technology, image-to-3D (Image-to-3D) generation technology has been gradually applied to this field, achieving a breakthrough in rapidly generating 3D models from single 2D images and significantly shortening the production cycle of 3D models. However, most of the generated 3D models are indivisible monolithic mesh structures, and their topological layout does not meet the skeletal binding requirements of animation production. At the same time, the models lack sufficient precision in local details, are prone to texture blurring, and are difficult to meet the high-precision standards of industrial-grade production. Summary of the Invention

[0004] One objective of this disclosure is to provide a new technical solution for generating three-dimensional character models of virtual characters.

[0005] According to a first aspect of this disclosure, a method for generating a 3D character model of a virtual character is provided, comprising: Obtain a two-dimensional character image of the virtual character; Based on the two-dimensional character image, a three-dimensional posture reference model representing the spatial posture of the virtual character and multiple independent three-dimensional components constituting the virtual character body are generated; wherein, the multiple three-dimensional components correspond to different body parts of the virtual character body. Based on the three-dimensional posture reference model, multiple three-dimensional components are spatially aligned and assembled to generate a three-dimensional character model of the virtual character.

[0006] Optionally, based on the two-dimensional character image, multiple independent three-dimensional components of the virtual character body are generated, including: Obtain the topological features of the virtual character entity represented by the two-dimensional character image; Based on the topological features, the two-dimensional character image is semantically segmented to obtain two-dimensional part images corresponding to each part of the virtual character body; Based on the two-dimensional part image of each of the main body parts, generate a multi-view image set corresponding to the main body parts; Based on the multi-view image set of the main body part, generate a three-dimensional component corresponding to the main body part. Optionally, the step of semantically segmenting the two-dimensional character image based on the topological features to obtain two-dimensional part images corresponding to each part of the virtual character body includes: Based on the topological features, multiple body parts of the virtual character and part masks corresponding to each body part are determined; wherein, the part masks are used to identify the pixel regions of the body parts in the two-dimensional character image; Based on each of the aforementioned part masks, a two-dimensional part image corresponding to the body part is extracted from the two-dimensional character image.

[0007] Optionally, the method further includes: Detect whether the visual information of any of the two-dimensional parts represented by the image is complete; When the visual information of the part represented by the two-dimensional part image is incomplete, an image repair operation is performed on the two-dimensional part image to complete the missing part visual information.

[0008] Optionally, generating a three-dimensional component corresponding to the body part based on the multi-view image set of the body part includes: For each of the body parts, the multi-view image set of the body part is input into a pre-generated 3D image generation network to generate the 3D component corresponding to the body part.

[0009] Optionally, the multi-view image set includes at least two of the following: front view, side view, and top view.

[0010] Optionally, the step of assembling the three-dimensional components corresponding to each part into a three-dimensional character model of the virtual character based on the three-dimensional posture reference model includes: Determine the target position and spatial attitude of each of the body parts in the three-dimensional attitude reference model; Based on the target position and the spatial attitude, a target spatial transformation is obtained to position each of the three-dimensional components to the corresponding body part in the three-dimensional attitude reference model; The target space transformation is applied to the corresponding three-dimensional part to generate a three-dimensional character model of the virtual character.

[0011] Optionally, the method further includes: A skeleton to be bound is generated based on the three-dimensional components in the three-dimensional character model, and the skeleton to be bound corresponds one-to-one with the three-dimensional component; The 3D character model is rigged and / or skinned based on the bones to be rigged.

[0012] According to a second aspect of this disclosure, an electronic device is provided, including a memory and a processor, the memory being configured to store a computer program, and the processor being configured to execute the method described according to the first aspect of this disclosure under the control of the computer program.

[0013] According to a third aspect of this disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, the computer program implementing the method according to a first aspect of this disclosure when executed by a processor.

[0014] One beneficial effect of this embodiment is that the method can generate a 3D pose reference model representing the spatial posture of the virtual character based on the 2D character image, and 3D components corresponding to different parts of the virtual character's body. Then, based on the 3D pose reference model, multiple 3D components are spatially aligned and assembled to generate a 3D character model of the virtual character. It can automatically produce a 3D character model with a component-based structure, directly adapting to skeletal binding, significantly reducing secondary optimization costs. Furthermore, by employing a layered strategy of pose anchoring and part-specific generation, it ensures the accuracy of the overall posture of the virtual character while significantly improving the accuracy of local detail restoration. It eliminates the need for manual intervention by professional artists throughout the entire process, improving the efficiency of 3D character model generation while ensuring that the produced model meets industrial-grade high-precision standards.

[0015] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.

[0017] Figure 1a A schematic diagram of the hardware structure of an electronic device that can be used to implement a method for generating a three-dimensional character model of a virtual character according to embodiments of the present disclosure is shown. Figure 1b The diagram illustrates application scenarios of an electronic device according to some embodiments; Figure 2 A flowchart illustrating a method for generating a 3D character model of a virtual character according to a first embodiment is shown. Figure 3 A schematic diagram of a two-dimensional character image according to the first embodiment is shown; Figure 4 A schematic diagram of a preset image database according to the first embodiment is shown; Figure 5A flowchart illustrating a method for generating a 3D character model of a virtual character according to a second embodiment is shown. Figure 6 A flowchart illustrating a method for generating a 3D character model of a virtual character according to a third embodiment is shown. Figure 7 A flowchart illustrating a three-dimensional character model generation apparatus for virtual characters according to some embodiments is shown; Figure 8 A block diagram of an electronic device according to some embodiments is shown. Detailed Implementation

[0018] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0019] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the embodiments of this specification or their application or use.

[0020] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0021] It should be noted that all actions involving the acquisition of signals, information, or data in this embodiment are carried out in compliance with the relevant data protection laws and regulations of the country where the location is situated, and with authorization from the owner of the relevant equipment.

[0022] This disclosure provides a method for generating a 3D character model of a virtual character. For ease of understanding, the application scenarios of the 3D character model generation method provided in this disclosure are illustrated below. Figure 1a As shown, Figure 1a This is a schematic diagram of the hardware structure of an electronic device for generating a three-dimensional character model of a virtual character according to an embodiment of the present disclosure.

[0023] Electronic device 1000 is a device capable of running computer programs. These computer programs can be local applications installed on the electronic device, or web applications, lightweight applications, or mini-programs, etc., without limitation. The electronic device 1000 can be a mobile phone, tablet computer, PC, etc., without limitation.

[0024] like Figure 1a As shown, the electronic device 1000 may include a processor 1101, a memory 1102, an interface device 1103, a communication device 1104, an output device 1105, an input device 1106, etc. Figure 1a The hardware configuration shown is illustrative only and is not intended to limit this disclosure, its application, or its use.

[0025] The processor 1101 executes computer programs, which can be written using instruction sets of architectures such as x86, Arm, RISC, MIPS, and SSE. The memory 1102 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk. The interface device 1103 includes, for example, a USB interface, a network cable interface, and a headphone jack. The communication device 1104 is capable of wired or wireless communication. The communication device 1104 may include at least one short-range communication module, such as any module for short-range wireless communication based on short-range wireless communication protocols such as Hilink, WiFi (IEEE 802.11), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, and LiFi. The communication device 1104 may also include a long-range communication module, such as any module for WLAN, GPRS, or 2G / 3G / 4G / 5G long-range communication. The output device 1105 may include, for example, an LCD screen or touch screen, and a speaker. Input device 1106 may include, for example, a touch screen, a keyboard, a microphone, various sensors, etc.

[0026] In this embodiment, the memory 1102 of the electronic device 1000 is used to store a computer program that controls the processor 1101 to perform an operation to execute a method for generating a three-dimensional character model of a virtual character according to any embodiment of the present disclosure.

[0027] Reference Figure 1b The electronic device 1000 first acquires a two-dimensional character image 11 of the virtual character. It then employs a dual-branch parallel processing strategy. Branch 1 generates a low-precision three-dimensional pose reference model based on the two-dimensional character image. This three-dimensional pose reference model can completely represent the overall spatial pose of the virtual character. Branch 2 generates multiple high-precision independent three-dimensional components that constitute the virtual character's body based on the two-dimensional character image, with each three-dimensional component corresponding one-to-one with different parts of the virtual character's body. Next, the electronic device aligns and assembles these multiple three-dimensional components spatially based on the three-dimensional pose reference model to generate a three-dimensional character model of the virtual character.

[0028] The following is combined Figure 1a The electronic device shown illustrates the method for generating a 3D character model of a virtual character according to embodiments of this disclosure. It should be noted that... Figure 1a This is merely one application scenario of the method for generating a 3D character model of a virtual character provided in this disclosure embodiment, and does not mean that the method for generating a 3D character model of a virtual character can only be applied to... Figure 1a The application scenarios shown.

[0029] <First Embodiment> Figure 2 The diagram illustrates a flowchart of a method for generating a 3D character model of a virtual character according to some embodiments. This method is implemented by an electronic device, such as... Figure 1a The electronic device 1000 may include the following steps S210 to S230: Step S210: Obtain a two-dimensional character image of the virtual character.

[0030] Virtual characters can be objects that move in a virtual scene, including but not limited to people and other non-human living beings (such as dogs, cats, etc.). The virtual scene mentioned above can be a virtual activity space that simulates the real world provided to users by entertainment products such as games with mobile interfaces.

[0031] Among them, a two-dimensional (2D) character image can be an image that can represent the visual features and basic form of a virtual character, and a two-dimensional character image can be the original two-dimensional character image. (Refer to...) Figure 3 Taking a virtual character as an example, a two-dimensional character image 32 of the virtual character 31 can be obtained.

[0032] In this embodiment, the 2D character image of the virtual character acquired by the electronic device can be a single 2D character image, such as providing only a single perspective view of the virtual character, such as a front view or a side view. Alternatively, the acquired 2D character image of the virtual character can be multiple 2D character images, such as different perspective views of the virtual character, such as a front view, a side view, and a top view. Here, by combining images of different poses, the three-dimensional structure of the virtual character, the spatial relationships of its various body parts, and multi-dimensional details can be comprehensively covered.

[0033] In one example, an electronic device can actively acquire a two-dimensional image of a virtual character. For instance, the electronic device can provide an input interface through which a user can input a two-dimensional image of the virtual character, which the electronic device then receives. Alternatively, the electronic device can also acquire a two-dimensional image of the virtual character by using an image acquisition device such as a camera or scanner to capture the original two-dimensional image of the virtual character.

[0034] In one example, the electronic device can retrieve a stored 2D original image of the virtual character from a pre-set image database. Typically, this pre-set image database can store 2D original images of various virtual characters in a virtual scene from different perspectives and / or different poses. (See reference...) Figure 4The preset image database 41 stores, for example, the original 2D character images 42, 43, and 44 of virtual character 1, and the original 2D character images 45, 46, and 47 of virtual character 2. Among them, the original 2D character images 42, 43, and 44 are the original 2D character images of virtual character 1 from three different perspectives, and the original 2D character images 45, 46, and 47 are the original 2D character images of virtual character 2 from three different perspectives.

[0035] In one example, the electronic device can also preprocess the acquired 2D character image of the virtual character, such as performing noise reduction, cropping, resolution adjustment, and / or background stripping, to generate a valid 2D character image that meets the requirements of subsequent generation processes. That is, the 2D character image mentioned below can be a valid 2D character image generated after preprocessing.

[0036] In this embodiment, the two-dimensional character image typically needs to meet the following requirements: it should be able to clearly present the overall outline of the virtual character, the body part segmentation features of the virtual character body such as the boundary distinction of the head, torso, and limbs, and key appearance details such as limb proportions and posture trends; and the image resolution and / or clarity should be able to support the accuracy requirements of subsequent semantic segmentation and 3D generation processes, so as to ensure the accuracy of the generation of 3D components corresponding to different body parts of the virtual character body and the 3D posture reference model.

[0037] Step S320: Based on the two-dimensional character image, generate a three-dimensional posture reference model representing the spatial posture of the virtual character and multiple independent three-dimensional components constituting the virtual character body; wherein, the multiple three-dimensional components correspond to different body parts of the virtual character body.

[0038] The 3D pose reference model can be a low-mesh model with a mesh count below a first threshold, while the 3D parts can be high-mesh models with a mesh count above a second threshold, where the second threshold is typically greater than the first. Typically, the 3D pose reference model is used to anchor the overall spatial pose and limb proportions of a virtual character. The 3D parts can reproduce the detailed structure of corresponding body parts of the virtual character, and each 3D part can be independently disassembled, with a regular mesh structure and a topological structure suitable for subsequent skeletal rigging.

[0039] In this embodiment, after the electronic device acquires a two-dimensional character image of the virtual character, it can employ a dual-branch parallel processing strategy. One branch can generate a low-precision three-dimensional pose reference model based on the two-dimensional character image. This three-dimensional pose reference model can completely represent the overall spatial pose of the virtual character. The other branch can generate multiple high-precision independent three-dimensional components that constitute the virtual character's body based on the two-dimensional character image, with each three-dimensional component corresponding one-to-one with different body parts of the virtual character's body.

[0040] In one example, an electronic device can input a two-dimensional character image into a pre-generated three-dimensional image generation network. This network, trained with samples, is capable of inferring three-dimensional spatial morphology from two-dimensional images. Through feature extraction and spatial mapping, it can directly output a three-dimensional pose reference model. This model is mainly used to characterize the overall skeletal structure, joint angles, and spatial pose of the virtual character, providing a spatial reference for subsequent assembly.

[0041] In one example, an electronic device can perform semantic segmentation on a two-dimensional character image, identify and extract the various body parts of the virtual character, obtain two-dimensional part images of each body part, and perform three-dimensional reconstruction based on the two-dimensional part images of each body part to generate a three-dimensional component of the corresponding body part.

[0042] For example, taking a virtual character as an example, each body part can be specifically divided into: head, neck, torso, right upper arm, left upper arm, right forearm, left forearm, right hand, left hand, right thigh, left thigh, right calf, left calf, right foot, and left foot. Through the above processing, the electronic device can obtain a set of high-precision three-dimensional components that correspond one-to-one with each body part.

[0043] Step S330: Based on the three-dimensional posture reference model, spatial alignment and assembly of multiple three-dimensional components are performed to generate a three-dimensional character model of the virtual character.

[0044] In this embodiment, the electronic device can use a three-dimensional posture reference model as a unified spatial benchmark to precisely match and non-adhesively assemble multiple independent three-dimensional components, ultimately generating a three-dimensional character model that meets industrial-grade precision standards and can be directly adapted to subsequent skeletal binding and skinning processing.

[0045] In some embodiments, the electronic device can first establish a spatial correspondence between each 3D component and its corresponding body part in the 3D posture reference model, and determine the target position and spatial posture of each body part in the 3D posture reference model based on the spatial correspondence. Then, based on the determined target position and spatial posture, the electronic device calculates the target spatial transformation that positions each 3D component to its corresponding body part in the 3D posture reference model. Finally, the electronic device applies the target spatial transformation to the corresponding 3D part to generate a 3D character model of the virtual character.

[0046] Through the embodiments of this disclosure, a three-dimensional posture reference model representing the spatial posture of a virtual character can be generated from a two-dimensional character image, along with three-dimensional components corresponding to different parts of the virtual character's body. Then, based on the three-dimensional posture reference model, multiple three-dimensional components are spatially aligned and assembled to generate a three-dimensional character model of the virtual character. This method can automatically produce three-dimensional character models with a component-based structure, directly adapting to skeletal binding, significantly reducing secondary optimization costs. Furthermore, by employing a layered strategy of posture anchoring and part-specific generation, it ensures the accuracy of the overall posture of the virtual character while significantly improving the accuracy of local detail reproduction. It eliminates the need for manual intervention by professional artists throughout the entire process, improving the efficiency of three-dimensional character model generation while ensuring that the produced model meets industrial-grade high-precision standards, achieving a dual improvement in efficiency and quality. This method is suitable for large-scale game content production scenarios.

[0047] <Second Embodiment> In this embodiment, in order to accurately generate multiple independent three-dimensional parts of the virtual character body based on the two-dimensional character image, a progressive method of topological structure guidance, semantic segmentation, and multi-view expansion can be used to generate three-dimensional parts corresponding to each body part. This can ensure the accuracy of part segmentation and improve the stereo accuracy and detail reproduction of the three-dimensional parts.

[0048] In these embodiments, relative to the first embodiment described above, step S220 may include the following steps S221 to S224: Step S221: Obtain the topological features of the virtual character ontology represented by the two-dimensional character image.

[0049] The topological features can include the overall outline topology of the virtual character body and the boundary topology of each part of the body.

[0050] Taking a virtual character as an example, the overall outline topology can be specifically represented by the connection topology of the torso and limbs, the transition topology of the head and neck, and the connection topology of each segment of the limbs, etc., which are macroscopic structural relationships. The boundary topology of each part of the body can be specifically represented by the outline of the upper arm and forearm, the connection boundary of the torso and pelvis, the connection boundary of the palm and fingers, etc., which are microscopic outlines.

[0051] In this embodiment, the electronic device can use a topological feature extraction algorithm to mine the topological features that represent the virtual character ontology structure in a two-dimensional character image.

[0052] Taking virtual characters as an example, electronic devices can use topological feature extraction algorithms to extract 18-24 key feature points of the human body, and construct the topological structure features of the virtual character's body based on the spatial relationship and connection logic of these key feature points.

[0053] Step S222: Perform semantic segmentation on the two-dimensional character image based on topological features to obtain two-dimensional part images corresponding to each part of the virtual character body.

[0054] In this embodiment, the electronic device can input a two-dimensional character image into a preset semantic segmentation model. This model can then perform semantic segmentation operations based on topological features as constraints, outputting two-dimensional part images corresponding to each part of the virtual character body. That is, by using topological constraints, the virtual character body is accurately divided into multiple independent local images, and each local image uniquely corresponds to a part of the virtual character body, achieving the effect of part independence processing and providing a high-precision image input foundation for the subsequent generation of independent three-dimensional parts.

[0055] Step S223: Generate a multi-view image set for each body part based on the two-dimensional part image of each body part.

[0056] A multi-view image set can include images from at least two different perspectives, such as at least two of a front view, a side view, and a top view. For example, a multi-view image set can include orthogonal front views, side views, and top views.

[0057] In this embodiment, for each body part's two-dimensional part image, the electronic device can use the outline, detail texture, and topological features of the two-dimensional part image as a basis to generate images of the part from multiple different perspectives through a viewpoint conversion algorithm, thereby constructing a multi-view image set.

[0058] It should be noted that after generating the multi-view image set, the electronic device can also perform detail restoration operations on each image in the multi-view image set to ensure that the clarity and topological integrity of each view image meet the accuracy requirements of 3D reconstruction.

[0059] In this embodiment, constructing a multi-view image set can effectively compensate for the limitations of a single two-dimensional part image lacking spatial depth and three-dimensional structural information. The multi-view images provide multi-dimensional spatial constraints for subsequent three-dimensional modeling, significantly improving the three-dimensional accuracy and detail reproduction of the generated three-dimensional parts.

[0060] Step S224: Generate a three-dimensional component corresponding to the main body part based on the multi-view image set of the main body part.

[0061] In some embodiments, generating a three-dimensional component corresponding to a body part based on a multi-view image set of the body part can be achieved in the following way: for each body part, input the multi-view image set of the body part into a pre-generated three-dimensional image generation network to generate a three-dimensional component corresponding to the body part.

[0062] In this embodiment, for each body part corresponding to a multi-view image set, the electronic device can input the multi-view image set of a single body part into a pre-trained 3D image generation network. The 3D image generation network can accurately construct the 3D component of the corresponding body part by performing feature fusion, spatial dimension modeling and mesh generation on the multi-view image set.

[0063] It should be noted that the 3D image generation network here and the 3D image generation network used to generate the 3D pose reference model can be the same network or different networks; this embodiment does not impose any limitations.

[0064] This embodiment enables precise 3D reconstruction of a single part by leveraging a pre-trained 3D image generation network based on a multi-view image set specific to the generated body part. This not only ensures that the generated 3D parts have complete spatial morphology and accurate detail reproduction, but also ensures that each 3D part meets the core requirements of being independent and separable with a regular topological structure, and can be directly adapted to subsequent processes such as spatial alignment, part assembly, and skeleton binding.

[0065] <Third Embodiment> In this embodiment, the topological features of the virtual character body can be used to determine each body part and the corresponding part mask, which can effectively avoid segmentation defects such as boundary misalignment and cross-part adhesion, ensure that the part division conforms to the body structure logic of the virtual character, and thus improve the accuracy of the extracted two-dimensional part images.

[0066] In these embodiments, relative to the second embodiment described above, step S222, which performs semantic segmentation on the two-dimensional character image based on topological features to obtain two-dimensional part images corresponding to each part of the virtual character body, may include: determining multiple parts of the virtual character and part masks corresponding to each part based on topological features; and extracting two-dimensional part images of the corresponding parts from the two-dimensional character image based on each part mask.

[0067] The part mask can be used to identify the pixel region of the body part in a 2D character image. The part mask can be binary image data, where the pixel value of the pixel region corresponding to the body part can be set to a first preset value, such as 255, and the pixel region of non-body parts can be set to a second preset value, such as 0, so as to achieve pixel-level accurate differentiation between the body part and the background region.

[0068] In this embodiment, the electronic device can determine multiple standard body parts of the virtual character based on the extracted topological features, such as contour topology and boundary topology, combined with the ontological structure logic of the virtual character.

[0069] For each specific body part, the electronic device can generate a part mask for the corresponding body part through a mask generation algorithm. The resolution and pixel coordinate system of the part mask are completely consistent with the original two-dimensional character image, ensuring that the identification of the pixel area of ​​the target part has pixel-level accuracy.

[0070] This embodiment uses the topological features of a two-dimensional character image as the core constraint to accurately divide the virtual character body parts and define the pixel range of each part. By using part masks, it achieves the extraction of two-dimensional part images without redundancy, ensuring that the extracted two-dimensional part images have clear boundaries and that each part is independent, providing high-precision and compliant input for subsequent multi-view image generation and three-dimensional component construction.

[0071] <Fourth Embodiment> Figure 5 A flowchart illustrating a method for generating a 3D character model of a virtual character according to some embodiments is shown. Unlike the third embodiment described above, in this embodiment, after extracting a 2D part image of the corresponding body part from a 2D character image based on any part mask, if the visual information represented by the 2D part image is incomplete, an image inpainting operation is required to complete the missing information, thereby ensuring that the subsequently generated 3D components are consistent with the design intent of the original 2D character image. For example... Figure 5 As shown, the method for generating a 3D character model for a virtual character in this embodiment may include the following steps S510 to S540: Step S510: Based on the topological characteristics, determine multiple body parts of the virtual character and the corresponding part masks for each body part.

[0072] Step S520: Based on the mask of each part, extract the two-dimensional part image of the corresponding body part from the two-dimensional character image.

[0073] Step S530: Detect whether the visual information of any two-dimensional part image is complete.

[0074] In this embodiment, for any two-dimensional part image, it is possible to detect whether the visual information of the part represented by the two-dimensional part image is complete, such as identifying edge damage, texture loss and / or local area loss of the image, so as to locate the pixel coordinates of the missing area.

[0075] Step S540: When the visual information of the part represented by the two-dimensional part image is incomplete, perform an image repair operation on the two-dimensional part image to complete the missing visual information of the part.

[0076] In this embodiment, different repair methods can be used for different types of missing data: for example, for missing data with broken image edges, the contour can be completed using a boundary topology fitting algorithm based on the topological features of the area; for missing textures, a texture database with the same style as the original 2D character image can be retrieved, and texture filling can be used to ensure consistent texture distribution; for missing local areas, a generative repair model can be used to generate content topology, generating missing area content that perfectly matches the topological structure and detail features of the original image, thus avoiding structural misalignment or style inconsistency.

[0077] In this embodiment, the integrity of the visual information of the repaired two-dimensional part image can also be checked again. If the visual information of the repaired two-dimensional part image is incomplete, image repair operation can be performed on the repaired two-dimensional part image to complete the missing visual information of the part.

[0078] <Fifth Embodiment> Figure 6 A flowchart illustrating a method for generating a 3D character model of a virtual character according to some embodiments is shown. Unlike the first embodiment described above, in this embodiment, after spatially aligning and assembling multiple 3D components based on a 3D pose reference model to generate a 3D character model of the virtual character, skeletal rigging and skinning can be performed on the generated 3D character model to obtain an animation-driven 3D character model. For example... Figure 6 As shown, the method for generating a 3D character model of a virtual character in this embodiment may include the following steps S610 to S650: Step S610: Obtain a two-dimensional character image of the virtual character.

[0079] Referring to the first embodiment, different methods can be used to acquire two-dimensional character images in step S610. The acquired two-dimensional character images can clearly present the overall outline of the virtual character, the body part segmentation features of the virtual character body such as the boundary distinction of the head, torso, and limbs, and key appearance details such as limb proportions and posture trends. Furthermore, the image resolution and / or clarity can support the accuracy requirements of subsequent semantic segmentation and 3D generation processes, so that the generation accuracy of the 3D parts corresponding to the different body parts of the virtual character body is achieved.

[0080] Step S620: Based on the two-dimensional character image, generate a three-dimensional posture reference model representing the spatial posture of the virtual character and multiple independent three-dimensional components constituting the virtual character body; wherein, the multiple three-dimensional components correspond to different body parts of the virtual character body.

[0081] Referring to the first embodiment, a three-dimensional posture reference model representing the spatial posture of a virtual character and multiple independent three-dimensional components constituting the virtual character's body can be generated from a two-dimensional character image. The three-dimensional posture reference model can be used to anchor the overall spatial posture and limb proportions of the virtual character. The three-dimensional components can reproduce the detailed structure of corresponding body parts of the virtual character, and each three-dimensional component can be independently disassembled, with a regular mesh structure, naturally possessing a topological structure suitable for subsequent skeletal binding.

[0082] Step S630: Based on the three-dimensional posture reference model, spatial alignment and assembly of multiple three-dimensional components are performed to generate a three-dimensional character model of the virtual character.

[0083] Step S640: Generate bones to be bound based on the three-dimensional parts in the three-dimensional character model. The bones to be bound correspond one-to-one with the three-dimensional parts.

[0084] In this embodiment, the electronic device can configure dedicated skeletons for each independent 3D component of the 3D character model and establish a precise correspondence between the component and the skeleton. This ensures that the joint distribution and structural dimensions of the skeleton are highly compatible with the topology and kinematic requirements of each 3D component, providing a structural foundation for subsequent skeleton driving, motion simulation and animation production.

[0085] Step S650: Perform skeletal binding and / or skinning on the 3D character model based on the bones to be bound.

[0086] In this embodiment, through bone binding and skinning, an efficient and precise driving relationship can be established between the skeleton and the three-dimensional parts, so that the displacement, rotation and other movements of the skeleton can be synchronously transmitted to the corresponding parts, ensuring that the parts have a natural and smooth shape during the movement and deformation process, and finally generating a high-precision movable three-dimensional character model that can be directly connected to the animation production pipeline.

[0087] <Sixth Embodiment> This embodiment provides a three-dimensional character model generation device. Figure 7 A schematic diagram illustrating the composition of a three-dimensional character model generation apparatus for virtual characters according to an embodiment of the present disclosure is shown. Figure 7 As shown, the 3D character model generation device 700 for the virtual character includes an acquisition module 710, a first generation module 720, and a second generation module 730.

[0088] The acquisition module 710 is used to acquire a two-dimensional character image of the virtual character; The first generation module 720 is used to generate a three-dimensional posture reference model representing the spatial posture of the virtual character and multiple independent three-dimensional components constituting the virtual character body based on the two-dimensional character image; wherein, the multiple three-dimensional components respectively correspond to different body parts of the virtual character body. The second generation module 730 is used to spatially align and assemble multiple three-dimensional components according to the three-dimensional posture reference model to generate a three-dimensional character model of the virtual character.

[0089] In some embodiments, the first generation module 720 is specifically used to obtain the topological features of the virtual character body represented by the two-dimensional character image; perform semantic segmentation on the two-dimensional character image according to the topological features to obtain two-dimensional part images corresponding to each part of the virtual character body; generate a multi-view image set corresponding to each part of the body according to the two-dimensional part image of each part of the body; and generate a three-dimensional component corresponding to each part of the body according to the multi-view image set of the part of the body.

[0090] In some embodiments, the first generation module 720 is specifically used to determine multiple body parts of the virtual character and part masks corresponding to each body part based on the topological features; wherein the part mask is used to identify the pixel region of the body part in the two-dimensional character image; and to extract a two-dimensional part image corresponding to the body part from the two-dimensional character image based on each part mask.

[0091] In some embodiments, the device 700 further includes an image restoration module (not shown).

[0092] The image restoration module is used to detect whether the visual information of any of the two-dimensional parts represented by the image is complete; if the visual information of the two-dimensional parts represented by the image is incomplete, an image restoration operation is performed on the two-dimensional parts image to complete the missing visual information.

[0093] In some embodiments, the first generation module 720 is specifically used to input a multi-view image set of the body part into a pre-generated three-dimensional image generation network for each of the body parts to generate the three-dimensional component corresponding to the body part.

[0094] In some embodiments, the multi-view image set includes at least two of a front view, a side view, and a top view. In some embodiments, the second generation module 730 is specifically used to determine the target position and spatial posture of each of the body parts in the three-dimensional posture reference model; obtain the target spatial transformation for locating each of the three-dimensional parts to the corresponding body parts in the three-dimensional posture reference model based on the target position and the spatial posture; and apply the target spatial transformation to the corresponding three-dimensional parts to generate the three-dimensional character model of the virtual character.

[0095] In some embodiments, the device 700 further includes a third generation module (not shown in the figure).

[0096] The third generation module is used to generate skeletons to be bound based on the three-dimensional parts in the three-dimensional character model, wherein the skeletons to be bound correspond one-to-one with the three-dimensional parts; and to perform skeleton binding and / or skinning processing on the three-dimensional character model based on the skeletons to be bound.

[0097] <Seventh Embodiment> This embodiment provides an electronic device for implementing a three-dimensional character model generation method according to any embodiment of this disclosure. Figure 8 The basic hardware components of this electronic device are shown. For example... Figure 8 As shown, the electronic device 800 includes a processor 810 and a memory 820. The memory 820 stores a computer program that controls the processor 810 to perform an operation to control the electronic device 800 to execute a method for generating a three-dimensional character model of a virtual character according to any embodiment of the present disclosure.

[0098] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for generating a three-dimensional character model of a virtual character according to any embodiment of this disclosure.

[0099] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0100] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0101] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0102] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.

[0103] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, can execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0104] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0105] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0106] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0108] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for generating a 3D character model for a virtual character, wherein, include: Obtain a two-dimensional character image of the virtual character; Based on the two-dimensional character image, a three-dimensional posture reference model representing the spatial posture of the virtual character and multiple independent three-dimensional components constituting the virtual character body are generated; wherein, the multiple three-dimensional components correspond to different body parts of the virtual character body. Based on the three-dimensional posture reference model, multiple three-dimensional components are spatially aligned and assembled to generate a three-dimensional character model of the virtual character.

2. The method according to claim 1, wherein, Based on the two-dimensional character image, multiple independent three-dimensional components of the virtual character body are generated, including: Obtain the topological features of the virtual character entity represented by the two-dimensional character image; Based on the topological features, the two-dimensional character image is semantically segmented to obtain two-dimensional part images corresponding to each part of the virtual character body; Based on the two-dimensional part image of each of the main body parts, generate a multi-view image set corresponding to the main body parts; Based on the multi-view image set of the main body part, generate a three-dimensional component corresponding to the main body part.

3. The method according to claim 2, wherein, The step of semantically segmenting the two-dimensional character image based on the topological features to obtain two-dimensional part images corresponding to each part of the virtual character body includes: Based on the topological features, multiple body parts of the virtual character and part masks corresponding to each body part are determined; wherein, the part masks are used to identify the pixel regions of the body parts in the two-dimensional character image; Based on each of the aforementioned part masks, a two-dimensional part image corresponding to the body part is extracted from the two-dimensional character image.

4. The method according to claim 3, wherein, The method further includes: Detect whether the visual information of any of the two-dimensional parts represented by the image is complete; When the visual information of the part represented by the two-dimensional part image is incomplete, an image repair operation is performed on the two-dimensional part image to complete the missing part visual information.

5. The method according to claim 2, wherein, The step of generating a three-dimensional component corresponding to the body part based on the multi-view image set of the body part includes: For each of the body parts, the multi-view image set of the body part is input into a pre-generated 3D image generation network to generate the 3D component corresponding to the body part.

6. The method according to claim 2, wherein, The multi-view image set includes at least two of the following: front view, side view, and top view.

7. The method according to claim 1, wherein, The step of assembling the three-dimensional components corresponding to each part into a three-dimensional character model of the virtual character based on the three-dimensional posture reference model includes: Determine the target position and spatial attitude of each of the body parts in the three-dimensional attitude reference model; Based on the target position and the spatial attitude, a target spatial transformation is obtained to position each of the three-dimensional components to the corresponding body part in the three-dimensional attitude reference model; The target space transformation is applied to the corresponding three-dimensional part to generate a three-dimensional character model of the virtual character.

8. The method according to any one of claims 1 to 6, wherein, The method further includes: A skeleton to be bound is generated based on the three-dimensional components in the three-dimensional character model, and the skeleton to be bound corresponds one-to-one with the three-dimensional component; The 3D character model is rigged and / or skinned based on the bones to be rigged.

9. An electronic device, wherein, It includes a memory and a processor, the memory being used to store a computer program, and the processor being used, under the control of the computer program, to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, wherein, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images