Virtual object generation method and device, medium, electronic equipment and program product

Abstract

The invention belongs to the technical field of artificial intelligence, and particularly relates to a virtual object generation method, a virtual object generation device, a computer readable medium, electronic equipment and a computer program product. The method comprises the following steps: acquiring a three-dimensional model of an object body; according to the attachment position of the attachment on the object body, intercepting a local area model corresponding to the attachment position from the three-dimensional model of the object body; and generating a three-dimensional model of the attachment according to the local area model. The expandability and the control flexibility of the three-dimensional model can be improved.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a method for generating a virtual object, a device for generating a virtual object, a computer-readable medium, an electronic device, and a computer program product. Background Art

[0002] A virtual 3D model is a model created in a 3D space to describe the shape, structure and appearance of an object. This model plays an important role in many fields, including architecture, game development, medicine, engineering and education. 3D models can provide intuitive display and communication methods, effectively assisting decision-making and innovation.

[0003] In the construction industry, 3D models are used in architectural design, visualization, construction planning, etc. In game development, 3D models are important tools for developing game scenes, characters, and animations. In the medical field, 3D models are used in medical image processing, simulated surgery, pathological analysis, etc. Engineers can use 3D models for product design, simulation, and analysis, while the education field is also using 3D models for teaching assistance to help students understand abstract concepts more deeply.

[0004] Traditional three-dimensional models usually have an integrated structure, which maintains the consistent display effect of each part of the model, but also limits the model's expansion capabilities and has the problem of poor model control flexibility. Summary of the invention

[0005] The present application provides a method for generating a virtual object, a device for generating a virtual object, a computer-readable medium, an electronic device, and a computer program product, aiming to improve the scalability and control flexibility of a three-dimensional model.

[0006] According to one aspect of an embodiment of the present application, a method for generating a virtual object is provided, wherein the virtual object includes an object body and attachments, and the generating method includes: acquiring a three-dimensional model of the object body; based on the attachment position of the attachment on the object body, intercepting a local area model corresponding to the attachment position from the three-dimensional model of the object body; and generating a three-dimensional model of the attachment based on the local area model.

[0007] According to one aspect of an embodiment of the present application, a device for generating a virtual object is provided, wherein the virtual object includes an object body and attachments, and the generating device includes: an acquisition module configured to acquire a three-dimensional model of the object body; a capture module configured to capture, based on the attachment position of the attachment on the object body, a local area model corresponding to the attachment position from the three-dimensional model of the object body; and a generation module configured to generate a three-dimensional model of the attachment based on the local area model.

[0008] In some embodiments of the present application, based on the above technical solution, the three-dimensional model includes a main structure and a secondary structure bound to the surface of the main structure, the main structure is used to represent a grid frame in three-dimensional space, and the secondary structure is used to represent a plane figure covered on the surface of the grid frame.

[0009] In some embodiments of the present application, based on the above technical solution, the interception module may further include: an identification module, configured to identify a local grid frame corresponding to the attachment position on the main structure of the object body; a release module, configured to intercept a local plane graphic bound to the local grid frame on the secondary structure of the object body, and release the binding relationship between the local plane graphic and the local grid frame to obtain a local area model composed of the local plane graphic.

[0010] In some embodiments of the present application, based on the above technical solution, the generation module may further include: a main structure establishment module, configured to establish the main structure of the attachment according to the local area model; a secondary structure binding module, configured to bind the secondary structure of the attachment on the surface of the main structure of the attachment; a secondary structure updating module, configured to fix the model parameters of the main structure of the attachment and update the model parameters of the secondary structure of the attachment to obtain a three-dimensional model of the attachment.

[0011] In some embodiments of the present application, based on the above technical solution, the secondary structure updating module may further include: a rendering module, configured to render the main structure of the attachment and the secondary structure of the attachment together to obtain a rendered image of the attachment; a denoising module, configured to add noise data to the rendered image of the attachment to obtain a noisy image of the attachment; a denoising module, configured to denoise the noise image of the attachment according to the prompt information used to describe the attachment to obtain a denoised image of the attachment; an updating module, configured to obtain a loss error between the rendered image of the attachment and the denoised image of the attachment, and update the model parameters of the secondary structure of the attachment according to the loss error.

[0012] In some embodiments of the present application, based on the above technical solution, the generation module may also include: a local model rendering module, configured to jointly render the three-dimensional model of the object body and the local area model to obtain a rendered image of the virtual object; a local model updating module, configured to update the model parameters of the local area model according to the prompt information used to describe the attachment and the rendered image of the virtual object.

[0013] In some embodiments of the present application, based on the above technical solution, the local model update module can be further configured to: add noise data to the rendered image of the virtual object to obtain a noise image of the virtual object; denoise the noise image of the virtual object according to the prompt information used to describe the attachment to obtain a denoised image of the virtual object; obtain the loss error between the rendered image of the virtual object and the denoised image of the virtual object, and update the model parameters of the local area model according to the loss error.

[0014] In some embodiments of the present application, based on the above technical solution, the local model update module can be further configured to: perform image segmentation processing on the rendered image of the virtual object according to the prompt information used to describe the attachment, and obtain the contour mask of the attachment; obtain the loss error between the contour mask of the attachment and the contour area of ​​the local area model, and update the model parameters of the local area model according to the loss error.

[0015] In some embodiments of the present application, based on the above technical solution, the acquisition module may further include: a main structure extraction module, configured to extract the main structure of the object body from a preset object parameter model; a secondary structure binding module, configured to bind the secondary structure of the object body on the main structure surface of the object body; a structure updating module, configured to update the main structure of the object body and the secondary structure of the object body respectively according to the prompt information used to describe the object body.

[0016] In some embodiments of the present application, based on the above technical solution, the main structure update part in the structure update module can be configured as follows: rendering the main structure of the object body to obtain a first rendered image of the object body; adding noise data to the first rendered image of the object body to obtain a first noise image of the object body; denoising the first noise image of the object body according to the prompt information used to describe the object body to obtain a first denoised image of the object body; obtaining a loss error between the first rendered image of the object body and the first denoised image of the object body, and updating the main structure of the object body according to the loss error.

[0017] In some embodiments of the present application, based on the above technical solution, the secondary structure update part in the structure update module can be configured to: render the main structure and the secondary structure of the object body together to obtain a second rendered image of the object body; add noise data to the second rendered image of the object body to obtain a second noise image of the object body; denoise the second noise image of the object body according to the prompt information used to describe the object body to obtain a second denoised image of the object body; obtain the loss error between the second rendered image of the object body and the second denoised image of the object body, and update the secondary structure of the object body according to the loss error.

[0018] In some embodiments of the present application, based on the above technical solution, the device for generating the virtual object may further include a texture enhancement module, and the texture enhancement module is configured to: render the three-dimensional model of the object body and / or the attachment to obtain a rendered image of the three-dimensional model; add noise data to the rendered image of the three-dimensional model to obtain a noise image of the three-dimensional model; denoise the noise image of the three-dimensional model according to the prompt information used to describe the three-dimensional model to obtain a denoised image of the three-dimensional model; obtain a loss error between the rendered image of the three-dimensional model and the denoised image of the three-dimensional model, and update the three-dimensional model according to the loss error.

[0019] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for generating a virtual object in the above technical solution is implemented.

[0020] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement a method for generating a virtual object as in the above technical solution.

[0021] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method for generating a virtual object in the above technical solution is implemented.

[0022] In the technical solution provided in the embodiment of the present application, the three-dimensional model of the object body is first obtained, and then the local area model corresponding to the attachment position is intercepted from the three-dimensional model of the object body according to the attachment position of the attachment on the object body, and finally the three-dimensional model of the attachment is generated according to the local area model; the three-dimensional models of the object body and the attachment are generated in stages by adopting the object structure decomposition method, which can overcome the technical problems such as poor model scalability and poor controllable flexibility caused by integrated molding; moreover, the embodiment of the present application utilizes the attachment relationship between the object body and the attachment to accurately intercept the local area model matching the attachment position of the attachment from the three-dimensional model of the object body, and then adjust the initialization parameters based on the local area model to generate the corresponding three-dimensional model of the attachment, which can improve the generation efficiency of the three-dimensional model of the attachment on the one hand, and maintain the attachment matching relationship between the attachment and the object body on the other hand, and improve the refinement and detail expression of the three-dimensional model of the attachment; finally, the three-dimensional model composed of the object body and the attachment has stronger authenticity, scalability and controllable flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0025] Figure 2 A flowchart of a method for generating a virtual object in one embodiment of the present application is shown.

[0026] Figure 3 A schematic diagram showing a three-dimensional model formed by combining Mesh grids and 3D-GS points in an application scenario according to an embodiment of the present application.

[0027] Figure 4 A flowchart of obtaining a three-dimensional model of an object body in one embodiment of the present application is shown.

[0028] Figure 5 A schematic diagram of a process for establishing a three-dimensional model of a human body in an application scenario according to an embodiment of the present application is shown.

[0029] Figure 6 A flow chart of generating a three-dimensional model of an attachment in one embodiment of the present application is shown.

[0030] Figure 7An application scenario of a method for updating model parameters of two local area models is shown in one embodiment of the present application.

[0031] Figure 8 An application scenario of updating the secondary structure model parameters of an attachment in one embodiment of the present application is shown.

[0032] Fig. 9 The virtual object generation process performed in an application scenario according to an embodiment of the present application is shown.

[0033] Fig.10 It shows that in an application scenario, an embodiment of the present application realizes an image display effect of changing clothes of a human body by generating multiple three-dimensional models of attachments.

[0034] Fig.11 The embodiment of the present application shows the image display effect of a person object wearing different clothing forms in an application scenario.

[0035] Fig.12 The structural block diagram of the device for generating a virtual object provided in an embodiment of the present application is schematically shown.

[0036] Fig.13 The structure block diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0038] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0039] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0040] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0041] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0042] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0043] like Figure 1 As shown, the system architecture 100 may include a terminal device 110, a network 120 and a server 130. The terminal device 110 may include various electronic devices such as smart phones, tablet computers, laptops, desktop computers, smart speakers, smart wearable devices, smart vehicle-mounted devices, and smart payment terminals. The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The network 120 may be a communication medium of various connection types that can provide a communication link between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0044] According to the implementation requirements, the system architecture in the embodiment of the present application can have any number of terminal devices, networks and servers. For example, the server 130 can be a server group composed of multiple server devices. In addition, the technical solution provided in the embodiment of the present application can be applied to the terminal device 110, can also be applied to the server 130, or can be implemented by the terminal device 110 and the server 130 together, and the present application does not make any special restrictions on this.

[0045] Taking the method for generating a virtual object jointly performed by the terminal device 110 and the server 130 as an example, the user can send a request message for generating a virtual object to the server 130 by operating the terminal device 110. After parsing the received request message, the server 130 obtains key information of the virtual object to be generated, such as the type, size, appearance, and other information of the virtual object. Then, based on the pre-configured model parameters combined with the key information obtained by parsing the request message, the server 130 can generate a three-dimensional model that meets the user's needs. After the data of the three-dimensional model is sent to the terminal device 110, the generated three-dimensional model, such as the human body model shown in the figure, can be displayed to the user through the interactive interface of the terminal device 110. In addition, the terminal device 110 can make structural and appearance adjustments to the three-dimensional model displayed on the interactive interface according to the user's instructions, so that the three-dimensional model presents a display effect that meets the user's needs.

[0046] by Figure 1 The human body model shown is taken as an example. In the related technology of this application, the human body and external appendages such as clothes are usually mixed together to perform an integrated three-dimensional model generation scheme. Although this scheme can obtain an overall consistent display effect, it also limits the expansion capability and control flexibility of the three-dimensional model. On the one hand, the clothes of the generated human body three-dimensional model are fixed, and we cannot change the clothes of the generated human body three-dimensional model, which will greatly limit the application scenarios of the human body three-dimensional model and its application in downstream tasks. On the other hand, since the clothes and the human body are integrated, during the driving process of the human body three-dimensional model, we cannot simulate the deformation process of the clothes during the movement. The clothes will fit tightly to the human body, which will greatly reduce the realism of the human body movement picture.

[0047] In view of the problems existing in the above related technologies, this application provides a virtual object generation solution that decouples the object body from the attachments. For example, it can realize a drivable digital human generation technology that supports decoupling the human body and clothes. The generated digital human and clothes can be used as digital assets in downstream tasks with different requirements. For example, games, AR / VR virtual social networking, virtual human live broadcast, virtual human interaction and other tasks.

[0048] In addition, in order to facilitate the explanation of the implementation details of the technical solution, the various embodiments of the present application mainly use digital humans as examples of virtual objects, but the present application is not limited to this.

[0049] In some application scenarios, virtual objects can be three-dimensional models generated by virtual modeling of any physical object, such as human bodies, animal bodies, buildings, vehicles, daily necessities or office supplies, etc. In other application scenarios, virtual objects can be three-dimensional models without physical images designed and shaped based on the user's imagination, such as fictional characters, mythical beasts, pets, mounts, props, environmental spaces, etc. in game scenes.

[0050] The following is a detailed description of the technical solutions such as the method for generating a virtual object, the device for generating a virtual object, the computer-readable medium, the electronic device, and the computer program product provided by the present application in conjunction with specific implementation methods.

[0051] Figure 2 A flowchart of a method for generating a virtual object in one embodiment of the present application is shown. The method for generating a virtual object can be Figure 1 The method is executed by the terminal device or the server shown, or can be executed by the terminal device and the server together. The embodiment of the present application takes the method for generating a virtual object executed by the terminal device as an example for explanation.

[0052] like Figure 2 As shown, the virtual object in the embodiment of the present application includes an object body and attachments, and the method for generating the virtual object may include the following steps S210 to S230.

[0053] S210: Acquire a three-dimensional model of the object body.

[0054] In the embodiment of the present application, the virtual object to be generated can be firstly structurally decomposed to form an object body as a main structure and attachments as auxiliary structures. For example, when the virtual object is a character, the object body can be the body of the character, and the attachments can include clothes worn on the body, accessories worn, or tools used.

[0055] In one embodiment of the present application, the terminal device can obtain a pre-established three-dimensional model of an object entity from a local database or a remote server by initiating an acquisition request. The present application embodiment can pre-build and store a variety of different types of three-dimensional models of object entities, which can be selected and used by users when needed. For example, three-dimensional models of the bodies of people of different genders, ages, and races can be built on the server, and when the terminal device initiates an acquisition request, the pre-built three-dimensional model can be downloaded from the server to the local computer.

[0056] In another embodiment of the present application, the terminal device can generate a three-dimensional model of the object in real time using the locally stored model data and the user-defined parameters input by the user. For example, the user can input a prompt information for describing the figure of a person, and then the terminal device generates a three-dimensional model that conforms to the description effect of the prompt information based on the model data stored in the object. The prompt information can include information in any media form such as text, image, audio, video, etc.

[0057] S220: According to the attachment position of the attachment on the object body, a local area model corresponding to the attachment position is intercepted from the three-dimensional model of the object body.

[0058] There is a specified attachment relationship between the attachment and the object body, which specifically includes the attachment position of the attachment on the object body. Taking the object body as an example, when the attachment is the clothes worn by the character, the attachment position may include the part of the body covered by the clothes, such as the chest, abdomen, limbs and torso; when the attachment is an accessory worn by the character, the attachment position may include the wearing position of the accessory, such as the head, face, neck, wrist or hand; when the attachment is a tool used by the character, the attachment position may include the holding or assembly position of the tool, such as fingers, arms, legs or feet.

[0059] According to the attachment position of the attachment on the object body, the local area model corresponding to the attachment position can be intercepted from the three-dimensional model of the object body. For example, when the attachment is the clothes worn by the upper body of the character, the chest, abdomen and arm parts can be intercepted from the three-dimensional model of the character's body to form the local area model. For another example, when the attachment is the clothes worn by the lower body of the character, the abdomen and legs can be intercepted from the three-dimensional model of the character's body to form the local area model.

[0060] S230: Generate a three-dimensional model of the attachment according to the local area model.

[0061] The intercepted local area model is used as the initialization model, and the parameters of the local area model can be adjusted according to the prompt information describing the attachment, so as to generate a three-dimensional model of the attachment. For example, the user can enter a prompt text to describe the clothes worn by the character, and add the corresponding offset to the position parameters in the local area model according to the tightness of the clothes to form the framework of the clothes model. Then, the material and color parameters can be adjusted to finally form a clothes model that meets the user's description.

[0062] In the method for generating a virtual object provided in an embodiment of the present application, a three-dimensional model of the object body is first obtained, and then a local area model corresponding to the attachment position is intercepted from the three-dimensional model of the object body according to the attachment position of the attachment on the object body, and finally a three-dimensional model of the attachment is generated according to the local area model; the three-dimensional models of the object body and the attachment are generated in stages by decomposing the object structure, which can overcome the technical problems such as poor model scalability and poor controllable flexibility caused by integrated molding; moreover, the embodiment of the present application utilizes the attachment relationship between the object body and the attachment to accurately intercept a local area model matching the attachment position of the attachment from the three-dimensional model of the object body, and then adjust the initialization parameters based on the local area model to generate the corresponding three-dimensional model of the attachment, which can improve the generation efficiency of the three-dimensional model of the attachment on the one hand, and maintain the attachment matching relationship between the attachment and the object body on the other hand, and improve the refinement and detail expression of the three-dimensional model of the attachment; finally, the three-dimensional model composed of the object body and the attachment has stronger real reliability, scalability and controllable flexibility.

[0063] In one embodiment of the present application, the three-dimensional model includes a main structure and a secondary structure bound to the surface of the main structure, the main structure is used to represent a grid frame in three-dimensional space, and the secondary structure is used to represent a plane figure covered on the surface of the grid frame.

[0064] In the method for generating a virtual object provided in the embodiment of the present application, corresponding structural models are established at the two levels of three-dimensional space and two-dimensional space respectively, and a composite model composed of two model expressions is obtained, which can take into account the three-dimensional operational controllability and two-dimensional visual stability of the model. Specifically, at the three-dimensional space level, a main structure as a grid framework is established, and the main structure has the characteristics of simple structure, flexibility and controllability. Without the need for complex calculation simulation, the main structure can respond to control instructions in real time to complete the driving operation of the virtual object, so that the virtual object presents an efficient and accurate driving response effect. At the same time, at the two-dimensional space level, a secondary structure as a plane figure is established, and the secondary structure is bound to the surface of the main structure, thereby obtaining a control linkage effect between the secondary structure and the main structure, which improves the visual authenticity of the virtual object in the surface presentation details on the one hand, and on the other hand, does not require complex driving operations to obtain a dynamic surface view presentation that follows the main structure. It can be seen that compared with a single-level model structure, the embodiment of the present application uses a three-dimensional model expression method combining a main structure with a secondary structure, which can simultaneously improve the driving flexibility and visual authenticity of the virtual object.

[0065] The embodiment of the present application achieves efficient construction and management of complex three-dimensional objects by decomposing the three-dimensional model into a main structure and a secondary structure. The main structure, as a grid framework in three-dimensional space, provides a stable geometric support for the model, ensuring the overall shape and spatial positioning of the model; while the secondary structure covers the surface of the main structure, enriching the details and appearance of the model in the form of a plane graphic. This layered design method not only optimizes the generation and editing process of the model, improves the operability and flexibility of the model, but also enables the model to quickly adapt and adjust in different application scenarios, thereby improving the practicality and visual effect of the model. At the same time, this structured method also facilitates the systematic management and maintenance of the model, bringing innovation and convenience to the field of three-dimensional modeling.

[0066] In one embodiment of the present application, the main structure of the three-dimensional model may include a mesh composed of multiple points, lines, and surfaces (usually triangles), which defines the shape and surface details of the three-dimensional model. In computer graphics, Mesh is a data structure used to build models for various irregular objects. The surface of objects in the real world is simulated in a computer by countless small polygonal patches, and a collection of these small patches is called a Mesh. Mesh can be composed of triangles or other planar shapes such as quadrilaterals, pentagons, etc. Since planar polygons can actually be further subdivided into triangles, it is also general to use a triangle mesh (Triangle Mesh) composed entirely of triangles to represent the surface of an object.

[0067] In a 3D model, each polygon is called a face, and the faces are connected by shared vertices to form a mesh. This model has high flexibility and fine detail expression capabilities, and is often used in 3D design and simulation in various fields. For example, in game development, developers can use Mesh 3D models to create realistic scenes and characters to enhance the visual effects of the game; in the field of medical imaging, doctors can use Mesh 3D models to simulate surgery and improve the success rate of surgery. The advantage of the Mesh structure is that it can more finely express the details and surfaces of an object, making the model more realistic and real. In addition, since the Mesh model is composed of many small faces, the shape and structure of the model can be adjusted and modified more flexibly to meet different needs. Moreover, the file size of the Mesh model is relatively small, which is convenient for storage and transmission, and improves work efficiency.

[0068] Mesh specifies the shape of a three-dimensional model, which encapsulates data such as vertex coordinates, faces, and face normals of the three-dimensional model. Once the Mesh grid is defined, the basic shape of the object in three-dimensional space is defined. The main structure based on Mesh is composed of triangular planes, and the data of these triangular planes and vertices are recorded in the Mesh grid. For example, each face of a cube is composed of 2 triangles, and the entire cube is composed of 12 triangles; while a sphere is composed of planes spliced ​​together by many triangles, and the inside is hollow. The surface of the sphere is not smooth, but composed of planes. The more triangular planes used to make up the sphere, the smoother it looks and the more detailed the display, but the corresponding GPU consumption is also getting higher and higher.

[0069] In one embodiment of the present application, the secondary structure of the three-dimensional model may include spatial points bound to the surface of the main structure, for example, 3D-GS points bound to the triangular plane of the Mesh grid. 3D-GS points refer to data points represented by placing Gaussian functions in three-dimensional space based on 3D Gaussian Splatting (3D-GS) technology, and the center of each Gaussian function represents a data point, and its shape and size are defined by the covariance matrix. By combining multiple Gaussian functions, complex three-dimensional structures can be constructed. 3D-GS is a technology for real-time rendering and high-quality scene reconstruction, which is modeled by converting point cloud data in the scene into the form of Gaussian functions, thereby achieving fast and accurate rendering and reconstruction. 3D-GS technology can be used to accelerate volume rendering, point cloud processing and rendering of three-dimensional scenes, because it can effectively reduce the amount of calculation while maintaining high quality. 3D-GS technology achieves accurate modeling of scenes by optimizing the properties of Gaussian points such as point coordinates, covariance matrix, opacity, and spherical harmonics. During rendering, 3D-GS projects Gaussian points onto a pixel-based image plane, a process called "splatting". 3DGS then sorts these Gaussian points and calculates the value of each pixel to achieve real-time rendering.

[0070] Figure 3 A schematic diagram showing a three-dimensional model formed by combining Mesh grids and 3D-GS points in an application scenario according to an embodiment of the present application.

[0071] like Figure 3 As shown, the application scenario takes a three-dimensional model of a piece of clothing as an example, the main structure 301 of the three-dimensional model is a Mesh grid frame, and the secondary structure 302 includes 3D-GS points bound to each triangular face of the Mesh grid frame. By binding the 3D-GS points to each triangular face of the Mesh grid, the rendering results of the clothing under different viewing angles can be obtained through 3D-GS.

[0072] For example, according to the mesh structure of the specified clothes, an independent local coordinate system can be generated on each triangular face of the mesh structure. This coordinate system takes the centroid of the triangle as the origin, a certain side of the triangle and its perpendicular direction as the x-axis and y-axis, and the normal direction of the face as the z-axis. On each triangular face, 3D-GS points are attached to the triangle in a high-density and uniform manner. Through the transformation matrix from the local coordinate system to the global coordinate system, the local three-dimensional coordinates of all 3D-GS points on all triangular facets can be mapped to the global coordinate system, and then the rendered image of the clothes can be obtained through 3D-GS rendering.

[0073] Using Mesh as the main structure and combining it with the secondary structure composed of 3D-GS points, it not only has the realistic rendering effect of 3D Gaussian technology, but also has the drivable characteristics of the Mesh structure, which can further improve the controllable flexibility of the three-dimensional model.

[0074] In one embodiment of the present application, the secondary structure of the three-dimensional model may include a texture map bound to the surface of the main structure. Texture mapping is a technique used in three-dimensional computer graphics, which involves mapping a two-dimensional image (texture) to the surface of a three-dimensional model to enhance visual detail and realism. This technique can simulate the surface characteristics of the material, such as color, gloss, roughness, etc., without increasing the geometric complexity of the model.

[0075] For example, the embodiment of the present application can map a texture image to the surface of a main structure (such as a mesh structure) through texture mapping. In texture mapping, surface mapping points are usually represented by UV coordinates, which define the positions of surface mapping points on the texture image. UV mapping is the process of converting the surface coordinates of the model into texture image coordinates. When the texture image is not enough to cover the entire main structure, the texture image can be tiled or repeatedly covered on the surface of the main structure through texture wrapping. If the resolution of the model surface and the texture image does not match, the texture filtering technology can be used to determine how to interpolate texture pixels to match the model surface. Common texture filtering methods include nearest neighbor filtering, bilinear filtering, and trilinear filtering. In certain specified areas of the main structure, texture blending technology can be used to mix multiple textures together, such as alpha blending, mix mapping, or multi-texturing.

[0076] The texture mapping in the embodiments of the present application may include one or more of normal mapping, environment mapping and displacement mapping. Normal mapping simulates complex surface details by using a special texture image containing surface normal information without adding actual geometric details. Environment mapping stores the environmental information reflected by the model in a texture to simulate reflection and gloss effects. Displacement mapping uses a texture image to change the geometry of the model surface to simulate more complex surface details, such as water ripples or uneven surfaces.

[0077] The following is a detailed description of each step of the method for generating a virtual object in the embodiment of the present application in conjunction with a three-dimensional model including a main structure and a secondary structure.

[0078] Figure 4 FIG. 1 shows a flow chart of obtaining a three-dimensional model of an object body in one embodiment of the present application. Figure 4 As shown, based on the above embodiment, the method of obtaining the three-dimensional model of the object body in step S210 may further include the following steps S211 to S213.

[0079] S211: extracting the main structure of the object body from the preset object parameter model.

[0080] The object parameter model is a pre-established statistical model for modeling virtual objects, and may include, for example, a three-dimensional human body parameter model SMPL (Skinned Multi-Person Linear model).

[0081] SMPL can generate human meshes of different poses and shapes with a small number of parameters, and is widely used in computer vision, animation, virtual reality and other fields. The core idea of ​​the SMPL model is to decouple the shape and pose of the human body, and map the shape parameters and pose parameters to the 3D mesh through linear regression to generate a high-quality human body model. This model simplifies the complex human body modeling process, so that realistic human 3D models can be quickly generated in different application scenarios. The SMPL model can capture the changes in human body shape and pose by learning a large amount of 3D human body scan data. The SMPL model uses a set of pose parameters (usually the rotation angles of 24 joints, a total of 72 parameters) and a set of shape parameters (usually 10 principal component analysis (PCA) coefficients) to define a person's body shape and pose. These parameters can be used to drive the model to generate human models of different body shapes and poses.

[0082] The embodiment of the present application can extract shape parameters from the SMPL model, and then construct the main structure of the object body, such as a Mesh grid structure, based on the extracted shape parameters.

[0083] S212: Binding the secondary structure of the object body to the main structure surface of the object body.

[0084] The secondary structure of the object body may include, for example, 3D-GS points or texture maps. The binding method of the secondary structure on the main structure can be referred to Figure 3 The embodiments shown will not be described in detail here.

[0085] S213: According to the prompt information used to describe the object body, the main structure of the object body and the secondary structure of the object body are updated respectively.

[0086] In one embodiment of the present application, the model structure can be updated by rendering an image of the three-dimensional model. For example, the main structure of the object body can be updated according to the image obtained by rendering the main structure, and the secondary structure of the object body can be updated according to the image obtained by rendering the main structure and the secondary structure.

[0087] In one embodiment of the present application, a method for updating the main structure of an object body may include: rendering the main structure of the object body to obtain a first rendered image of the object body; adding noise data to the first rendered image of the object body to obtain a first noise image of the object body; denoising the first noise image of the object body according to prompt information used to describe the object body to obtain a first denoised image of the object body; obtaining a loss error between the first rendered image of the object body and the first denoised image of the object body, and updating the main structure of the object body according to the loss error.

[0088] The embodiment of the present application iteratively executes the denoising and denoising process of the rendered image, calculates the loss error between the rendered image and the denoised image, and uses the back-propagated loss error to update the main structure of the object body. Specifically, the shape parameters of the main structure can be updated so that the main structure continuously approaches the model display effect described by the prompt information.

[0089] In one embodiment of the present application, a method for updating the secondary structure of an object body may include: rendering the main structure and the secondary structure of the object body together to obtain a second rendered image of the object body; adding noise data to the second rendered image of the object body to obtain a second noise image of the object body; denoising the second noise image of the object body according to prompt information used to describe the object body to obtain a second denoised image of the object body; obtaining a loss error between the second rendered image of the object body and the second denoised image of the object body, and updating the secondary structure of the object body according to the loss error.

[0090] The embodiment of the present application iteratively executes the denoising and denoising process of the rendered image in a manner similar to the main structure, calculates the loss error between the rendered image and the denoised image, and uses the back-propagated loss error to update the secondary structure of the object body, so that the secondary structure continuously approaches the model display effect described by the prompt information.

[0091] Figure 5 A schematic diagram of a process for establishing a three-dimensional model of a human body in an application scenario according to an embodiment of the present application is shown.

[0092] like Figure 5 As shown, in this application scenario, the embodiment of the present application uses the SMPL model as the object parameter model, and derives the human body Mesh as the main structure according to the shape parameters. Then, the Mesh can be rendered using rasterization rendering to obtain a first rendered image of the human body. At the same time, 3D-GS points can be attached to each triangular face of the Mesh as a secondary structure, and then 3D-GS rendering can be used to obtain a second rendered image of the human body.

[0093] Rasterization rendering is a technique in computer graphics that generates images by converting a three-dimensional scene or model into two-dimensional pixel data. This process involves converting the vertices, edges, and faces of a three-dimensional object into pixels on the screen, which usually includes setting the viewing point, calculating projections, clipping, rasterization, and finally pixel shading. The result of rasterization rendering is a two-dimensional image with color and texture, which can realistically display the details and light and shadow effects of the three-dimensional scene.

[0094] By inputting the first rendered image, the second rendered image, and the prompt information used to describe the character's body into the diffusion model Stable Diffusion, the diffusion model is used to perform the denoising and denoising process on the rendered image, so as to calculate the loss error between the rendered image and the denoised image. The Stable Diffusion model is a generative deep learning model specifically used for image generation tasks. It is based on the concept of the diffusion process, which generates high-quality images by gradually injecting noise into the image and learning how to reverse this process. Stable Diffusion converts random noise into meaningful images through the steps of diffusion denoising, and follows specific inputs (such as text prompt information) during the generation process.

[0095] A loss error is calculated based on the first rendered image and the first denoised image obtained after denoising and denoising, and is used to update the model parameters of the main structure Mesh. Another loss error is calculated based on the second rendered image and the second denoised image obtained after denoising and denoising, and is used to update the model parameters of the secondary structure 3D-GS point, which may include, for example, position parameter u, size parameter s, color parameter c, transparency parameter o, rotation parameter r, and the like.

[0096] In step S220, according to the attachment position of the attachment on the object body, a local area model corresponding to the attachment position is intercepted from the three-dimensional model of the object body.

[0097] In one embodiment of the present application, a method for extracting a local area model corresponding to an attachment position from a three-dimensional model of an object body may include:

[0098] A local grid frame corresponding to the attachment position is identified on the main structure of the object body; a local plane figure bound to the local grid frame is intercepted on the secondary structure of the object body, and the binding relationship between the local plane figure and the local grid frame is released to obtain a local area model composed of the local plane figures.

[0099] Taking the three-dimensional model of a human body composed of Mesh grids and 3D-GS points as an example, the embodiment of the present application can identify the local Mesh grid corresponding to the attachment position on the three-dimensional model of the human body, and then intercept the 3D-GS points corresponding to the local Mesh grid on the secondary structure, and release the binding relationship between these 3D-GS points and the Mesh grid to obtain a local area model composed of local 3D-GS points.

[0100] In step S230, a three-dimensional model of the attachment is generated according to the local area model.

[0101] Figure 6A flowchart of generating a three-dimensional model of an attachment in one embodiment of the present application is shown. Figure 6 As shown, based on the above embodiment, the method of generating a three-dimensional model of an attachment according to a local area model in step S230 may include the following steps S231 to S233.

[0102] S231: Establish the main structure of the attachment according to the local area model.

[0103] In the embodiment of the present application, the local area model obtained by intercepting the three-dimensional model of the object body may include a secondary structure after being unbound from the main structure, and the secondary structure is used as the initial model of the attachment to establish the main structure of the attachment. For example, a multi-view depth map of clothing can be expressed by 3D-GS point rendering of a local area intercepted on the human body, and then the Mesh structure of the clothing can be reconstructed based on the multi-view depth map.

[0104] In one embodiment of the present application, before establishing the main structure of the attachment, the parameters of the local area model used to initialize the attachment model can be updated to make it conform to the prompt information used to describe the attachment. The updating process may include: rendering the three-dimensional model of the object body and the local area model together to obtain a rendered image of the virtual object; updating the model parameters of the local area model according to the prompt information used to describe the attachment and the rendered image of the virtual object.

[0105] The embodiment of the present application renders the three-dimensional model of the object body and the local area model together, and combines the prompt information used to describe the attachments to simulate the complete model display effect of the virtual object composed of the object body and the attachments. On the basis of fixing the model parameters of the object body, the model parameters of the local area model are iteratively updated to continuously optimize the matching display effect of the attachments on the object body.

[0106] In one embodiment of the present application, updating the model parameters of the local area model according to the prompt information for describing the attachment and the rendered image of the virtual object may further include: adding noise data to the rendered image of the virtual object to obtain a noise image of the virtual object; denoising the noise image of the virtual object according to the prompt information for describing the attachment to obtain a denoised image of the virtual object; obtaining a loss error between the rendered image of the virtual object and the denoised image of the virtual object, and updating the model parameters of the local area model according to the loss error.

[0107] The embodiment of the present application iteratively executes the denoising and denoising process of the rendered image, calculates the loss error between the rendered image and the denoised image, and uses the back-propagated loss error to update the model parameters of the local area model, so that the local area model continuously approaches the model display effect of the attachment described by the prompt information.

[0108] In one embodiment of the present application, updating the model parameters of the local area model based on the prompt information used to describe the attachment and the rendered image of the virtual object may further include: performing image segmentation processing on the rendered image of the virtual object based on the prompt information used to describe the attachment to obtain a contour mask of the attachment; obtaining the loss error between the contour mask of the attachment and the contour area of ​​the local area model, and updating the model parameters of the local area model based on the loss error.

[0109] The embodiment of the present application iteratively executes the combination and segmentation process of the attachment and the object body, calculates the loss error between the contour mask of the segmentation result and the original contour area, and uses the back-propagation loss error to update the model parameters of the local area, so that the local area model continues to approach the model display effect of the attachment described by the prompt information.

[0110] In one embodiment of the present application, the above two local area model model parameter updating methods may be executed separately or simultaneously.

[0111] Figure 7 An application scenario of a method for updating model parameters of two local area models is shown in one embodiment of the present application.

[0112] like Figure 7 As shown in the figure, firstly, the 3D-GS points of part of the human body area are intercepted from the human body as the initialization model of the 3D-GS points of the clothes. At this stage, the 3D-GS points of the clothes are not bound to the triangle face of a certain mesh, so the 3D-GS points of the clothes can move freely in space, which is conducive to forming the shapes of different styles of clothes.

[0113] By rendering the 3D-GS points of the human body and clothes together, a rendered image of a human body wearing a certain piece of clothing can be obtained.

[0114] The rendered image is input into Stable Diffusion, and combined with the input prompt information for describing the clothes, the denoising loss error between the noise image and the denoised image can be calculated, and then the denoising loss error is used to back propagate and update the model parameters of the 3D-GS points of the clothes. In this process, the model parameters of the 3D-GS points of the human body remain unchanged. Based on the denoising loss error, the model parameters of the 3D-GS points of the clothes are iteratively updated (for example, they may include position parameters, size parameters, transparency parameters, rotation parameters, and color parameters), so that the rendering effect of the 3D-GS points of the clothes can be continuously close to the appearance and shape of the clothes described by the prompt information. The prompt information may include text prompt information, for example, the prompt text information corresponding to the image rendering effect shown in the figure may be "A beautiful caucasian woman wearing a pair of lightblue jeans".

[0115] At the same time, the rendered image of a human body wearing a certain piece of clothing is input into the semantic segmentation model (SegmentAnything Model, SAM), and the contour mask of the clothing in the rendered image is extracted using the SAM model. Then, the segmentation loss error between the contour mask and the contour area rendered by the clothing 3D-GS points is calculated, and the model parameters of the clothing 3D-GS points (for example, position parameters, size parameters, transparency parameters, and rotation parameters) are updated using the segmentation loss error back propagation. The parameter update based on the segmentation loss error can suppress the noise points in the clothing 3D-GS points that do not belong to the clothing and improve the distribution accuracy of the clothing 3D-GS points.

[0116] S232: Binding the secondary structure of the attachment to the surface of the main structure of the attachment.

[0117] Still taking the modeling process of clothes as an example, after obtaining the 3D-GS points of the clothes that meet the text description, the multi-perspective depth map of the clothes can be rendered through these 3D-GS points, and then the Mesh mesh of the clothes can be reconstructed according to the multi-perspective depth map as the main structure of the clothes.

[0118] After obtaining the Mesh grid of the clothes, new 3D-GS points can be attached to the Mesh grid of the clothes as the secondary structure of the clothes.

[0119] S233: Fixing the model parameters of the main structure of the attachment, and updating the model parameters of the secondary structure of the attachment, to obtain a three-dimensional model of the attachment.

[0120] In one embodiment of the present application, a method for updating model parameters of a secondary structure of an attachment may include: rendering the main structure of the attachment and the secondary structure of the attachment together to obtain a rendered image of the attachment; adding noise data to the rendered image of the attachment to obtain a noise image of the attachment; denoising the noise image of the attachment based on prompt information used to describe the attachment to obtain a denoised image of the attachment; obtaining a loss error between the rendered image of the attachment and the denoised image of the attachment, and updating the model parameters of the secondary structure of the attachment based on the loss error.

[0121] Figure 8 An application scenario of updating the secondary structure model parameters of an attachment in one embodiment of the present application is shown.

[0122] like Figure 8 As shown, after the rendered image of the clothes is obtained by 3D-GS rendering, it is input into the Stable Diffusion model together with the prompt information describing the clothes, and the denoising loss error is calculated, so as to update the model parameters of the 3D-GS points attached to the clothes Mesh according to the denoising loss error. Among them, the prompt information describing the clothes can be text prompt information, for example, the prompt text information corresponding to the image rendering effect shown in the figure can be "A pair of light blue jeans".

[0123] In one embodiment of the present application, after generating a three-dimensional model of an attachment based on a local area model, texture enhancement may be performed on the three-dimensional model of the object body and / or the attachment. The specific texture enhancement method may include: rendering the three-dimensional model of the object body and / or the attachment to obtain a rendered image of the three-dimensional model; adding noise data to the rendered image of the three-dimensional model to obtain a noisy image of the three-dimensional model; denoising the noise image of the three-dimensional model based on prompt information used to describe the three-dimensional model to obtain a denoised image of the three-dimensional model; obtaining a loss error between the rendered image of the three-dimensional model and the denoised image of the three-dimensional model, and updating the three-dimensional model based on the loss error.

[0124] The embodiment of the present application utilizes the denoising and denoising process of the rendered image to calculate the denoising loss error, update the three-dimensional model of the object body and / or attachments, and obtain a virtual object three-dimensional model with higher quality and higher definition texture.

[0125] For example, the embodiment of the present application can render a multi-view image of a human body and / or clothing expressed by a mixture of Mesh grids and 3D-GS points, add a certain proportion (e.g., 25%) of Gaussian noise to the rendered image, and then input it into a Stable Diffusion model with text prompt information describing the human body and / or clothing for denoising. After obtaining the denoised image, the denoised image is used as a label to update the model parameters of the 3D-GS points on the human body and / or clothing through the pixel loss of the 3D-GS rendered image and the denoised image.

[0126] Fig. 9 The virtual object generation process performed in an application scenario according to an embodiment of the present application is shown. Fig. 9 As shown, taking a person object as an example, the generation process of the person object may include a body generation stage, a clothing generation stage and a texture enhancement stage.

[0127] In the first stage, the human body generation stage, a human body wearing only underwear is obtained by inputting English text (e.g., "Abeautifulcaucasian woman wearing white one-piece swimsuit."). In the second stage, the clothing generation stage, English text (e.g., "A beautiful caucasian woman wearing a pair of light blue jeans.") is input to obtain clothes for the digital human. In the third stage, the texture enhancement stage, the texture quality of the generated human body and clothing is further improved to make the texture clearer and more realistic.

[0128] Fig.10 The embodiment of the present application shows that in an application scenario, multiple three-dimensional models of attachments are generated to achieve an image display effect of a human body changing clothes. By decoupling the human body and clothes, the same digital human can be changed into different clothes.

[0129] like Fig.10 As shown, for the same digital human 3D model, clothes such as jeans, denim skirt and white lace skirt were replaced respectively.

[0130] Fig.11 The embodiment of the present application shows the image display effect of a person wearing different clothing forms in an application scenario. Since the embodiment of the present application adopts a hybrid representation of Mesh grid + 3D-GS points for the three-dimensional model, the deformation of the clothes when the human body moves can be calculated through the cloth simulation of the clothes, which greatly improves the authenticity of the rendered video when the human body moves.

[0131] like Fig.11As shown in the figure, the rendering effect of the shape of clothes when a person wears a skirt at different times and in different movements is shown.

[0132] Based on the introduction of the above application scenarios, it can be known that the embodiment of the present application realizes a drivable digital human generation technology that decouples the human body and clothes. Using multiple 3D models to represent the human body and each piece of clothing, during the generation process, a human body wearing only underwear can be generated first, and then the clothes worn by the digital human can be generated in sequence. By decoupling the human body and clothes during the generation process, it is possible to achieve seamless changes in the generated digital human clothing while maintaining the same underlying human structure. Moreover, the embodiment of the present application proposes a three-dimensional model based on a hybrid expression of 3D-GS points and Mesh grids to express the human body and clothes, which helps to greatly improve the realism of human motion animation through cloth simulation of the collision between the human body and clothes in human body driving.

[0133] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0134] The following introduces an apparatus embodiment of the present application, which can be used to execute the method for generating a virtual object in the above-mentioned embodiment of the present application. Fig.12 The structure block diagram of the device for generating a virtual object provided in an embodiment of the present application is schematically shown. Fig.12 As shown, the virtual object includes an object body and attachments, and the virtual object generation device 1200 includes:

[0135] An acquisition module 1210 is configured to acquire a three-dimensional model of the object body;

[0136] The interception module 1220 is configured to intercept a local area model corresponding to the attachment position from the three-dimensional model of the object body according to the attachment position of the attachment on the object body;

[0137] The generating module 1230 is configured to generate a three-dimensional model of the attachment according to the local area model.

[0138] In some embodiments of the present application, based on the above technical solution, the three-dimensional model includes a main structure and a secondary structure bound to the surface of the main structure, the main structure is used to represent a grid frame in three-dimensional space, and the secondary structure is used to represent a plane figure covered on the surface of the grid frame.

[0139] In some embodiments of the present application, based on the above technical solution, the interception module 1220 may further include:

[0140] an identification module configured to identify a local grid frame corresponding to the attachment position on the main structure of the object body;

[0141] The releasing module is configured to intercept the local plane graphics bound to the local grid framework on the secondary structure of the object body, and release the binding relationship between the local plane graphics and the local grid framework to obtain a local area model composed of the local plane graphics.

[0142] In some embodiments of the present application, based on the above technical solution, the generating module 1230 may further include:

[0143] A main structure establishing module, configured to establish the main structure of the attachment according to the local area model;

[0144] A secondary structure binding module is configured to bind the secondary structure of the attachment on the surface of the main structure of the attachment;

[0145] The secondary structure updating module is configured to fix the model parameters of the main structure of the attachment and update the model parameters of the secondary structure of the attachment to obtain a three-dimensional model of the attachment.

[0146] In some embodiments of the present application, based on the above technical solution, the secondary structure update module may further include:

[0147] A rendering module is configured to render the main structure of the attachment and the secondary structure of the attachment together to obtain a rendered image of the attachment;

[0148] A noise adding module is configured to add noise data to the rendered image of the attachment to obtain a noise image of the attachment;

[0149] a denoising module configured to perform denoising on the noise image of the attachment according to the prompt information used to describe the attachment, so as to obtain a denoised image of the attachment;

[0150] The updating module is configured to obtain a loss error between a rendered image of the attachment and a denoised image of the attachment, and update a model parameter of a secondary structure of the attachment according to the loss error.

[0151] In some embodiments of the present application, based on the above technical solution, the generating module 1230 may further include:

[0152] A local model rendering module is configured to render the three-dimensional model of the object body and the local area model together to obtain a rendered image of the virtual object;

[0153] The local model updating module is configured to update the model parameters of the local area model according to the prompt information used to describe the attachment and the rendered image of the virtual object.

[0154] In some embodiments of the present application, based on the above technical solution, the local model update module can be further configured to: add noise data to the rendered image of the virtual object to obtain a noise image of the virtual object; denoise the noise image of the virtual object according to the prompt information used to describe the attachment to obtain a denoised image of the virtual object; obtain the loss error between the rendered image of the virtual object and the denoised image of the virtual object, and update the model parameters of the local area model according to the loss error.

[0155] In some embodiments of the present application, based on the above technical solution, the local model update module can be further configured to: perform image segmentation processing on the rendered image of the virtual object according to the prompt information used to describe the attachment, and obtain the contour mask of the attachment; obtain the loss error between the contour mask of the attachment and the contour area of ​​the local area model, and update the model parameters of the local area model according to the loss error.

[0156] In some embodiments of the present application, based on the above technical solution, the acquisition module 1210 may further include:

[0157] A main structure extraction module is configured to extract the main structure of the object body from a preset object parameter model;

[0158] A secondary structure binding module is configured to bind the secondary structure of the object body on the main structure surface of the object body;

[0159] The structure updating module is configured to update the main structure of the object body and the secondary structure of the object body respectively according to the prompt information used to describe the object body.

[0160] In some embodiments of the present application, based on the above technical solution, the main structure update part in the structure update module can be configured as follows: rendering the main structure of the object body to obtain a first rendered image of the object body; adding noise data to the first rendered image of the object body to obtain a first noise image of the object body; denoising the first noise image of the object body according to the prompt information used to describe the object body to obtain a first denoised image of the object body; obtaining a loss error between the first rendered image of the object body and the first denoised image of the object body, and updating the main structure of the object body according to the loss error.

[0161] In some embodiments of the present application, based on the above technical solution, the secondary structure update part in the structure update module can be configured to: render the main structure and the secondary structure of the object body together to obtain a second rendered image of the object body; add noise data to the second rendered image of the object body to obtain a second noise image of the object body; denoise the second noise image of the object body according to the prompt information used to describe the object body to obtain a second denoised image of the object body; obtain the loss error between the second rendered image of the object body and the second denoised image of the object body, and update the secondary structure of the object body according to the loss error.

[0162] In some embodiments of the present application, based on the above technical solution, the device for generating the virtual object may further include a texture enhancement module, and the texture enhancement module is configured to: render the three-dimensional model of the object body and / or the attachment to obtain a rendered image of the three-dimensional model; add noise data to the rendered image of the three-dimensional model to obtain a noise image of the three-dimensional model; denoise the noise image of the three-dimensional model according to the prompt information used to describe the three-dimensional model to obtain a denoised image of the three-dimensional model; obtain a loss error between the rendered image of the three-dimensional model and the denoised image of the three-dimensional model, and update the three-dimensional model according to the loss error.

[0163] The specific details of the virtual object generation device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0164] Fig.13 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0165] It should be noted that Fig.13 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0166] like Fig.13 As shown, the computer system 1300 includes a central processing unit 1301 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1302 (ROM) or the program loaded from the storage part 1308 to the random access memory 1303 (RAM). Various programs and data required for system operation are also stored in the random access memory 1303. The central processing unit 1301, the read-only memory 1302 and the random access memory 1303 are connected to each other through a bus 1304. The input / output interface 1305 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1304.

[0167] The following components are connected to the input / output interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the input / output interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0168] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the central processor 1301, various functions defined in the system of the present application are executed.

[0169] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0170] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0171] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0172] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0173] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0174] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating a virtual object, characterized in that: The virtual object includes an object body and attachments, and the generation method includes: Acquire a three-dimensional model of the object body; According to the attachment position of the attachment on the object body, intercepting a local area model corresponding to the attachment position from the three-dimensional model of the object body; A three-dimensional model of the attachment is generated according to the local area model.

2. The method for generating a virtual object according to claim 1, characterized in that: The three-dimensional model includes a main structure and a secondary structure bound to the surface of the main structure, the main structure is used to represent a grid frame in a three-dimensional space, and the secondary structure is used to represent a plane figure covered on the surface of the grid frame.

3. The method for generating a virtual object according to claim 2, characterized in that: Extracting a local area model corresponding to the attachment position from the three-dimensional model of the object body includes: identifying a local grid frame corresponding to the attachment location on the main structure of the object body; A local plane figure bound to the local grid framework is intercepted on the secondary structure of the object body, and the binding relationship between the local plane figure and the local grid framework is released to obtain a local area model composed of the local plane figure.

4. The method for generating a virtual object according to claim 3, characterized in that: Generating a three-dimensional model of the attachment according to the local area model includes: Establishing the main structure of the attachment according to the local area model; Binding the secondary structure of the attachment to the surface of the main structure of the attachment; The model parameters of the main structure of the attachment are fixed, and the model parameters of the secondary structure of the attachment are updated to obtain a three-dimensional model of the attachment.

5. The method for generating a virtual object according to claim 4, characterized in that: Updating model parameters of the secondary structure of the attachment, including: Rendering the main structure of the attachment and the secondary structure of the attachment together to obtain a rendered image of the attachment; Adding noise data to the rendered image of the attachment to obtain a noise image of the attachment; De-noising the noise image of the attachment according to the prompt information used to describe the attachment to obtain a de-noised image of the attachment; A loss error between a rendered image of the attachment and a denoised image of the attachment is obtained, and a model parameter of a secondary structure of the attachment is updated according to the loss error.

6. The method for generating a virtual object according to claim 4, characterized in that: Before establishing the main structure of the attachment according to the local area model, the method further includes: Rendering the three-dimensional model of the object body and the local area model together to obtain a rendered image of the virtual object; The model parameters of the local area model are updated according to the prompt information for describing the attachment and the rendered image of the virtual object.

7. The method for generating a virtual object according to claim 6, characterized in that: Updating the model parameters of the local area model according to the prompt information for describing the attachment and the rendered image of the virtual object includes: Adding noise data to the rendered image of the virtual object to obtain a noise image of the virtual object; De-noising the noise image of the virtual object according to the prompt information used to describe the attachment to obtain a de-noised image of the virtual object; A loss error between a rendered image of the virtual object and a denoised image of the virtual object is obtained, and a model parameter of the local area model is updated according to the loss error.

8. The method for generating a virtual object according to claim 6, characterized in that: Updating the model parameters of the local area model according to the prompt information for describing the attachment and the rendered image of the virtual object includes: According to the prompt information used to describe the attachment, performing image segmentation processing on the rendered image of the virtual object to obtain a contour mask of the attachment; A loss error between the contour mask of the attachment and the contour area of ​​the local area model is obtained, and a model parameter of the local area model is updated according to the loss error.

9. The method for generating a virtual object according to claim 2, characterized in that: Acquiring a three-dimensional model of the object body includes: Extracting the main structure of the object body from a preset object parameter model; Binding the secondary structure of the object body to the surface of the main structure of the object body; According to the prompt information used to describe the object body, the main structure of the object body and the secondary structure of the object body are updated respectively.

10. The method for generating a virtual object according to claim 9, characterized in that: According to the prompt information used to describe the object body, the main structure of the object body is updated, including: Rendering the main structure of the object body to obtain a first rendered image of the object body; Adding noise data to the first rendered image of the object body to obtain a first noisy image of the object body; De-noising the first noise image of the object body according to the prompt information used to describe the object body, so as to obtain a first de-noised image of the object body; A loss error between a first rendered image of the object body and a first denoised image of the object body is obtained, and a main structure of the object body is updated according to the loss error.

11. The method for generating a virtual object according to claim 9, characterized in that: Updating the secondary structure of the object body according to the prompt information used to describe the object body includes: Rendering the main structure and the secondary structure of the object body together to obtain a second rendered image of the object body; adding noise data to the second rendered image of the object body to obtain a second noisy image of the object body; De-noising the second noise image of the object body according to the prompt information used to describe the object body to obtain a second de-noised image of the object body; A loss error between a second rendered image of the object body and a second denoised image of the object body is obtained, and a secondary structure of the object body is updated according to the loss error.

12. The method for generating a virtual object according to any one of claims 1 to 11, characterized in that: After generating the three-dimensional model of the attachment according to the local area model, the method further includes: Rendering the three-dimensional model of the object body and / or the attachment to obtain a rendered image of the three-dimensional model; Adding noise data to the rendered image of the three-dimensional model to obtain a noise image of the three-dimensional model; De-noising the noise image of the three-dimensional model according to the prompt information used to describe the three-dimensional model to obtain a de-noised image of the three-dimensional model; A loss error between a rendered image of the three-dimensional model and a denoised image of the three-dimensional model is obtained, and the three-dimensional model is updated according to the loss error.

13. A device for generating a virtual object, characterized in that: The virtual object includes an object body and attachments, and the generating device includes: An acquisition module, configured to acquire a three-dimensional model of the object body; A capture module is configured to capture a local area model corresponding to the attachment position from the three-dimensional model of the object body according to the attachment position of the attachment on the object body; A generating module is configured to generate a three-dimensional model of the attachment according to the local area model.

14. A computer readable medium, characterized in that The computer readable medium stores a computer program, and when the computer program is executed by a processor, the method for generating a virtual object according to any one of claims 1 to 12 is implemented.

15. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the executable instructions to implement the method for generating a virtual object as described in any one of claims 1 to 12.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for generating a virtual object according to any one of claims 1 to 12 is implemented.

Images