Virtual image generation method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a virtual image generation method and device, electronic equipment, a computer readable storage medium and a computer program product. The embodiment of the invention can be applied to virtual image generation scenes of games, virtual reality, animation production and the like; the method comprises the steps of performing reconstruction based on a two-dimensional image of a target object to obtain a first source grid model of the target object under a source model topology; reading dense vertex parameters used for describing the relation between the surface patch vertex of the template grid model and the grid surface patch of the first source grid model under the business model topology; based on the dense vertex parameter and the vertex information of the patch vertex of the first source grid model, updating the vertex information of the patch vertex of the template grid model to obtain an updated grid model under the service model topology; and determining a rendering result of the updated grid model as a three-dimensional virtual image of the target object. According to the method and the device, the generation efficiency of the virtual image can be improved while the generation effect of the virtual image is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a virtual image generation method and device, electronic equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] Virtual image generation for a target object refers to a process of generating a three-dimensional mesh model with appearance features of the target object based on a two-dimensional image of the target object. In order to ensure that the mesh model reconstructed based on the two-dimensional image can be applied to a business pipeline, the model topology of the reconstructed mesh model usually needs to be converted to the model topology used by the business pipeline. However, based on the model conversion method in the related art, the generation efficiency of the virtual image is low and the generation effect of the virtual image is poor. SUMMARY

[0003] The embodiments of the present application provide a virtual image generation method, device, electronic equipment, computer readable storage medium and computer program product, which can improve the generation efficiency of the virtual image while ensuring the generation effect of the virtual image.

[0004] The technical solutions of the embodiments of the present application are as follows:

[0005] The embodiments of the present application provide a virtual image generation method, which comprises the following steps:

[0006] reconstructing a first source mesh model of the target object under a source model topology based on a two-dimensional image of the target object;

[0007] reading a dense vertex parameter used to describe the connection between a facet vertex of a template mesh model under a business model topology and a mesh facet of the first source mesh model;

[0008] updating the vertex information of the facet vertex of the template mesh model based on the dense vertex parameter and the vertex information of the facet vertex of the first source mesh model to obtain an updated mesh model under the business model topology;

[0009] determining a rendering result of the updated mesh model as a three-dimensional virtual image of the target object.

[0010] The embodiments of the present application provide a virtual image generation device, which comprises the following modules:

[0011] a model reconstruction module configured to reconstruct a first source mesh model of the target object under a source model topology based on a two-dimensional image of the target object;

[0012] a parameter reading module, configured to read dense vertex parameters for describing a connection between a face vertex of a template mesh model under a business model topology and a mesh face of the first source mesh model;

[0013] a model updating module, configured to update vertex information of the face vertex of the template mesh model based on the dense vertex parameters and the vertex information of the face vertex of the first source mesh model, to obtain an updated mesh model under the business model topology;

[0014] a model rendering module, configured to determine a rendering result of the updated mesh model as the three-dimensional virtual image of the target object.

[0015] In some embodiments of the present application, the dense vertex parameters include target indexes and barycentric parameters, and the vertex information includes vertex coordinates; the target indexes are vertex indexes of mesh faces in the first source mesh model corresponding to the face vertex of the template mesh model.

[0016] In some embodiments of the present application, the model updating module is further configured to determine a target vertex from the face vertex of the first source mesh model according to the target indexes, and determine vertex coordinates of the target vertex as target coordinates; calculate dense vertex coordinates by using the target coordinates and the barycentric parameters, and update the vertex coordinates of the face vertex of the template mesh model by using the dense vertex coordinates, to obtain the updated mesh model under the business model topology.

[0017] In some embodiments of the present application, the parameter reading module is further configured to read the dense vertex parameters for describing the connection between the face vertex of the template mesh model and the mesh face of the first source mesh model from vertex indexes of the face vertex of the template mesh model.

[0018] In some embodiments of the present application, the virtual image generation apparatus further includes a parameter generation module;

[0019] The parameter generation module is configured to reconstruct a second source mesh model of a template object under the source model topology based on a two-dimensional image of the template object, align the second source mesh model and the template mesh model to obtain an aligned mesh model of the template object under the business model topology, determine target indexes and barycentric parameters based on the aligned mesh model and the second source mesh model, and generate the dense vertex parameters by using the target indexes and the barycentric parameters.

[0020] In some embodiments of the present application, the parameter generation module is further configured to: perform vertex densification on the second source mesh model to obtain a dense mesh model; determine a matching vertex for a face vertex of the aligned mesh model from face vertices of the dense mesh model, wherein the face vertices of the dense mesh model are determined from inside of mesh faces of the second source mesh model, and vertex coordinates of the face vertices of the dense mesh model are calculated from vertex coordinates of the face vertices of the second source mesh model and corresponding candidate parameters; determine a vertex index of a mesh face of the second source mesh model corresponding to the matching vertex as the target index, and determine the candidate parameters used to calculate the vertex coordinates of the matching vertex as the barycentric parameters.

[0021] In some embodiments of the present application, the parameter generation module is further configured to: perform the following processing through iteration i, where i is a positive integer: perform i-th mesh subdivision on the second source mesh model based on a subdivision level of the i-th iteration to obtain an i-th subdivided mesh model; determine a nearest candidate vertex for each face vertex of the aligned mesh model from subdivided vertices of the i-th subdivided mesh model; calculate a distance mean using distances between each face vertex of the aligned mesh model and a corresponding candidate vertex; and determine the i-th subdivided mesh model as the dense mesh model when the distance mean is less than or equal to a distance threshold.

[0022] In some embodiments of the present application, the parameter generation module is further configured to: generate N candidate parameters for mesh faces of the second source mesh model according to the subdivision level of the i-th iteration, where N is a positive integer; determine N subdivided vertices for each mesh face of the second source mesh model using vertex coordinates of each mesh face of the second source mesh model and the N candidate parameters; generate a subdivided face for each mesh face using the N subdivided vertices, and construct the i-th subdivided mesh model using the subdivided faces of each mesh face.

[0023] In some embodiments of the present application, the parameter generation module is further configured to: increase the subdivision level of the i-th iteration to obtain a subdivision level of an i+1-th iteration when the distance mean is greater than the distance threshold.

[0024] In some embodiments of the present application, the parameter generation module is further configured to: determine a projection vertex for a face vertex of the aligned mesh model from the second source mesh model; determine a vertex index of a mesh face of the second source mesh model to which the projection vertex belongs as the target index; and determine the barycentric parameters using vertex coordinates of the projection vertex and vertex coordinates of a face vertex of the mesh face to which the projection vertex belongs.

[0025] In some embodiments of the present application, the parameter generation module is further configured to obtain a first coordinate from a first display area of a model display interface and a second coordinate from a second display area of the model display interface, wherein the first display area and the second display area are respectively configured to display the second source mesh model and the template mesh model; form a vertex pair by using a vertex corresponding to the first coordinate and a vertex corresponding to the second coordinate; and align the second source mesh model and the template mesh model by using the vertex pair to obtain the aligned mesh model of the template object under the business model topology.

[0026] An electronic device is provided in an embodiment of the present application, and the electronic device comprises:

[0027] A memory is configured to store computer executable instructions or computer programs.

[0028] A processor is configured to execute the computer executable instructions or computer programs stored in the memory to implement the virtual image generation method provided in the embodiments of the present application.

[0029] A computer readable storage medium is provided in an embodiment of the present application, and the computer readable storage medium stores computer programs or computer executable instructions, which are configured to be executed by a processor to implement the virtual image generation method provided in the embodiments of the present application.

[0030] A computer program product is provided in an embodiment of the present application, and the computer program product comprises computer programs or computer executable instructions, which are configured to be executed by a processor to implement the virtual image generation method provided in the embodiments of the present application.

[0031] The embodiments of the present application have the following beneficial effects: the electronic device first uses a two-dimensional image to perform mesh model reconstruction on a target object under a source model topology to obtain a first source mesh model, and simultaneously determines more appropriate vertex information for the vertexes of a template mesh model from the first source mesh model in combination with the connection between the vertexes of the template mesh face provided by the dense vertex parameters and the mesh face of the first source mesh model to update the vertexes of the template mesh model, thereby not only automatically transferring the appearance features of the target object to the template mesh model to improve the generation efficiency of the virtual image, but also determining appropriate mesh faces for the vertexes of the template mesh model in combination with the dense vertex parameters, updating the vertex information based on the vertex information of all vertexes of the mesh face, thereby more completely retaining the appearance features of the target object to obtain a three-dimensional image model that is more consistent with the appearance of the target object, and thus improving the generation effect of the virtual image. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a schematic diagram of an architecture of a virtual image generation system provided by an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of a structure of a server in Figure 1 ;

[0034] Figure 3 is a schematic diagram of a flow of a virtual image generation method provided by an embodiment of the present application; Figure 1 ;

[0035] Figure 4 is a schematic diagram of a flow of a virtual image generation method provided by an embodiment of the present application; Figure 2 ;

[0036] Figure 5 is a schematic diagram of a flow of a virtual image generation method provided by an embodiment of the present application; Figure 3 ;

[0037] Figure 6 is a schematic diagram of a flow of a virtual image generation method provided by an embodiment of the present application; Figure 4 ;

[0038] Figure 7 is a schematic diagram of a comparison of a 3DMM model topology and a business model topology provided by an embodiment of the present application;

[0039] Figure 8 is a schematic diagram of a flow of a model topology conversion provided by an embodiment of the present application;

[0040] Figure 9 is a schematic diagram of a topology alignment of two mesh models provided by an embodiment of the present application;

[0041] Figure 10 is a schematic diagram of a result of a model topology alignment provided by an embodiment of the present application;

[0042] Figure 11 is a schematic diagram of a local region of a 3DMM mesh model of densified vertices provided by an embodiment of the present application;

[0043] Figure 12 is a schematic diagram of a 3D human head model provided by an embodiment of the present application;

[0044] Figure 13 is a schematic diagram of a mesh model after a topology conversion is completed provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purposes, technical solutions and advantages of the present application clearer, the following further describes the present application in conjunction with the accompanying drawings, the described embodiments should not be regarded as limiting the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0046] In the following description, "some embodiments" are referred to, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0047] In the following description, the term "first\second" is only to distinguish similar objects, and does not represent a specific order of the object, and it can be understood that "first\second" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0048] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0049] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0050] The relevant data collection process in the embodiments of the present application should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and within the scope of authorization of laws and regulations and the personal information subject, carry out subsequent data use and processing.

[0051] Before the embodiments of the present application are further described in detail, the terms and terms involved in the embodiments of the present application are explained, and the terms and terms involved in the embodiments of the present application are applicable to the following explanations.

[0052] 1) Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0053] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-training model technology, operation / interaction system, mechatronics, etc. Among them, the pre-training model is also called large model, basic model, which can be widely applied to downstream tasks in various directions of artificial intelligence after fine tuning. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0054] 2) Computer Vision (CV) is a science that studies how to make computers "see". Further, it refers to using cameras and computers to replace human eyes to identify, detect and measure targets, and further process graphics so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish an artificial intelligence system that can obtain information from images or multidimensional data. Large model technology brings important changes to the development of computer vision technology. Swin-transformer, ViT, V-MOE, MAE and other pre-training models in the field of vision can be quickly and widely applied to downstream specific tasks after fine tuning. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0055] 3) 3D avatar, refers to a 3D model generated by a computer for a target object, which has the appearance characteristics of the target object. Here, the target object can be a human face, or other objects such as animals, etc. Thus, the 3D avatar can be a 3D model generated by a computer with the appearance characteristics of a human face, or a 3D model with the appearance characteristics of other objects.

[0056] 4) mesh model, a widely used representation method in computer graphics, used to construct and render 3D models. The surface of a 3D object can be represented by a mesh model, the position of each vertex of a mesh patch determines the shape of the object surface, and the edges and faces connecting these vertexes can completely describe the appearance of the entire 3D object.

[0057] 5) mesh patch, refers to the polygon that constitutes the mesh model, such as triangle, quadrilateral, etc. These polygons can be connected to each other in 3D space to form a continuous surface that can describe the appearance of a 3D object. It should be noted that a mesh patch is formed by connecting patch vertices.

[0058] 6) model topology, used to describe the connection of patch vertices in a mesh model, i.e. how patch vertices are connected to form a mesh patch. That is, for a mesh model, the connection of patch vertices is different, then the model topology is different.

[0059] For example, assume that 3 patch vertices (vertex 1, vertex 2 and vertex 3) are connected to form a triangular mesh patch, model topology 1 represents connecting vertex 1, vertex 2 and vertex 3 in order to form a mesh patch, and model topology 2 represents connecting vertex 1, vertex 3 and vertex 2 in order to form a mesh patch.

[0060] 7) vertex index, stores the index of the points that constitute the polygon of the mesh patch, for example, for a triangular mesh patch, the vertex index stores the information of the three vertices that constitute the triangle, and the connection of the three vertices. The vertex index is usually stored in an array, each element in the array corresponds to the attribute data of a vertex, such as the coordinates of the vertex, texture coordinates, etc. The order of the array defines the order of the vertices required to be connected by the mesh patch, through the vertex index, the attribute data of the vertices can be read and processed in the correct order during rendering, so as to accurately render the 3D model.

[0061] 8) barycentric parameter, refers to the parameter used to calculate the barycentric coordinates of a polygon. In more detail, the barycentric coordinates of a polygon usually need to be calculated in combination with the vertex coordinates of the polygon and the barycentric parameter.

[0062] 9) BlendShapes technology, which is mainly used to capture real facial expression data and convert these expression data into a series of deformable shapes to drive a 3D avatar model to mimic or reflect real facial expressions in real time.

[0063] 10) 3D Morhpable Model (3DMM), which is a technique for creating and synthesizing facial models by combining multiple known facial scan data with one or more template facial features to generate a model that can be used to synthesize new facial images. In 3DMM, a deformable 3D face mesh with a fixed number of mesh vertices can be weighted by many independent control features of the face.

[0064] 11) K-Nearest Neighbors Algorithm (KNN), which finds the K nearest points from the training set for a given sample point to predict the class of the new sample point by the labels of the K nearest points.

[0065] Virtual avatar generation for a target object refers to a process of generating a 3D mesh model with appearance features of the target object based on a 2D image of the target object. Virtual avatar generation can be applied to many fields, such as game and animation fields, so that a game avatar or an animation avatar with appearance features of the target object can be obtained.

[0066] Currently, there are some mature 3D modeling methods that can reconstruct corresponding mesh models based on 2D images. Taking a face as an example, 3DMM technology can be used to reconstruct a mesh model with facial features of a target object from a 2D image. However, the model topology used by 3DMM is often different from the model topology used by a specific business pipeline. When applying the mesh model generated based on 3DMM to a business scenario, the model topology of the mesh model needs to be converted to the model topology used by the business pipeline, that is, a mesh model under the model topology used by the business pipeline needs to be generated to ensure the application of the mesh model generated based on 3DMM in the business pipeline.

[0067] In the related art, there are generally three model topology alignment methods. The first method is manual alignment in software. For example, based on the face mesh model reconstructed by 3DMM and the template mesh model under the business pipeline, both are imported into Wrap3D software, and then corresponding points are manually selected and marked to align the model topology, so as to obtain the face mesh model under the model topology used by the business pipeline. The second method is to re-model based on the model topology used by the business pipeline. For example, using a new 3DMM model to reconstruct the face based on the model topology used by the business pipeline to obtain the face mesh model under the model topology used by the business pipeline. The third method is to convert the model topology through nearest neighbor search. For example, for the face patch vertices in the face mesh model reconstructed by 3DMM, the corresponding face patch vertices are found from the template mesh model in the business pipeline using the nearest neighbor algorithm, and then the vertex coordinates of the template mesh model under the business pipeline are updated according to the correspondence of the face patch vertices of the two mesh models.

[0068] However, the above methods have some defects. For example, the first method needs a lot of manual operation to obtain the face mesh model under the model topology used by the business pipeline, which not only makes the virtual image generation efficiency low, but also directly matches the vertices, which causes the loss of appearance features, making the precision of the three-dimensional virtual image low (for example, the face mesh model obtained may lose some facial features, resulting in a large difference between the three-dimensional virtual image obtained and the actual face); the second method needs to spend a lot of time to prepare data, resulting in low virtual image generation efficiency; and in the third method, since the vertex density of the mesh model reconstructed by 3DMM and the vertex density of the mesh model used by the business pipeline are quite different, it is difficult to find matching mesh vertices from the mesh vertices of the mesh model reconstructed by 3DMM through nearest neighbor search, that is, the face patch vertices of the two mesh models cannot be completely matched. At this time, directly replacing the vertex coordinates of the mesh model reconstructed by 3DMM with the mesh vertices of the mesh model used by the business pipeline will cause the loss of the appearance features of the target object, resulting in low model precision of the mesh model of the business pipeline, that is, poor virtual image generation effect.

[0069] In summary, based on the model topology conversion method in the related art, it is impossible to simultaneously ensure the virtual image generation efficiency and the virtual image generation effect. That is, based on the model topology conversion method in the related art, the problem of low virtual image generation efficiency and poor virtual image generation effect will occur.

[0070] The embodiment of the present application provides a virtual image generation method and device, electronic equipment, computer readable storage medium and computer program product, which can guarantee the generation effect of the virtual image and improve the generation efficiency of the virtual image. The following describes an exemplary application of the electronic equipment provided by the embodiment of the present application. The electronic equipment provided by the embodiment of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart television, a vehicle-mounted terminal and various types of terminals, and can also be implemented as a server. The following describes an exemplary application of the electronic equipment implemented as a server.

[0071] Referring to Figure 1 , Figure 1 is an architecture schematic diagram of a virtual image generation system provided by the embodiment of the present application. To realize the support of a virtual image generation application, in the virtual image generation system 100, the terminal 400 (exemplarily shows the terminal 400-1 and the terminal 400-2) is connected to the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two. The virtual image generation system 100 is also configured with a database 500 to provide data support to the server 200, for example, to provide related data of a template mesh model and the like, and the database 500 can be independent of the server 200, or can be configured in the server 200. Figure 1 It is shown that the database 500 is independent of the server 200.

[0072] The terminal 400 is used to collect a two-dimensional image of a target object in response to a trigger operation of a user on an image collection and uploading mark displayed on a graphical interface (exemplarily shows the graphical interface 410-1 and the graphical interface 410-2), and upload the two-dimensional image to the server 200 through the network 300.

[0073] The server 200 is used to reconstruct a first source mesh model of a target object under a source model topology based on a two-dimensional image of the target object; read a dense vertex parameter for describing a connection between a facet vertex of a template mesh model under a business model topology and a mesh facet of the first source mesh model; update vertex information of the facet vertex of the template mesh model based on the dense vertex parameter and the vertex information of the facet vertex of the first source mesh model, to obtain an updated mesh model under the business model topology; determine a rendering result of the updated mesh model as a three-dimensional virtual image of the target object; and issue picture content of the three-dimensional virtual image to the terminal 400.

[0074] The terminal 400 displays the picture content of the three-dimensional virtual image on the graphical interface.

[0075] The embodiments of the present application can be implemented with the help of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network within a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0076] Cloud computing is a general term for network technologies, information technology, integration technologies, management platforms, and application technologies used in the cloud computing business model. These technologies can form resource pools that are used on demand and are flexible and convenient. Cloud computing technology will become a crucial support. System backend services within technical networks require extensive computing and storage resources, which must be implemented through cloud computing.

[0077] Exemplarily, server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and the server may be connected directly or indirectly via wired or wireless communication, which is not limited in the embodiments of the present application.

[0078] See also Figure 2 , Figure 2 This embodiment of the present application provides Figure 1 A schematic diagram of the structure of a server (an implementation of an electronic device) in FIG. Figure 2 The server 200 shown includes: at least one processor 210, a memory 250, and at least one network interface 220. The various components in the server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 2 Various buses are labeled as bus system 240 .

[0079] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0080] The memory 250 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, and the like. The memory 250 optionally includes one or more storage devices remotely located from the processor(s) 210.

[0081] The memory 250 includes volatile memory or nonvolatile memory, and can also include both volatile and nonvolatile memory. Nonvolatile memory can be read only memory (ROM), volatile memory can be random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0082] In some embodiments, the memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or a subset or superset thereof, which are exemplarily illustrated below.

[0083] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks;

[0084] The network communication module 252 is used to communicate with other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 include Bluetooth, wireless compatibility certification (WiFi), and universal serial bus (USB), and the like;

[0085] In some embodiments, the virtual image generation device provided by the embodiments of the present application can be realized in a software manner, Figure 2 The virtual image generation device 255 stored in the memory 250 is shown, which can be software in the form of programs and plug-ins, including the following software modules: model reconstruction module 2551, parameter reading module 2552, model updating module 2553, model rendering module 2554 and parameter generation module 2555, these modules are logical, so according to the realized function can be any combination or further split. The functions of each module will be described below.

[0086] In other embodiments, the virtual image generation device provided in the embodiments of the present application can be implemented in hardware. As an example, the virtual image generation device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the virtual image generation method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0087] In some embodiments, a terminal or a server (both are possible implementations of electronic devices) can implement the virtual image generation method provided in the embodiments of the present application by running various computer-executable instructions or computer programs. For example, computer-executable instructions may be microprogram-level commands, machine instructions, or software instructions. A computer program may be a native program or software module in an operating system; it may be a native application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as a live broadcast APP or a game APP; it may also be a small program that can be embedded in any APP, that is, a program that only needs to be downloaded to a browser environment to run. In short, the above-mentioned computer-executable instructions may be instructions in any form, and the above-mentioned computer program may be an application, module, or plug-in in any form.

[0088] The embodiments of the present application can be applied to virtual image generation scenarios in games, virtual reality, animation production, etc. Below, the virtual image generation method provided by the embodiments of the present application will be described in conjunction with the exemplary application and implementation of the electronic device provided by the embodiments of the present application. As mentioned above, the electronic device that implements the virtual image generation method of the embodiments of the present application can be a terminal, a server, or a combination of the two. Therefore, the execution entity of each step will not be repeated below.

[0089] See also Figure 3 , Figure 3 This is a flow chart of the virtual image generation method provided in the embodiment of the present application. Figure 1 , will combine Figure 3 The steps shown are explained, Figure 3 The main body of the step is the electronic device.

[0090] Step 101, based on the two-dimensional image of the target object, a first source mesh model of the target object in the source model topology is reconstructed.

[0091] Embodiments of the present application are implemented in the scene of generating a three-dimensional virtual image with appearance features of a target object. The generated three-dimensional virtual image can be applied to a game scene, for example, creating a game character for the target object based on the three-dimensional virtual image, and can also be applied to an animation scene, for example, generating an animated character for the target object based on the three-dimensional virtual image. After the virtual image generation starts, the electronic device will first acquire a two-dimensional image of the target object, and perform three-dimensional model reconstruction on the target object based on the two-dimensional image. The reconstructed mesh model is determined as the first source mesh model.

[0092] It should be noted that in the embodiments of the present application, the model topology used in the three-dimensional model reconstruction is not the same as the model topology used in the business pipeline. The model topology used in the three-dimensional model reconstruction is called source model topology, and the business model topology refers to the model topology used in the business pipeline. Therefore, the first source mesh model is a mesh model in the source model topology. The model topology describes the connection mode of the patch vertex in the mesh model. Different connection modes of the patch vertex, for example, different connection sequences of the patch vertex, will result in different adjacent nodes for the same patch vertex, thereby causing different adjacent vertices to be affected when a certain patch vertex moves, resulting in different deformation modes of the mesh patch.

[0093] In the embodiments of the present application, the target object can be a face, a human body, or other objects such as a robot or an animal. The two-dimensional image of the target object can be a color image (RGB image), a grayscale image, or an infrared image. The present application does not limit the embodiments. The two-dimensional image can be collected by another device (such as a terminal) and sent to the electronic device, or collected by the electronic device, or downloaded from the network by the electronic device. The present application does not limit the embodiments.

[0094] In the embodiments of the present application, the electronic device can use existing three-dimensional reconstruction techniques (such as deep learning models, rasterization algorithms, etc.) to reconstruct the three-dimensional model of the target object.

[0095] Taking a human face as an example, the electronic device can use a 3DMM model to reconstruct a three-dimensional model to obtain a first source mesh model. The electronic device first extracts features from a two-dimensional image to obtain facial features (such as key points, facial contours, etc.) of the target object, and then performs weighted combination of the obtained facial features and principal components of the 3DMM model through linear interpolation or weighted summation. Then, according to the result of the weighted combination, a corresponding three-dimensional mesh model is generated, thereby completing the reconstruction of the three-dimensional model. The electronic device can also use a deep learning model such as a convolutional neural network or a generative adversarial network to learn the underlying features of the face from the two-dimensional image, and generate a mesh model according to the extracted underlying features, thereby obtaining the first source mesh model.

[0096] In step 102, a dense vertex parameter describing the correspondence between the patch vertices of the template mesh model under the business model topology and the first source mesh model is read.

[0097] The electronic device reads the dense vertex parameter from its own storage space or from the network, where the dense vertex parameter describes the correspondence between the patch vertices of the template mesh model and the mesh patches of the first source mesh model, that is, the required parameters for determining the vertex information of the patch vertices of the template mesh model through the mesh patches of the first source mesh model. It should be noted that the correspondence determined by the dense vertex parameter in the embodiments of the present application under the same appearance feature, that is, before the electronic device starts to generate the virtual image for the target object, the source mesh model (i.e., the second source mesh model in the following) with the appearance feature of the other object and the template mesh model with the appearance feature of the other object can be used to perform vertex matching.

[0098] In the embodiments of the present application, the template mesh model is a three-dimensional mesh model that has been constructed under the business model topology, and the electronic device can generate a completely new three-dimensional mesh model under the business model topology by adjusting the vertex information of its patch vertices. Thus, the three-dimensional mesh model of the target object under the business model topology can be obtained by adjusting the template mesh model. Therefore, the template mesh model does not have the appearance feature of the target object, but has the appearance feature of another object (e.g., the template object in the following), and the appearance feature of the target object needs to be adjusted to the template mesh model to be reused in the template mesh model.

[0099] In some embodiments, the template mesh model can be a three-dimensional mesh model constructed for a specific object; taking a face as an example, the template mesh model can be a three-dimensional mesh model constructed for a specific template face. In other embodiments, the template mesh model can be a three-dimensional mesh model constructed for any one object of the same kind as the target object; taking a face as an example, the template mesh model can be a three-dimensional mesh model constructed for any one face other than the target face.

[0100] It should be noted that the dense vertex parameter at least includes the target index and the target index in the barycentric parameter. The target index is the vertex index of the mesh patch corresponding to the determination of the patch vertex of the template mesh model in the first source mesh model, so that the target index provides the correspondence between the vertex and the patch. The barycentric parameter is used to calculate the barycentic coordinate of the mesh patch of the first source mesh model.

[0101] In some embodiments of the present application, Figure 3 The step 102 in the method 100, i.e., reading the dense vertex parameter for describing the connection between the patch vertex of the template mesh model under the business model topology and the mesh patch of the first source mesh model, can be implemented by the following processing: reading the dense vertex parameter for describing the connection between the patch vertex of the template mesh model and the mesh patch of the first source mesh model from the vertex index of the patch vertex of the template mesh model.

[0102] That is, in the embodiments of the present application, the dense vertex parameter can be stored in the vertex index of the template mesh model, and the electronic device reads the vertex index of each patch vertex of the template mesh model after obtaining the template mesh model, so as to obtain the dense vertex parameter.

[0103] In other embodiments of the present application, Figure 3 The step 102 in the method 100, i.e., reading the dense vertex parameter for describing the connection between the patch vertex of the template mesh model under the business model topology and the mesh patch of the first source mesh model, can also be implemented by the following processing: reading the dense vertex parameter from the storage area for storing the dense vertex parameter.

[0104] That is, in the embodiments of the present application, the dense vertex parameter is stored in a separate storage area, and the electronic device reads the storage area to obtain the dense vertex parameter.

[0105] It should be noted that in the embodiments of the present application, the execution order between the step 101 and the step 102 does not affect the final three-dimensional virtual image, so in some embodiments, the electronic device can also execute the step 102 first, and then execute the step 101, or execute the step 101 and the step 102 simultaneously, which is not limited in the embodiments of the present application.

[0106] In step 103, vertex information of a vertex of a patch of the template mesh model is updated based on the dense vertex parameter and the vertex information of the vertex of the patch of the first source mesh model, to obtain an updated mesh model under the business model topology.

[0107] After obtaining the dense vertex parameter and the first source mesh model, the electronic device determines, from the mesh patches of the first source mesh model, a mesh patch that matches a vertex of a patch of the template mesh model in combination with the dense vertex parameter, and updates the vertex information of the vertex of the patch of the template mesh model by using the vertex information of the vertices of the mesh patch, and determines the template mesh model after the vertex information is updated as the updated mesh model. It can be understood that the model topology of the template mesh model is not changed in the process of updating the vertex information of the vertex of the patch of the template mesh model, and therefore the updated mesh model is also under the business model topology.

[0108] It can be understood that, due to the difference in vertex density between the two mesh models, if a vertex of a mesh model is matched with a vertex of another mesh model to transfer the appearance feature, some vertex information of the vertex is likely to be lost, which leads to the loss of the appearance feature described by the vertex information. In the embodiment of the present application, the electronic device updates the vertex information of the vertex of the patch of the template mesh model based on the dense vertex parameter and the vertex information of the vertex of the patch of the first source mesh model, which actually determines a suitable mesh patch from the first source mesh model, and then updates the vertex information of the vertex of the patch of the template mesh model based on the vertex information of all the vertices of the mesh patch, so that the loss of the vertex information of some vertices of the first source mesh model is avoided.

[0109] The vertex information of the vertex of the patch can include at least one of a vertex coordinate of the vertex of the patch, a texture coordinate, color information, and a vertex weight. The vertex coordinate is used to determine the position of the vertex of the patch in a three-dimensional space, the texture coordinate is used to determine the position of the vertex of the patch on a texture map, and the color information is used to describe the color and glossiness and other visual effects of the vertex of the patch.

[0110] It should be noted that the appearance features of different objects are embodied by the vertex information of the vertices of the patches of the mesh models. Taking a face as an example, if a vertex of a patch is used to represent a nose tip, the vertex coordinate of the vertex actually embodies the position of the nose tip on the face, and the texture coordinate embodies the specific shape of the nose tip. It can be seen that the appearance features of the faces of different objects are actually embodied by the vertex information of the vertices of the patches. Therefore, in the embodiment of the present application, the vertex information of the vertices of the patches of the template mesh model is updated in combination with the vertex information of the vertices of the patches of the first source mesh model, which actually introduces the appearance features of the target object into the template mesh model.

[0111] When only the target index is included in the dense vertex parameter, the electronic device can update the vertex information of the face vertex of the template grid model only according to the target index and the vertex information of the face vertex of the first source grid model. For example, for the face vertex of the template grid model, the target vertex is determined from the face vertex of the first source grid model according to the target index, and the vertex information of the face vertex of the template grid model is updated using the mean value of the vertex information of the plurality of target vertices to obtain the updated grid model.

[0112] When the target index and the barycenter parameter are included in the dense vertex parameter, the electronic device updates the vertex information of the face vertex of the template grid model by combining the target index, the barycenter parameter, and the vertex information of the face vertex of the first source grid model.

[0113] In more detail, refer to Figure 4 , Figure 4 is a flowchart of a virtual image generation method provided by an embodiment of the present application Figure 2 In some embodiments of the present application, the dense vertex parameter includes a target index and a barycenter parameter, and the vertex information includes vertex coordinates, Figure 3 Step 103 in the method, i.e., updating the vertex information of the face vertex of the template grid model based on the dense vertex parameter and the vertex information of the face vertex of the first source grid model to obtain the updated grid model under the business model topology, can be implemented by the following processing:

[0114] Step 1031, determining a target vertex from the face vertex of the first source grid model according to the target index, and determining the vertex coordinates of the target vertex as target coordinates.

[0115] The electronic device determines the face vertex recorded by the target index as the target vertex, and obtains the vertex coordinates of the target vertex, and determines the obtained vertex coordinates as the target coordinates.

[0116] For example, when the grid face is a triangular face, the electronic device extracts the vertex coordinates of the 3 face vertices constituting the triangular face according to the vertex index of the triangular face, and determines the vertex coordinates as the target coordinates.

[0117] Step 1032, calculating the dense vertex coordinates using the target coordinates and the barycenter parameter, and updating the vertex coordinates of the face vertex of the template grid model using the dense vertex coordinates to obtain the updated grid model under the business model topology.

[0118] After determining the target coordinates, the electronic device calculates the barycentric coordinates based on the target coordinates and the barycentric parameters (the specific calculation process is to perform weighted summation on the target coordinates by using the sub-parameters in the barycentric parameters), and determines the calculated barycentric coordinates as the dense vertex coordinates, and then uses the obtained dense vertex coordinates to cover the vertex coordinates of the patch vertices of the template mesh model, so that the updating of the vertex coordinates of the patch vertices of the template mesh model can be completed, and the updated mesh model is obtained.

[0119] It should be noted that, for a polygon, assuming that the number of vertices thereof is N, the barycentric parameters need to satisfy the condition of being added to 1 (i.e., the sum of the sub-parameters contained in the barycentric parameters is 1). Therefore, in order to save storage space and data transmission consumption, only the first N-1 sub-parameters in the barycentric parameters can be recorded, and the Nth sub-parameter can be calculated by using 1 and the first N-1 sub-parameters when calculating the barycentric coordinates, and then the weighted summation processing is performed on each vertex coordinate in the target coordinates.

[0120] For example, according to the target index, the shape of the target patch is determined to be a triangle, the barycentric parameters include a first sub-parameter and a second sub-parameter, and the target coordinates include a first vertex coordinate, a second vertex coordinate and a third vertex coordinate. At this time, the electronic device can first obtain a third sub-parameter by subtracting the first sub-parameter and the second sub-parameter from 1 in turn, and then perform weighted summation calculation on the first vertex coordinate, the second vertex coordinate and the third vertex coordinate by using the first sub-parameter, the second sub-parameter and the third sub-parameter, so as to obtain the barycentric coordinates, i.e., the dense vertex coordinates.

[0121] In step 104, the rendering result of the updated mesh model is determined as the three-dimensional virtual image of the target object.

[0122] After obtaining the updated mesh model, the electronic device can call a rendering engine and import the updated mesh model into the rendering engine for rendering processing, such as performing texture mapping, adding shadows, adding special effects and the like on the updated mesh model, and output the rendering result of the rendering engine as the three-dimensional virtual image of the target object.

[0123] It should be noted that the electronic device can use an existing rendering engine to render the updated mesh model, for example, using Open Graphics Library (OpenGL) or Blender for rendering.

[0124] It can be understood that, compared with the model topology conversion method in the related art, the generation efficiency of the virtual image is low and the generation effect of the virtual image is poor. In the embodiment of the present application, the electronic device will first use the two-dimensional image to perform grid model reconstruction for the target object under the source model topology to obtain a first source grid model. Meanwhile, the relationship between the patch vertices of the template grid model and the grid patches of the first source grid model is combined with the dense vertex parameters. More appropriate vertex information is determined for the patch vertices of the template grid model from the first source grid model to update the patch vertices of the template grid model. Thus, not only can the appearance features of the target object be automatically transferred to the template grid model to improve the generation efficiency of the virtual image, but also the appropriate grid patches can be determined for the patch vertices of the template grid model in combination with the dense vertex parameters. The vertex information is updated based on the vertex information of all patch vertices of the grid patches, so that the appearance features of the target object can be more completely retained to obtain a three-dimensional image model that is more consistent with the appearance of the target object, thereby improving the generation effect of the virtual generation.

[0125] Next, the determination process of the dense vertex parameters in the embodiment of the present application is described.

[0126] Referring to Figure 5 , Figure 5 is the flowchart of the virtual image generation method provided by the embodiment of the present application Figure 3 In some embodiments of the present application, the dense vertex parameters can be obtained by the following processing:

[0127] Step 201, based on the two-dimensional image of the template object, a second source grid model of the template object under the source model topology is reconstructed.

[0128] When generating the dense vertex parameters, the electronic device will first acquire the two-dimensional image of the template object, and perform three-dimensional grid model reconstruction on the two-dimensional image of the template object to obtain a second source grid model under the source model topology. The specific process of the three-dimensional grid model reconstruction of the template object is similar to that of the three-dimensional grid model reconstruction of the target object, which is not described here.

[0129] It should be noted that the template object is an object used for generating dense vertex parameters, which can be an object of the same type as the target object but with a different appearance. If the target object is the face of a human being, the template object can be the face of another human being.

[0130] Step 202, aligning the second source grid model and the template grid model to obtain an aligned grid model of the template object under the business model topology.

[0131] After obtaining the second source grid model, the electronic device aligns the second source grid model and the template grid model topology, so that the second source grid model is converted from the source model topology to the business model topology, and the obtained new grid model is the aligned grid model. That is, the aligned grid model is under the business model topology and has the appearance characteristics of the template object.

[0132] In some embodiments of the present application, Figure 5 The step 201 of aligning the second source grid model and the template grid model to obtain the aligned grid model of the template object under the business model topology can be implemented by the following processing process: obtaining the first coordinate from the first display area of the model display interface, and obtaining the second coordinate from the second display area of the model display interface; wherein the first display area and the second display area are respectively used to display the second source grid model and the template grid model; forming a vertex pair by using the face vertex corresponding to the first coordinate and the face vertex corresponding to the second coordinate; aligning the second source grid model and the template grid model through the vertex pair to obtain the aligned grid model of the template object under the business model topology.

[0133] In the embodiments of the present application, the electronic device displays the second source grid model and the template grid model in the first display area and the second display area of the model display interface respectively, wherein the second source grid model can be displayed in any one of the first display area and the second display area, and the template grid model can be displayed in the remaining display area. Then, when the electronic device detects that the worker has performed a vertex designation operation (such as a click, double-click, or other operation that can designate a vertex) in the first display area and the second display area, the electronic device obtains the first coordinate and the second coordinate based on the vertex designation operation, determines the corresponding face vertex in the second source grid model and the template grid model based on the first coordinate and the second coordinate respectively (when displaying the grid model, the face vertex will be associated with the coordinate on the model display interface to be displayed, so the face vertex can be determined based on the obtained coordinate), and forms a vertex pair by using the two determined face vertices. Finally, the electronic device can align the grid model based on the vertex pair. For example, the electronic device reconnects the vertices in the grid face to which the face vertex under the business model topology belongs in the vertex pair.

[0134] In some embodiments of the present application, Figure 5The step 201 in the method 1000, i.e., aligning the second source mesh model and the template mesh model to obtain the aligned mesh model of the template object under the business model topology, can also be implemented through the following process: calling a neural network model for model alignment, and aligning the second source mesh model and the template mesh model through the neural network model to obtain the aligned mesh model of the template object under the business model topology.

[0135] The step 203 is to determine the target index and the gravity center parameter based on the aligned mesh model and the second source mesh model, and generate the dense vertex parameter by using the target index and the gravity center parameter.

[0136] Since the aligned mesh model has the appearance feature of the template object and the mesh model of the business model topology, it is the same as the second source mesh model in appearance feature, i.e., two mesh models under the same appearance feature but different model topologies, so based on the appearance feature, for example, the vertex coordinates, the connection between the face vertices of the aligned mesh model and the mesh faces of the second source mesh model can be obtained, and the target index and the gravity center parameter are also obtained, and finally the target index and the gravity center parameter are used to constitute the dense vertex parameter.

[0137] Referring to Figure 6 , Figure 6 is a flowchart of a virtual image generation method provided by an embodiment of the present application Figure 4 In some embodiments of the present application, Figure 5 The step 203 in the method 1000, i.e., determining the target index and the gravity center parameter based on the aligned mesh model and the second source mesh model, can be implemented through the following steps:

[0138] The step 2031 is to perform vertex densification on the second source mesh model to obtain a dense mesh model.

[0139] The electronic device performs subdivision on the second source mesh model to increase the number of face vertices of the second source mesh model, so as to obtain a second source mesh model with more dense vertices, and the mesh model is denoted as a dense mesh model.

[0140] In some embodiments of the present application, Figure 6The process of step 2031 in the method 2000, i.e., the process of vertex densification on the second source mesh model to obtain the dense mesh model, can be implemented by the following processing: performing the following processing by iteration i, where i is a positive integer: performing the i-th mesh subdivision on the second source mesh model based on the subdivision degree of the i-th iteration to obtain an i-th subdivided mesh model; for each face vertex of the aligned mesh model, determining the nearest candidate vertex from the subdivided vertices of the i-th subdivided mesh model; calculating the distance mean value by using the distance between each face vertex of the aligned mesh model and the corresponding candidate vertex; and determining the i-th subdivided mesh model as the dense mesh model when the distance mean value is less than or equal to the distance threshold value.

[0141] That is, in the embodiment of the present application, the electronic device performs one mesh subdivision on the second source mesh model, and finds the nearest candidate vertex for each face vertex of the aligned mesh model from the face vertices of the subdivided mesh model obtained by the mesh subdivision, and then calculates the distance mean value between each face vertex and the corresponding candidate vertex, and compares the distance mean value with the distance threshold value as a processing process of one iteration. When the distance mean value is less than the distance threshold value, it indicates that each face vertex of the aligned mesh model is close enough to the corresponding candidate vertex, and thus the subdivision degree of the i-th iteration can exactly reach the subdivision degree at which a suitable vertex can be determined for each face vertex of the aligned mesh model. Therefore, the electronic device will stop iteration when it is determined that the distance mean value reaches the distance threshold value, and directly takes the subdivided mesh model of the last iteration as the dense mesh model.

[0142] It should be noted that the distance threshold value can be set according to actual needs, for example, set to 2, set to 1, etc., and can also be determined according to the required degree of refinement of the template object in three-dimensional modeling, which is not limited in the embodiment of the present application.

[0143] The subdivision degree of the i-th iteration can be random or can be obtained by increasing the subdivision degree of the (i-1)-th iteration (when i=1, the subdivision degree can take a random value), which is not limited in the embodiment of the present application.

[0144] In some embodiments of the present application, after the distance mean value is calculated by using the distance between each face vertex of the aligned mesh model and the corresponding candidate vertex, the following processing can also be performed: when the distance mean value is greater than the distance threshold value, increasing the subdivision degree of the i-th iteration to obtain the subdivision degree of the (i+1)-th iteration.

[0145] The increase here can be superimposing an incremental value on the basis of the subdivision degree of the ith iteration, or directly multiplying the subdivision degree of the ith iteration by a factor. For example, when the subdivision degree of the ith iteration is 10, the subdivision degree of the (i+1)th iteration can be obtained by iterating an incremental value of 5, or by multiplying by a factor of 2 to obtain a subdivision degree of 20 in the (i+1)th iteration. Embodiments of the present application do not limit this.

[0146] In some embodiments of the present application, based on the subdivision degree of the ith iteration, the second source mesh model is subjected to the ith mesh subdivision to obtain the ith subdivided mesh model, which can be achieved by the following processing: according to the subdivision degree of the ith iteration, N candidate parameters are generated for each mesh patch of the second source mesh model, where N is a positive integer; using the vertex coordinates of each mesh patch of the second source mesh model and the N candidate parameters, N subdivided vertices are determined for each mesh patch of the second source mesh model; using the N subdivided vertices, subdivided patches of each mesh patch are generated, and the subdivided patches of each mesh patch are used to constitute the ith subdivided mesh model.

[0147] That is, the electronic device first determines how many subdivided vertices need to be added in each mesh patch of the second source mesh model according to the subdivision degree of the ith iteration, thereby determining how many candidate parameters need to be generated. After generating the required number of candidate parameters, the electronic device performs weighted summation on the vertex coordinates of each mesh patch of the second source mesh model using the sub-parameters in the generated candidate parameters to obtain the corresponding subdivided vertices. Finally, the electronic device connects the subdivided vertices to become subdivided patches, and finally uses the subdivided patches to constitute the subdivided mesh model. In this way, the process of the ith mesh subdivision is completed.

[0148] It should be noted that each candidate parameter here contains the same number of sub-parameters as the patch vertices of the mesh patches of the second source mesh model, and these sub-parameters satisfy the condition that the sum is 1.

[0149] In other embodiments of the present application, Figure 6 The step 2031 in the above, i.e., the process of performing vertex densification on the second source mesh model to obtain a dense mesh model, can also be achieved by the following processing: randomly inserting vertices in the mesh patches of the second source mesh model, and obtaining the dense mesh model after completing the vertex insertion.

[0150] It should be noted that the number of inserted vertices in different mesh patches of the second source mesh model can be the same or different. For example, three vertices can be inserted at any three positions for a mesh patch of the second source mesh model, and three vertices can be inserted for another mesh patch, or two vertices can be inserted at any two positions, which is not limited in the embodiments of the present application.

[0151] In step 2032, a matching vertex is determined from the patch vertex of the dense mesh model for the patch vertex of the aligned mesh model.

[0152] The electronic device filters the matching vertex for each patch vertex of the aligned mesh model from the patch vertices of the dense mesh model, and takes the filtered patch vertex of the dense mesh model as the matching vertex corresponding to each patch vertex of the aligned mesh model.

[0153] It should be noted that the patch vertex of the dense mesh model is determined from the inside of the mesh patch of the second source mesh model, so that the corresponding matching vertex of the dense mesh model is determined for the patch vertex of the aligned mesh model, that is, the process of determining the corresponding mesh patch of the second source mesh model for the patch vertex of the aligned mesh model.

[0154] In the embodiments of the present application, the vertex coordinates of the patch vertex of the dense mesh model are calculated from the vertex coordinates of the patch vertex of the second source mesh model and the corresponding candidate parameters. In more detail, when calculating the vertex coordinates of the patch vertex of the dense mesh model, the electronic device randomly generates a plurality of candidate parameters capable of calculating the barycentric coordinates (the patch vertex of the dense mesh model is equivalent to the barycentric coordinates of the mesh patch of the second source mesh model, each candidate parameter includes a plurality of sub-parameters which are the same as the number of patch vertices of the mesh patch of the second source mesh model, as long as the cumulative result is 1, these sub-parameters can be used as candidate parameters, and based on different candidate parameters, the calculated vertex coordinates of the patch vertex of the dense mesh model are different), and then the vertex coordinates of the patch vertex of the dense mesh model are calculated based on the candidate parameters.

[0155] In some embodiments of the present application, Figure 6 The process of step 2032 in the above, that is, determining the matching vertex from the patch vertex of the dense mesh model for the patch vertex of the aligned mesh model, can be implemented by the following processing: calculating the distance between the patch vertex of the aligned mesh model and the patch vertex of the dense mesh model to obtain the vertex distance, and taking the patch vertex of the dense mesh model closest to the patch vertex of the aligned mesh model as the matching vertex of the patch vertex of the aligned mesh model.

[0156] That is, the electronic device can determine the corresponding matching vertices by searching for the nearest neighbors of the patch vertices of the aligned mesh model from the dense mesh model.

[0157] In other embodiments of the present application, Figure 6 Step 2032, i.e., the process of determining matching vertices for the patch vertices of the aligned mesh model from the patch vertices of the dense mesh model, can also be achieved by projecting the patch vertices of its mesh model onto the dense mesh model, and determining the patch vertex in the dense mesh model that is closest to the projected position as the matching vertex for the patch vertices of the aligned mesh model.

[0158] Step 2033: Determine the vertex index of the mesh patch corresponding to the matching vertex in the second source mesh model as the target index, and determine the candidate parameter used in calculating the vertex coordinates of the matching vertex as the center of gravity parameter.

[0159] After obtaining the matching vertices of the patch vertices of the aligned mesh model, the electronic device will search for the mesh patches corresponding to the patch vertices of the aligned mesh model from the second source mesh model based on the matching vertices, that is, it will search for the mesh patches in the second source mesh model that contain the matching vertices, and then use the vertex index of the mesh patch as the target index. At the same time, the electronic device will directly determine the candidate parameters used to calculate the vertex coordinates of the matching vertex as the final center of gravity parameters. In this way, the electronic device can obtain the target index and center of gravity parameters to facilitate the generation of dense vertex parameters.

[0160] In other embodiments of the present application, Figure 5 Determining the target index and center of gravity parameters based on the aligned mesh model and the second source mesh model in step 203 can be achieved by the following steps: determining the projection vertex from the second source mesh model for the patch vertices of the aligned mesh model; determining the vertex index of the mesh patch to which the projection vertex in the second source mesh model belongs as the target index; and determining the center of gravity parameters using the vertex coordinates of the projection vertex and the vertex coordinates of the patch vertices of the mesh patch to which the projection vertex belongs.

[0161] It should be noted that the process of determining the projected vertex in the embodiments of the present application can be completed by projection from a point to a surface. When calculating the center of gravity parameters, the electronic device can use the vertex coordinates of the projected vertex as the center of gravity coordinates, and then combine them with the vertex coordinates of each mesh facet vertex and the relationship between the center of gravity coordinates and the vertex coordinates to inversely calculate the center of gravity parameters.

[0162] It can be understood that, by means of projection, a more suitable vertex can be quickly determined in the second source mesh model for the vertex of the patch in the aligned mesh model, so as to accelerate the generation speed of the target index and the barycenter parameter, and thus accelerate the generation speed of the dense vertex parameter.

[0163] Next, an application scenario in which the embodiments of the present application can be used will be exemplified.

[0164] The embodiments of the present application can be applied to a virtual image generation scenario of a game, at this time, the two-dimensional image of the target object is a facial image of a game player, and the three-dimensional virtual image is a three-dimensional role image of the target object in the game. At this time, the embodiments of the present application can solve the problems of slow generation of the three-dimensional role image of the game and large difference between the three-dimensional role image and the real face of the target object, and can achieve the effect of improving the generation efficiency of the three-dimensional role image while ensuring the generation effect of the three-dimensional role image.

[0165] The embodiments of the present application can also be applied to a virtual image generation scenario of a three-dimensional animation, at this time, the two-dimensional image of the target object is a two-dimensional design draft of an animation role, and the three-dimensional virtual image is a three-dimensional role image of the animation role. At this time, the embodiments of the present application can solve the problems of low generation efficiency of the three-dimensional role image of the animation and large difference between the three-dimensional role image and the two-dimensional design draft, and can achieve the effect of improving the generation efficiency of the three-dimensional role image while ensuring the generation effect of the three-dimensional role image.

[0166] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0167] The embodiments of the present application are implemented in a scenario of photo face pinching in a game to obtain a 3D head model with an appearance consistent with the appearance of a player and under a topology specified by a business pipeline.

[0168] First, the difference between the 3DMM model topology and the business model topology will be described.

[0169] Exemplarily, Figure 7 is a comparison diagram of the 3DMM model topology and the business model topology provided by the embodiments of the present application. Referring to Figure 7 , the head model of the 3DMM model topology 7-1 is more detailed, and the number of vertices reaches 20,000, while the head model of the business model topology 7-2 has only 1,000 vertices. It can be seen that the vertex density of the head model of the 3DMM model topology and the head model of the business model topology is quite different.

[0170] To this end, in the embodiments of the present application, the mapping relationship between the 3DMM model topology and the business model topology is first determined under the same face (referred to as a template object), and the mapping relationship is marked by the center of gravity parameters of the densified vertices and the vertex index of the face sheet (referred to as a target index). For the face of the player, first, the 2D face picture of the player (referred to as a target object) (referred to as a two-dimensional image of the target object) is reconstructed by 3DMM to obtain a 3D head mesh model (referred to as a first source mesh model) under the 3DMM model topology (referred to as a source model topology), and then the identity features (referred to as appearance features) of the player are transferred to the 3D mesh model (referred to as a template mesh model) of any face under the business model topology based on the mapping relationship between the 3DMM model topology and the business model topology, and based on the 3D mesh model with the identity features of the player, the 3D game character (referred to as a three-dimensional virtual image) of the player is rendered.

[0171] Exemplary, Figure 8 is a flowchart of the model topology conversion provided by the embodiments of the present application. Referring to Figure 8 , in the mapping relationship construction stage 8-1, first, the 3DMM mesh model 8-11 (referred to as a second source mesh model) under the 3DMM model topology and the target mesh model 8-12 (referred to as a template mesh model) under the business model topology are aligned to obtain a new mesh model 8-13 (referred to as an aligned mesh model) with the identity features of the 3DMM mesh model and the model topology of the target mesh model; at the same time, the 3DMM mesh model 8-11 is mesh subdivided to obtain the 3DMM mesh model 8-14 (referred to as a dense mesh model) of the densified vertices; then, the nearest vertices (referred to as matching vertices) for each vertex (referred to as a face sheet vertex) in the new mesh model 8-13 are searched from the 3DMM mesh model 8-14 of the densified vertices by KNN nearest neighbor search, so that the mapping relationship 8-15 between the 3DMM model topology and the business model topology is obtained. In the mapping relationship application stage 8-2, the mapping relationship 8-15 between the 3DMM model topology and the business model topology is applied to the 3D head mesh model 8-21 reconstructed by 3DMM for the player to obtain a new mesh model 8-22 (referred to as an updated mesh model) with the identity features of the 3D head mesh model and the business model topology.

[0172] Next, the mapping relationship determination process between the 3DMM model topology and the business model topology is described.

[0173] First, in the embodiments of the present application, the 3DMM mesh model with the open-mouth action and the target mesh model are obtained at the same time, the 3DMM mesh model and the target mesh model are aligned using software capable of topological alignment, and the identity features of the 3DMM mesh model and the model topology of the target mesh model, i.e., a new mesh model of the business model topology, are obtained. Here, the mesh model with the open-mouth action is selected to make the mesh models of two different model topologies consistent in shape as much as possible, so as to obtain the correspondence between points and points and between points and surfaces in different topologies.

[0174] Exemplary, Figure 9 is a schematic diagram provided by the embodiments of the present application for topological alignment of two mesh models. In the embodiments of the present application, the electronic device can provide the interface 9-1 (referred to as a model display interface) of the topological alignment software to the R&D personnel, the target mesh model is displayed in the area 9-11 (referred to as a first display area), and the 3DMM mesh model is displayed in the area 9-12 (referred to as a second display area). Then, the electronic device can obtain the coordinates of the key points (referred to as first coordinates and second coordinates) specified by the R&D personnel through a click operation on the interface 9-1, and these key points can include forehead, eye corner, nose tip, nose wing, mouth corner, lip, ear, etc. Then, the topological alignment is realized through these coordinates.

[0175] Figure 10 is a schematic diagram of the model topology alignment result provided by the embodiments of the present application, so that, Figure 10 The mesh model displayed in the above is actually a new mesh model 10-1 with the identity features of the 3DMM mesh model and the topology of the target mesh model.

[0176] Then, the electronic device will perform mesh subdivision on the 3DMM mesh model to obtain a 3DMM mesh model with dense vertices, and the subdivision degree needs to be adjusted according to the average distance of subsequent calculation. The specific process will be described below. The position of the vertex of the 3DMM mesh model with dense vertices can be obtained through the original vertex position (referred to as vertex coordinates) of the 3DMM mesh model and the corresponding parameters (referred to as candidate parameters) for calculating the gravity center.

[0177] In more detail, any vertex of the 3DMM mesh model with dense vertices can be represented as {fid, λ1, λ2}, where fid represents the serial number of the triangular patch (referred to as mesh patch) of the 3DMM mesh model, and λ1, λ2 are parameters for calculating the gravity center. The position of any subdivided vertex can be obtained through the gravity center coordinate formula, i.e., for any point V in the spatial triangle △V1V2V3, there must be a unique λ1, λ2 to satisfy formula (1):

[0178]

[0179] For example, Figure 11 FIG11 is a schematic diagram of a local area of ​​a 3DMM mesh model with dense vertices provided in an embodiment of the present application. As can be seen from FIG11-1 , the 3DMM mesh model with dense vertices has more vertices than the original 3DMM mesh model.

[0180] After obtaining a new mesh model with the identity characteristics of the 3DMM mesh model and the topology of the target mesh model, as well as a 3DMM mesh model with dense vertices, the electronic device will use the nearest neighbor method to find the nearest neighbor point (called a matching vertex) for each vertex on the new mesh model from the 3DMM mesh model with dense vertices. The average distance between the matching points obtained by the nearest neighbor search determines the degree of subdivision mentioned above.

[0181] In more detail, assume a matching point set M, where for each vertex m of the new mesh model i , there is a nearest neighbor v in M ​​determined from the 3DMM mesh model of the densified vertices i , then the average distance d between all vertices and their nearest neighbors can be calculated by formula (2) avg (called the distance mean), formula (2) is as follows:

[0182]

[0183] Among them, ‖m i -v i ‖ is vertex m i and the nearest neighbor v i The distance between them, |M| is the size of the matching point set M. In the embodiment of the present application, 0.01 is used as the threshold of the average distance (called the distance threshold). If d avg >0.01, the subdivision level is considered insufficient. In this case, the subdivision level of the 3DMM mesh model needs to be increased until the average distance is less than 0.01.

[0184] When the subdivision level is determined, the matching points between the vertices of the new mesh model and the 3DMM model of the densified vertices are also determined. At this time, the vertices of the template mesh model are replaced with the vertices of the 3DMM mesh model of the densified vertices, so that the vertex index of each target mesh model can record the vertex index and the barycentric parameter of the face of the corresponding 3DMM mesh model. In this way, the 3DMM mesh model of the densified vertices can be used as an intermediate transmission topology to transmit the identity features of the 3DMM mesh model to the topology of the target mesh model (that is, in the topology of the target mesh model, the vertex positions of the target mesh model correspond one-to-one to the vertex positions of the 3DMM mesh model of the densified vertices, and the vertex positions of the 3DMM mesh model of the densified vertices can be associated with the vertex positions of the original 3DMM through the barycentric parameter, so that the transmission of the identity features can be realized. It should be noted that the identity features herein are essentially represented by the positions of the vertices.

[0185] In some embodiments of the present application, the vertex index and the barycentric parameter of the face can also be determined by means of point-to-face projection. At this time, the electronic device will directly project the vertices of the new mesh model onto the grid face of the 3DMM mesh model, then back-calculate the barycentric parameter, and record the grid face and the barycentric parameter of the projection point (referred to as the projection vertex). In this way, the grid subdivision process can be bypassed to speed up the solving speed of the vertex index and the barycentric parameter of the face.

[0186] Next, the application process of the mapping relationship between the 3DMM model topology and the business model topology will be described.

[0187] First, the electronic device can generate a 3D human head model with the identity features of the player based on a single color image. Exemplarily, Figure 12 is a schematic diagram of a 3D human head model provided by an embodiment of the present application. The 3D human head model in image 12-1 is reconstructed based on a 3DMM model.

[0188] Next, the electronic device will use the vertex index and the barycentric parameter of the face of the 3DMM mesh model recorded by the vertex index of the target mesh model (combined as a dense vertex parameter), and the coordinates of the vertices of the 3D human head model to quickly update the vertex coordinates of the target mesh model, and thus obtain a new mesh model with the topology of the target mesh model and the identity features of the player, thereby completing the topology conversion. Exemplarily, Figure 13 is a schematic diagram of a mesh model after completing the topology conversion provided by an embodiment of the present application. The mesh model shown in image 13-1 is a new mesh model with the topology of the target mesh model and the identity features of the player, and based on the mesh model, the 3D game character of the player can be rendered.

[0189] It can be understood that, in the embodiments of the present application, related data such as user information, for example, two-dimensional images, appearance characteristics of players, etc. need to obtain user permission or consent when the embodiments of the present application are applied to specific products or technologies, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions.

[0190] The following continues to illustrate an exemplary structure of the embodiment of the virtual image generation apparatus 255 provided by the present application as a software module. In some embodiments, as shown in FIG. 25, the software module stored in the virtual image generation apparatus 255 of the memory 250 can include: Figure 2

[0191] The model reconstruction module 2551 is configured to reconstruct a first source mesh model of a target object under a source model topology based on a two-dimensional image of the target object.

[0192] The parameter reading module 2552 is configured to read a dense vertex parameter for describing a connection between a face vertex of a template mesh model under a business model topology and a mesh face of the first source mesh model.

[0193] The model updating module 2553 is configured to update vertex information of the face vertex of the template mesh model based on the dense vertex parameter and the vertex information of the face vertex of the first source mesh model, to obtain an updated mesh model under the business model topology.

[0194] The model rendering module 2554 is configured to determine a rendering result of the updated mesh model as a three-dimensional virtual image of the target object.

[0195] In some embodiments of the present application, the dense vertex parameter includes a target index and a barycenter parameter, and the vertex information includes a vertex coordinate; wherein the target index is a vertex index of a mesh face of the first source mesh model corresponding to the face vertex of the template mesh model.

[0196] In some embodiments of the present application, the model updating module 2553 is further configured to determine a target vertex from the face vertex of the first source mesh model according to the target index, and determine a vertex coordinate of the target vertex as a target coordinate; calculate a dense vertex coordinate by using the target coordinate and the barycenter parameter, and update the vertex coordinate of the face vertex of the template mesh model by using the dense vertex coordinate, to obtain the updated mesh model under the business model topology.

[0197] ​In some embodiments of the present application, the parameter reading module 2552 is further configured to read, from vertex indexes of patch vertices of the template mesh model, the dense vertex parameters used to describe the correspondence between the patch vertices of the template mesh model and the mesh patches of the first source mesh model.

[0198] In some embodiments of the present application, the virtual image generation apparatus 255 further comprises a parameter generation module 2555.

[0199] The parameter generation module 2555 is configured to reconstruct a second source mesh model of the template object under the source model topology based on the two-dimensional image of the template object, align the second source mesh model and the template mesh model to obtain an aligned mesh model of the template object under the business model topology, determine target indexes and gravity center parameters based on the aligned mesh model and the second source mesh model, and generate the dense vertex parameters using the target indexes and the gravity center parameters.

[0200] In some embodiments of the present application, the parameter generation module 2555 is further configured to perform vertex densification on the second source mesh model to obtain a dense mesh model, determine matching vertices for patch vertices of the aligned mesh model from patch vertices of the dense mesh model, wherein the patch vertices of the dense mesh model are determined from the interior of mesh patches of the second source mesh model, and the vertex coordinates of the patch vertices of the dense mesh model are calculated from the vertex coordinates of the patch vertices of the second source mesh model and corresponding candidate parameters, determine the target indexes as vertex indexes of mesh patches corresponding to the matching vertices in the second source mesh model, and determine the gravity center parameters as the candidate parameters used to calculate the vertex coordinates of the matching vertices.

[0201] In some embodiments of the present application, the parameter generation module 2555 is further configured to perform the following processing through iteration i, where i is a positive integer: perform the i-th mesh subdivision on the second source mesh model based on the subdivision degree of the i-th iteration to obtain an i-th subdivided mesh model, determine the closest candidate vertex from subdivided vertices of the i-th subdivided mesh model for each patch vertex of the aligned mesh model, calculate a distance mean using the distance between each patch vertex of the aligned mesh model and the corresponding candidate vertex, and determine the dense mesh model as the i-th subdivided mesh model when the distance mean is less than or equal to a distance threshold.

[0202] In some embodiments of the present application, the parameter generation module 2555 is further configured to generate N candidate parameters for the mesh patches of the second source mesh model according to the subdivision level of the i th iteration, where N is a positive integer; determine N subdivision vertices for each mesh patch of the second source mesh model using the vertex coordinates of each mesh patch of the second source mesh model and the N candidate parameters; generate a subdivision patch for each mesh patch using the N subdivision vertices; and form an i th subdivision mesh model using the subdivision patches of each mesh patch.

[0203] In some embodiments of the present application, the parameter generation module 2555 is further configured to increase the subdivision level of the i th iteration to obtain a subdivision level of an (i+1) th iteration when the distance mean is greater than the distance threshold.

[0204] In some embodiments of the present application, the parameter generation module 2555 is further configured to determine a projection vertex from the second source mesh model for the patch vertex of the aligned mesh model; determine a vertex index of a mesh patch to which the projection vertex belongs in the second source mesh model as the target index; and determine the barycenter parameter using the vertex coordinates of the projection vertex and the vertex coordinates of the patch vertex of the mesh patch to which the projection vertex belongs.

[0205] In some embodiments of the present application, the parameter generation module 2555 is further configured to obtain a first coordinate from a first display area of a model display interface and a second coordinate from a second display area of the model display interface, where the first display area and the second display area are respectively configured to display the second source mesh model and the template mesh model; form a vertex pair using the patch vertex corresponding to the first coordinate and the patch vertex corresponding to the second coordinate; and align the second source mesh model and the template mesh model through the vertex pair to obtain the aligned mesh model of the template object under the business model topology.

[0206] The embodiment of the present application provides a computer program product, which includes a computer program or computer executable instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the virtual image generation method provided in the embodiment of the present application.

[0207] The embodiment of the present application provides a computer readable storage medium, wherein computer executable instructions or computer programs are stored, and when the computer executable instructions or computer programs are executed by a processor, the processor executes the virtual image generation method provided by the embodiment of the present application, for example, as shown in the following. Figure 3 The virtual image generation method is shown.

[0208] In some embodiments, the computer readable storage medium can be RAM, ROM, flash memory, magnetic surface memory, optical disc, or CD-ROM memory, etc.; and can also be various devices including one or any combination of the above storage.

[0209] In some embodiments, the computer executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0210] As an example, the computer executable instructions can but not necessarily correspond to files in a file system, can be stored in a part of a file storing other programs or data, for example, stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program in question, or stored in multiple cooperative files (for example, files storing one or more modules, subroutines or code parts).

[0211] As an example, the computer executable instructions can be deployed to be executed on one electronic device, or executed on multiple electronic devices located in one place, or executed on multiple electronic devices distributed in multiple places and interconnected through a communication network.

[0212] In summary, by the embodiments of the present application, the electronic device first uses a two-dimensional image to perform grid model reconstruction on a target object under a source model topology to obtain a first source grid model. Meanwhile, the relationship between the patch vertices of the template grid model and the grid patches of the first source grid model is combined with the dense vertex parameters to determine more appropriate vertex information for the patch vertices of the template grid model from the first source grid model, so as to update the patch vertices of the template grid model. Therefore, not only can the appearance features of the target object be automatically transferred to the template grid model to improve the generation efficiency of the virtual image, but also the appropriate grid patches can be determined for the patch vertices of the template grid model in combination with the dense vertex parameters. The vertex information of the patch vertices of the grid patches is updated based on the vertex information of all the patch vertices of the grid patches, so that the appearance features of the target object can be more completely retained, a three-dimensional image model more consistent with the appearance of the target object is obtained, and the generation effect of the virtual generation is improved.

[0213] The above merely describes the embodiments of the present application, but is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A virtual character generation method characterized by, The method comprises: reconstructing a first source mesh model of a target object under a source model topology based on a two-dimensional image of the target object; reading a dense vertex parameter for describing a connection between a face vertex of a template mesh model under a business model topology and a mesh face of the first source mesh model; updating vertex information of the face vertex of the template mesh model based on the dense vertex parameter and the vertex information of the face vertex of the first source mesh model, to obtain an updated mesh model under the business model topology; determining a rendering result of the updated mesh model as a three-dimensional virtual image of the target object.

2. The method of claim 1, wherein, The dense vertex parameter comprises a target index and a barycenter parameter, and the vertex information comprises a vertex coordinate. The target index is a vertex index of a mesh face of the first source mesh model corresponding to the face vertex of the template mesh model.

3. The method of claim 2, wherein, The updating of the vertex information of the face vertex of the template mesh model based on the dense vertex parameter and the vertex information of the face vertex of the first source mesh model to obtain the updated mesh model under the business model topology comprises: determining a target vertex from the face vertex of the first source mesh model according to the target index, and determining a vertex coordinate of the target vertex as a target coordinate; calculating a dense vertex coordinate by using the target coordinate and the barycenter parameter, and updating the vertex coordinate of the face vertex of the template mesh model by using the dense vertex coordinate, to obtain the updated mesh model under the business model topology.

4. The method according to any one of claims 1 to 3, characterized in that, The reading of the dense vertex parameter for describing the connection between the face vertex of the template mesh model and the mesh face of the first source mesh model comprises: reading the dense vertex parameter for describing the connection between the face vertex of the template mesh model and the mesh face of the first source mesh model from a vertex index of the face vertex of the template mesh model.

5. The method according to any one of claims 1 to 3, characterized in that, The dense vertex parameter is obtained by the following processing: reconstructing a second source mesh model of a template object under the source model topology based on a two-dimensional image of the template object; aligning the second source mesh model and the template mesh model to obtain an aligned mesh model of the template object under the business model topology; determining a target index and a barycenter parameter based on the aligned mesh model and the second source mesh model, and generating the dense vertex parameter by using the target index and the barycenter parameter.

6. The method of claim 5, wherein, The determination of the target index and the barycenter parameter based on the aligned mesh model and the second source mesh model comprises: performing vertex densification on the second source mesh model to obtain a dense mesh model; determining a matching vertex for the face vertex of the aligned mesh model from the face vertex of the dense mesh model; wherein the face vertex of the dense mesh model is determined from the inside of a mesh face of the second source mesh model, and a vertex coordinate of the face vertex of the dense mesh model is calculated by using a vertex coordinate of the face vertex of the second source mesh model and a corresponding candidate parameter. The vertex index of a mesh patch in the second source mesh model corresponding to the matched vertex is determined as the target index, and a candidate parameter used in calculating the vertex coordinate of the matched vertex is determined as the barycenter parameter.

7. The method of claim 6, wherein, The vertex densification is performed on the second source mesh model to obtain a dense mesh model, including: The following processing is performed through iteration i, where i is a positive integer: Performing the i th mesh subdivision on the second source mesh model based on the subdivision degree of the i th iteration to obtain an i th subdivided mesh model; For each patch vertex of the aligned mesh model, a nearest candidate vertex is determined from the subdivided vertices of the i th subdivided mesh model; A distance mean is calculated using the distance between each patch vertex of the aligned mesh model and the corresponding candidate vertex; When the distance mean is less than or equal to a distance threshold, the subdivided mesh model of the i th iteration is determined as the dense mesh model.

8. The method of claim 7, wherein, The i th mesh subdivision is performed on the second source mesh model based on the subdivision degree of the i th iteration to obtain an i th subdivided mesh model, including: According to the subdivision degree of the i th iteration, N candidate parameters are generated for the mesh patches of the second source mesh model, where N is a positive integer; Using the vertex coordinates of each mesh patch of the second source mesh model and the N candidate parameters, N subdivided vertices are determined for each mesh patch of the second source mesh model; Using the N subdivided vertices, a subdivided patch is generated for each mesh patch, and the subdivided patches of each mesh patch are used to form an i th subdivided mesh model.

9. The method of claim 7, wherein, After the distance mean is calculated using the distance between each patch vertex of the aligned mesh model and the corresponding candidate vertex, the method further includes: When the distance mean is greater than the distance threshold, the subdivision degree of the i th iteration is increased to obtain a subdivision degree of an i+1 th iteration.

10. The method of claim 5, wherein, The target index and the barycenter parameter are determined based on the aligned mesh model and the second source mesh model, including: For a patch vertex of the aligned mesh model, a projection vertex is determined from the second source mesh model; The vertex index of a mesh patch in the second source mesh model to which the projection vertex belongs is determined as the target index; Using the vertex coordinate of the projection vertex and the vertex coordinate of the patch vertex of the mesh patch to which the projection vertex belongs, the barycenter parameter is determined.

11. The method of claim 5, wherein, The second source mesh model and the template mesh model are aligned to obtain an aligned mesh model of the template object under the business model topology, including: A first coordinate is obtained from a first display area of a model display interface, and a second coordinate is obtained from a second display area of the model display interface; the first display area and the second display area are respectively used to display the second source mesh model and the template mesh model; A vertex pair is formed using the patch vertex corresponding to the first coordinate and the patch vertex corresponding to the second coordinate; Aligning the second source mesh model and the template mesh model through the vertex pairs to obtain the aligned mesh model of the template object under the business model topology.

12. An avatar generation apparatus characterized by comprising: The device comprises: a model reconstruction module, configured to reconstruct a first source mesh model of a target object under a source model topology based on a two-dimensional image of the target object; a parameter reading module, configured to read dense vertex parameters for describing a connection between vertexes of a mesh patch of a template mesh model under a business model topology and mesh patches of the first source mesh model; a model updating module, configured to update vertex information of vertexes of the template mesh model based on the dense vertex parameters and the vertex information of the vertexes of the mesh patches of the first source mesh model, to obtain an updated mesh model under the business model topology; a model rendering module, configured to determine a rendering result of the updated mesh model as a three-dimensional virtual image of the target object.

13. An electronic device, comprising: The electronic device comprises: a memory, configured to store computer executable instructions or computer programs; a processor, configured to execute the computer executable instructions or computer programs stored in the memory to implement the method in any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, wherein the computer-executable instructions or the computer program comprise the steps of: The computer executable instructions or computer programs are executed by the processor to implement the method in any one of claims 1 to 11. ​ 15. A computer program product comprising computer-executable instructions or a computer program, characterized in that, The computer executable instructions or computer programs are executed by the processor to implement the method in any one of claims 1 to 11. The computer executable instructions or computer programs are executed by the processor to implement the method in any one of claims 1 to 11.

Images