A three-dimensional special effects rendering method, device, storage medium and equipment

Abstract

The present specification provides a three-dimensional special effect rendering method, apparatus, storage medium and device, in which the deformation characteristics of a preset model are adjusted based on the deformation parameters of a target object in a target image to obtain a target model, and the target model is processed accordingly based on a first alignment parameter between the preset model and a reference model used to make special effect materials, so that the target model can be aligned with the material of the target three-dimensional special effect, so that the target three-dimensional special effect can be rendered according to the target model. In this way, without adjusting the special effect material, a new algorithm is implemented to stabilize the algorithm of aligning the special effect material, which is convenient for developers to improve and update.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a three-dimensional special effects rendering method, device, storage medium and equipment. Background Art

[0002] The materials for three-dimensional special effects are usually made by artists through modeling. In software products, the materials for three-dimensional special effects generally need to be made based on a fixed standard three-dimensional model, so that batches of materials can be quickly replaced and used. Software products can render three-dimensional special effects through algorithms. The algorithm generally obtains corresponding parameters based on the standard three-dimensional model to drive the virtual camera in the rendering space to obtain a rendered picture. If the original algorithm of the software product is replaced with a self-developed algorithm based on another standard three-dimensional model, and the self-developed algorithm is directly used to render special effects designed based on the original standard three-dimensional model of the software product, jitter is prone to occur and the rendering effect is poor. In response to this situation, the relevant technology generally requires artists to manually adjust the materials of each three-dimensional special effect to achieve this, which is undoubtedly a very cumbersome and time-consuming process. Summary of the invention

[0003] In a first aspect, a three-dimensional special effects rendering method is provided, comprising: acquiring a target image, adjusting the deformation characteristics of a preset model based on the deformation parameters of a target object in the target image to obtain a target model; acquiring a first alignment parameter between the preset model and a reference model, the preset model and the reference model having the same deformation characteristics; aligning the target model and the reference model based on the first alignment parameter, and rendering a target three-dimensional special effect according to the aligned target model, wherein the material of the target three-dimensional special effect is aligned with the reference model.

[0004] In some embodiments, the target object includes a human face, and the preset model is an average face model obtained based on a plurality of different human face model samples.

[0005] In some embodiments, the deformation parameter includes a first parameter characterizing a facial shape of a target object in the target image and a second parameter characterizing an expression of the target object in the target image.

[0006] In some embodiments, after adjusting the deformation features of the preset model based on the deformation parameters of the target object in the target image, it also includes: obtaining spatial transformation parameters between the target object in the target image and the preset model, the spatial transformation parameters including a third parameter characterizing the scaling ratio between the target object in the target image and the preset model and a fourth parameter characterizing the coordinate correspondence between the target object in the target image and the preset model; using the spatial transformation parameters, adjusting the spatial features of the preset model after the deformation parameters are adjusted so that the obtained target model is aligned with the target image.

[0007] In some embodiments, the target model is a model adjusted by a specified post-processing operation, and the specified post-processing operation is used to change the vertex coordinates of at least one vertex in the point cloud data of the model; the method also includes: adjusting the spatial features of the preset model based on the spatial transformation parameters, and performing the specified post-processing operation to obtain an intermediate model; aligning the intermediate model with the preset model to obtain second alignment parameters; and adjusting the target model using the second alignment parameters before aligning the target model with the reference model based on the first alignment parameters.

[0008] In some embodiments, the target three-dimensional special effects include headgear special effects and / or texture special effects.

[0009] In some embodiments, the target object includes any one of the following: human form, animal form; and the deformation parameters include limb movement parameters.

[0010] In a second aspect, a three-dimensional special effects rendering device is provided, including: an adjustment module, used to obtain a target image, and adjust the deformation characteristics of a preset model based on the deformation parameters of a target object in the target image to obtain a target model; an acquisition module, used to obtain a first alignment parameter between the preset model and a reference model, and the preset model and the reference model have the same deformation characteristics; an alignment module, used to align the target model and the reference model based on the first alignment parameter, and render a target three-dimensional special effect according to the aligned target model, and the material of the target three-dimensional special effect is aligned with the reference model.

[0011] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, any method in the embodiments of the specification is implemented.

[0012] In a fourth aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the methods in the embodiments of the specification is implemented.

[0013] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0014] In the embodiments of this specification, a three-dimensional special effect rendering method, device, storage medium and equipment are disclosed. In this method, the deformation characteristics of a preset model are adjusted based on the deformation parameters of a target object in a target image to obtain a target model, and the target model is processed accordingly based on the first alignment parameter between the preset model and a reference model used to make special effect materials, so that the target model can be aligned with the material of the target three-dimensional special effect, so that the target three-dimensional special effect can be rendered according to the target model. In this way, without adjusting the special effect material, a new algorithm is implemented to stabilize the algorithm of aligning the special effect material, which is convenient for developers to improve and update.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0017] Figure 1 is a schematic diagram of producing a three-dimensional special effect based on a standard three-dimensional model according to an exemplary embodiment of this specification;

[0018] Figure 2A is a flowchart of a 3D special effects rendering method according to an exemplary embodiment of the present specification;

[0019] Figure 2B is a schematic diagram of a rendering process of a headgear special effect according to an exemplary embodiment of this specification;

[0020] Figure 2C is a schematic diagram of a rendering process of a texture special effect according to an exemplary embodiment of this specification;

[0021] Figure 3A is a schematic diagram of a base model A and an alignment point in the topology of the base model A according to an exemplary embodiment of this specification;

[0022] Figure 3B is a schematic diagram of a base model B and an alignment point in the topology of the base model B according to an exemplary embodiment of this specification;

[0023] Figure 4 This is a hardware structure diagram of a computer device where a three-dimensional special effects rendering device is located according to an exemplary embodiment of this specification;

[0024] Figure 5It is a block diagram of a 3D special effects rendering device according to an exemplary embodiment of the present specification. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0026] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0027] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0028] The materials of 3D special effects, that is, the elements added to the rendering space when rendering 3D special effects, are usually made by artists through modeling. In software products, the materials of 3D special effects generally need to be made based on a fixed standard 3D model, so that batches of materials can be quickly replaced and used. Figure 1 As shown, Figure 1It is a schematic diagram of making three-dimensional special effects based on a standard three-dimensional model, wherein the first model 11 is the standard three-dimensional model used in the algorithm of the software product, and the second model 12 is the material of the three-dimensional special effects. Since the three-dimensional special effects are rendered using the rotation matrix and offset matrix of the standard three-dimensional model during actual rendering, the second model 12 is aligned with the first model 11, so that the second model 12 can fit the face in the first model 11, thereby achieving a better rendering effect. If the original algorithm of the software product is replaced with a self-developed algorithm based on another standard three-dimensional model to achieve special effects rendering, and the self-developed algorithm is directly used to render special effects designed based on the original standard three-dimensional model of the software product, jitter is likely to occur and the rendering effect is poor. In response to this situation, the relevant technology generally requires artists to manually adjust the materials of each three-dimensional special effect to achieve this, and this process is undoubtedly very cumbersome and time-consuming.

[0029] Next, the embodiments of this specification are described in detail.

[0030] like Figure 2A As shown, Figure 2A This is a flowchart of a 3D special effects rendering method according to an exemplary embodiment of the present specification, the method comprising:

[0031] Step 201: acquiring a target image, and adjusting the deformation characteristics of a preset model based on the deformation parameters of a target object in the target image to obtain a target model;

[0032] The method of this embodiment can be applied to multimedia applications, such as short video creation tools, video live broadcast software, etc. The multimedia application can be installed on portable terminals such as smart phones and tablet computers, and can also be installed on terminals such as desktop computers, which is not limited in this embodiment. The target image mentioned in this step can be an image obtained by the terminal in real time through a camera, or it can be an image uploaded to the terminal by a user. The target image can be one frame of a video image sequence, and the terminal can render each frame or part of the frames in the video image sequence.

[0033] The target object mentioned in this step can be a real object or a virtual object. The virtual object can be produced by professional technicians through computer graphics technology, such as digital people, cartoon animals, etc. It should be noted that the target object can be in human form or animal form, or a local part of a person or a local part of an animal, such as a human face, an animal hand, etc. The preset model mentioned in this step can be considered as a three-dimensional model of the target object. For example, if the target object is a human face, the preset model can be a three-dimensional human face model; or, for example, if the target object is in human form, the preset model can be a three-dimensional human body model.

[0034] In this embodiment, the preset model is a standard three-dimensional model to be used for rendering the target three-dimensional special effects. Therefore, in order to facilitate the universal replacement of various three-dimensional special effects materials, the preset model can be a three-dimensional model with standard deformation features, and the deformation features can be features that describe identity and emotions. For example, when the preset model is a three-dimensional face model, the deformation features can include face shape and / or expression, and the standard deformation features can be considered as standard face shape and / or standard expression, that is, preset fat and thin state and / or preset expression. In this case, the preset model can be an average face model obtained based on multiple different face model samples. Specifically, different face model samples can be sample matrices of specified dimensions obtained by modeling different face images, and the average face model can be obtained by averaging each dimensional feature of each face model sample. In an optional embodiment, the average face model may be a model constructed by a PCA (Principal Component Analysis) algorithm, the average face model provides a set of PCA bases of face shape and expression, and the positions of the vertices of the average face model may be changed by changing deformation parameters (equivalent to the weights of the PCA bases), thereby changing the face shape and / or expression of the average face model. For another example, when the preset model is a three-dimensional human body model, the deformation feature may include body movements. Accordingly, in this case, the preset model may be an average human body model obtained based on a plurality of different three-dimensional human body model samples.

[0035] In some examples, the deformation parameters may include a first parameter characterizing the facial shape of the target object in the target image and a second parameter characterizing the expression of the target object in the target image. The first parameter and the second parameter may be obtained by parsing the target image. Optionally, the terminal may use ARKit as a parsing tool. ARKit is an SDK (Software Development Kit) for developing AR applications. With the help of the facial tracking function provided by ARKit, deformation parameters for characterizing the facial shape and expression may be extracted from the target image. Of course, in other embodiments, the first parameter and the second parameter may also be obtained in other ways, for example, automatically identified by a trained neural network model, which is not limited in this specification.

[0036] After obtaining the deformation parameters, the deformation features of the preset model can be adjusted by the deformation parameters to obtain a target model, so that the target model has the same or similar deformation features as the target object in the target image. For example, if the target object in the target image is a slightly fat face with a laughing expression, the coordinates of the vertices of the corresponding facial contour, mouth and other positions in the preset model can be adjusted by the deformation parameters, so that the adjusted target model also has the deformation features of being slightly fat and laughing. In this way, the target model can be applied to a variety of three-dimensional special effects, including headgear special effects, texture special effects, etc. Headgear special effects include sunglasses special effects, animal ear special effects, etc. Figure 2B As shown, when the software product renders the headgear special effects through the algorithm, the headgear special effects material 21 can be added to the rendering space, the point cloud topology of the headgear special effects material 21 is aligned with the point cloud topology 22 of the standard three-dimensional model, and then the virtual camera 23 in the rendering space is driven in real time according to the rotation offset matrix of the 3D face predicted from the picture, so as to obtain the target picture 24 showing the headgear special effects; texture special effects include mask special effects, animal skin special effects, etc. Figure 2C As shown, when the software product renders texture special effects through an algorithm, the material 25 of the texture special effects can be added to the rendering space, and the standard three-dimensional model can be adjusted according to the facial expression in the picture, that is, the mesh is reconstructed, and the texture topology of the material 25 of the texture special effects is aligned with the texture topology 26 of the standard three-dimensional model, and then the virtual camera 27 in the rendering space is driven in real time according to the rotation offset matrix of the 3D face predicted from the picture, so that the target picture 28 showing the texture special effect can be obtained. The algorithm output used by the texture special effect is not only the rotation offset matrix, but also the reconstructed mesh. The difference between the texture special effect and the headgear special effect is that the texture special effect is veneer, and it is synchronized with the face shape and expression, so the texture special effect also involves the alignment problem of the reconstructed mesh. This step can achieve alignment with the target image by constructing a target model, which is not only applicable to headgear special effects, but also to other three-dimensional special effects such as texture special effects.

[0037] Considering that the coordinate space corresponding to the target image and the coordinate space corresponding to the preset model may be misaligned, for example, the size of the target image and the size of the preset model are of different orders of magnitude, therefore, in some examples, after adjusting the deformation features of the preset model based on the deformation parameters of the target object in the target image, it may also include: obtaining the spatial transformation parameters between the target object in the target image and the preset model, the spatial transformation parameters including a third parameter representing the scaling ratio between the target object in the target image and the preset model and / or a fourth parameter representing the coordinate correspondence between the target object in the target image and the preset model; using the spatial transformation parameters, adjusting the spatial features of the preset model adjusted based on the deformation parameters so that the obtained target model is aligned with the target image. The third parameter represents the scaling ratio between the target object in the target image and the preset model, and is used to scale the preset model adjusted based on the deformation parameters, that is, the reconstructed mesh, to align with the scale of the target image; the fourth parameter represents the coordinate correspondence between the target object in the target image and the preset model, and is used to transform the reconstructed mesh to the coordinate system where the target image is located. In this way, the target model can be aligned with the target image by aligning the points in the target model with the points of the face in the target image.

[0038] In an optional embodiment, the target model can be constructed based on the following formula:

[0039]

[0040] In the above formula, X projection is the feature matrix corresponding to the target model; is the feature matrix corresponding to the preset model, which is the average face model generated by the PCA algorithm; α i It is the PCA basis corresponding to the face shape constructed by PCA modeling. Its dimension can be the product of the shape dimension, the number of vertices of the point cloud corresponding to the preset model, and the geometric coordinate dimension. For example, if the shape dimension is 199, the number of vertices of the point cloud is 65536, and the geometric coordinate dimension is XYZ, then α i The dimension is 199*65536*3;s i is the shape coefficient, i.e. the first parameter mentioned above; m is the shape dimension; β i is the PCA basis of the corresponding expression constructed by PCA modeling. Similarly, its dimension can be the product of the expression dimension, the number of vertices of the point cloud corresponding to the preset model, and the geometric coordinate dimension; e i is the expression coefficient, i.e. the second parameter mentioned above; n is the expression dimension, optional, n can be 29; s is the scaling factor, i.e. the third parameter mentioned above; R and t 2dThey are the rotation matrix and the offset matrix, which are the fourth parameter mentioned above; P is the orthogonal projection matrix, optionally, P can be a 3*3 unit matrix.

[0041] Based on the above formula, a target model can be constructed that has the same shape and expression as the face in the target image and is aligned with the target image.

[0042] Step 202: obtaining a first alignment parameter between the preset model and the reference model, wherein the preset model and the reference model have the same deformation characteristics;

[0043] The reference model mentioned in this step is aligned with the material of the target 3D special effect, that is, the material of the target 3D special effect can be produced based on the reference model. The reference model can be a standard 3D model used in the original algorithm, and correspondingly, the preset model can be a standard 3D model used in a new algorithm that replaces the original algorithm. Therefore, the reference model can also be a 3D model with standard deformation characteristics, that is, the preset model has the same deformation characteristics as the reference model.

[0044] The preset model and the reference model may be constructed based on different sample data, so the topological structures of the two models, including the layout of points, lines and surfaces, connection conditions, etc., may be different. The target 3D special effect material is made based on the reference model, so the target model can be aligned with the target 3D special effect material using the first alignment parameter between the preset model and the reference model. Considering that the number of vertices in the point cloud of the preset model and the reference model may be different, in some examples, when obtaining the first alignment parameter, the preset alignment point of the preset model may be aligned with the preset alignment point of the reference model. The preset alignment point may be a vertex with specified semantics in the point cloud, such as an eye corner point, a nose tip point, a chin point, a face contour point, etc. The preset alignment points of the preset model form a first point set, and the preset alignment points of the reference model form a second point set. The number of points in the first point set and the second point set is consistent, and the points in the first point set correspond to the points in the second point set one by one, and the corresponding points represent the same semantics. In this way, by rotating and translating the vector formed by the points in the first point set, it is aligned with the vector formed by the points in the second point set as much as possible, that is, the target rotation matrix and the target translation matrix are obtained, so that the sum of the distances between all points in the first point set and the points in the corresponding positions in the second point set after the change is minimized. In other words, the first alignment parameter between the preset model and the reference model includes the target rotation matrix and the target translation matrix. Considering that the coordinate system corresponding to the preset model and the coordinate system corresponding to the reference model may have different orders of magnitude, for example, the vertex coordinates of the preset model are approximately in the order of 50000, and the vertex coordinates of the reference model are approximately in the order of 0.001, therefore, the first alignment parameter may also include a target scaling parameter, using which the order of magnitude of the coordinate system corresponding to the preset model may be aligned to the order of magnitude of the coordinate system corresponding to the reference model. In some examples, the first alignment parameter between the preset model and the reference model may be calculated using an SVD (Singular Value Decomposition) alignment algorithm, which may simplify the function to be optimized, thereby improving the accuracy and efficiency of the algorithm. In other examples, other algorithms, such as an ICP (Iterative Closest Point) algorithm, may also be used to obtain the first alignment parameter, which is not limited in this specification.

[0045] It should be noted that, since the first alignment parameter between the preset model and the target model is fixed, the first alignment parameter can be stored after being calculated and can be directly retrieved from the storage location for subsequent use, thereby reducing the processing workload of the terminal.

[0046] Step 203: align the target model with the reference model based on the first alignment parameter, and render a target three-dimensional special effect according to the aligned target model, wherein the material of the target three-dimensional special effect is aligned with the reference model.

[0047] After obtaining the first alignment parameter, the target model and the reference model can be aligned based on the first alignment parameter, that is, the target model is processed accordingly using the first alignment parameter so that the processed target model and the reference model are aligned. In this way, the aligned target model mentioned in this step can stably align the target three-dimensional special effect material. In this case, there is no need to adjust the target three-dimensional special effect material, and the rendering picture of the target three-dimensional special effect can be output through the aligned target model, and the rendering effect is highly stable. The rendering picture can be combined with the target image to realize the fusion of virtual three-dimensional special effects and the real environment, providing users with a sensory experience beyond reality.

[0048] The method of the embodiment of the present specification adjusts the deformation characteristics of the preset model based on the deformation parameters of the target object in the target image to obtain the target model, and processes the target model accordingly based on the first alignment parameter between the preset model and the reference model used to make the special effect material, so that the target model can be aligned with the target three-dimensional special effect material, so that the target three-dimensional special effect can be rendered according to the target model. In this way, without adjusting the special effect material, the new algorithm is implemented to stabilize the algorithm of aligning the special effect material, which is convenient for developers to improve and update.

[0049] Furthermore, in practical applications, the rendering of three-dimensional special effects often involves more complex business. For example, the target model of some scenes is a face model. The vertices corresponding to the eyes in the point cloud of the target model cannot form a "closed eyes" effect. In order to meet business needs, the vertex positions can be adjusted through additional processing to produce a "closed eyes" effect. These processing operations for changing the position of at least one vertex in the point cloud data of the model can be called designated post-processing operations. When the target model that performs the designated post-processing operation is aligned to the reference model, the alignment is prone to jitter. Based on this, an optional embodiment of the present specification provides the following solution: if the target model is a model adjusted by a designated post-processing operation, the spatial features of the preset model are adjusted based on the aforementioned spatial transformation parameters, and the designated post-processing operation is performed to obtain an intermediate model; the intermediate model is aligned with the preset model to obtain a second alignment parameter; before the target model is aligned with the reference model based on the first alignment parameter, the target model is adjusted using the second alignment parameter. The intermediate model here is constructed by decoupling the target model and removing the deformation features adjusted by the deformation parameters. When the target object includes a face, the intermediate model can be constructed based on the following formula:

[0050]

[0051] In the above formula, X centre is the feature matrix corresponding to the intermediate model, and the meanings of the remaining parameters are consistent with those in the formula of the target model, which will not be repeated here. It can be seen from the above formula that the intermediate model removes the facial shape and expression of the target object in the target image compared to the target model. The above scheme means that the intermediate model is first subjected to the same specified post-processing operation as the target model, and then the intermediate model is aligned with the preset model, and the first alignment parameter obtained by the alignment is retained. The target model is processed accordingly with this parameter, and the processed target model and the preset model can be aligned. Finally, the purpose of aligning the reference model is achieved according to the first alignment parameter between the preset model and the reference model. In this way, by constructing the intermediate model, the deviation caused by the deformation features adjusted by the deformation parameters in the process of aligning the target model and the reference model that perform the specified post-processing operation is removed, so it can be applied to more complex businesses.

[0052] In order to explain the solution of this specification in more detail, a specific embodiment is introduced below:

[0053] The solution of this embodiment is applied to an SDK embedded in a short video creation tool. The SDK uses a new algorithm different from the original algorithm of the short video creation tool. The standard three-dimensional model used by the original algorithm of the short video creation tool is recorded as base model A. Base model A and the alignment points in the topology of base model A are as follows: Figure 3A As shown, the alignment points 301 and 302 are eye corner points, the alignment point 303 is the nose tip point, the alignment point 304 is the chin point, and the alignment points 305 and 306 are face contour points. The standard three-dimensional model used by the new algorithm is denoted as base model B. The base model B and the alignment points in the topology of the base model B are as follows: Figure 3B As shown, correspondingly, alignment point 307 and alignment point 308 are eye corner points, alignment point 309 is nose tip point, alignment point 310 is chin point, alignment point 311 and alignment point 312 are face contour points. When aligning base model A and base model B, alignment can be performed based on alignment points with the same semantics. The 3D special effects material provided in the short video creation tool is based on base model A. The SDK renders the original 3D special effects based on the following method:

[0054] S301, obtaining a video image sequence of the user in real time through a camera, and obtaining deformation parameters and spatial transformation parameters for each frame of the image by analyzing the image, wherein the deformation parameters include face shape parameters s i and expression parameter e i The spatial transformation parameters include scaling parameters s, rotation transformation parameters R and translation transformation parameters t 2d ;

[0055] S302, adjusting the base model B using the acquired deformation parameters and space transformation parameters to obtain a base model B1. The construction formula of the base model B1 is as follows, where: is the characteristic matrix corresponding to the base module B:

[0056]

[0057] S303, based on the base model B1, performing a specified post-processing operation on the base model B1;

[0058] S304, adjusting the base model B using the acquired spatial transformation parameters to obtain a base model B2, the construction formula of the base model B2 is as follows:

[0059]

[0060] S305, based on the base model B2, performing a specified post-processing operation on the base model B2;

[0061] S306, based on Figure 3A , performing SVD alignment on the base model B2 and the base model B according to the alignment point selected in , retaining the first alignment parameter, which is a rotation and translation matrix;

[0062] S307, using the first alignment parameter to process the base model B1 after the specified post-processing operation is performed, so that the processed base model B1 is aligned with the base model B;

[0063] S308, obtaining a pre-calculated second alignment parameter to re-process the processed base model B1, so that the re-processed base model B1 can be aligned with the base model A, the second alignment parameter is an alignment parameter between the base model B and the base model A, and the second alignment parameter includes a rotation and translation matrix and a scaling transformation parameter;

[0064] S309, using the reprocessed base model B1 to render a three-dimensional special effect.

[0065] Through the above process, the 3D special effects materials that were originally not aligned with the standard 3D model topology used by the new algorithm can be output based on the new algorithm for rendering and display. In this way, the new algorithm can stabilize the algorithm of real-time alignment of materials, and there is no need for developers to re-adjust the 3D special effects materials, reducing the workload of developers. It should be noted that the above solution can be applied not only to 3D faces, but also to other models based on PCA-based topological structures, such as simplemodel (human body model), etc. When targeting a simple model, the deformation parameters can include limb-related PCA parameters.

[0066] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a three-dimensional special effects rendering device and a terminal to which it is applied.

[0067] The embodiments of the 3D special effects rendering device of this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the file processing in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 4 The figure is a hardware structure diagram of the computer device where the three-dimensional special effects rendering device of the embodiment of this specification is located. Figure 4 In addition to the processor 410, memory 430, network interface 420, and non-volatile memory 440 shown, the server or electronic device where the device 431 is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0068] Accordingly, an embodiment of the present specification further provides a computer storage medium, in which a program is stored. When the program is executed by a processor, the method in any of the above embodiments is implemented.

[0069] The embodiments of this specification may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0070] like Figure 5 As shown, Figure 5 This is a block diagram of a 3D special effects rendering device according to an exemplary embodiment of the present specification, the device comprising:

[0071] An adjustment module 51 is used to obtain a target image, and adjust the deformation characteristics of a preset model based on the deformation parameters of a target object in the target image to obtain a target model;

[0072] An acquisition module 52, used to acquire a first alignment parameter between the preset model and the reference model, wherein the preset model and the reference model have the same deformation characteristics;

[0073] The alignment module 53 is used to align the target model with the reference model based on the first alignment parameter, and render a target three-dimensional special effect according to the aligned target model, wherein the material of the target three-dimensional special effect is aligned with the reference model.

[0074] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0075] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0076] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0078] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0079] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A three-dimensional special effects rendering method, characterized in that: include: Acquire a target image, and adjust the deformation characteristics of a preset model based on the deformation parameters of the target object in the target image to obtain a target model; Acquiring a first alignment parameter between the preset model and the reference model, wherein the preset model and the reference model have the same deformation characteristics; The target model is aligned with the reference model based on the first alignment parameter, and a target three-dimensional special effect is rendered according to the aligned target model, wherein the material of the target three-dimensional special effect is aligned with the reference model.

2. The method according to claim 1, characterized in that The target object includes a human face, and the preset model is an average face model obtained based on a plurality of different human face model samples.

3. The method according to claim 2, characterized in that The deformation parameters include a first parameter representing a facial shape of a target object in the target image and a second parameter representing an expression of the target object in the target image.

4. The method according to claim 1, characterized in that After the deformation feature of the preset model is adjusted based on the deformation parameter of the target object in the target image, it also includes: Acquire spatial transformation parameters between the target object in the target image and the preset model, wherein the spatial transformation parameters include a third parameter representing a scaling ratio between the target object in the target image and the preset model and a fourth parameter representing a coordinate correspondence relationship between the target object in the target image and the preset model; The spatial transformation parameters are used to adjust the spatial features of the preset model adjusted based on the deformation parameters, so that the obtained target model is aligned with the target image.

5. The method according to claim 4, characterized in that The target model is a model adjusted by a specified post-processing operation, wherein the specified post-processing operation is used to change the vertex coordinates of at least one vertex in the point cloud data of the model; the method further includes: Adjusting the spatial features of the preset model based on the spatial transformation parameters, and performing the specified post-processing operation to obtain an intermediate model; Aligning the intermediate model with the preset model to obtain a second alignment parameter; Before aligning the target model with the reference model based on the first alignment parameter, the target model is adjusted using the second alignment parameter.

6. The method according to claim 1, characterized in that The target three-dimensional special effects include headgear special effects and / or texture special effects.

7. The method according to claim 1, characterized in that The target object includes any one of the following: human form, animal form; the deformation parameters include limb movement parameters.

8. A 3D special effects rendering device, characterized in that: include: An adjustment module, used to obtain a target image, and adjust the deformation characteristics of a preset model based on the deformation parameters of a target object in the target image to obtain a target model; An acquisition module, used for acquiring a first alignment parameter between the preset model and the reference model, wherein the preset model and the reference model have the same deformation characteristics; An alignment module is used to align the target model with the reference model based on the first alignment parameter, and render a target three-dimensional special effect according to the aligned target model, wherein the material of the target three-dimensional special effect is aligned with the reference model.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images