A method for determining an object model and a related device
By vertex expansion, posture alignment and offset parameters of the scanned three-dimensional non-parametric model, automatic conversion into a parameterized model is achieved, solving the application problem of three-dimensional model in product scenarios, and improving the efficiency of model generation and Accuracy.
Patent Information
- Application Number
- CN202111675879.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-31
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2041-12-31
AI Technical Summary
In the prior art, the scanned three-dimensional model does not have driving parameters, is difficult to apply to product scenarios, and is time-consuming and labor-intensive to convert it into a parameterized model through manual methods and is uncontrollable in accuracy.
By obtaining the non-parametric model, model vertex expansion, pose alignment and offset parameters are determined, and automatically converted into a parameterized model with driving parameters to avoid human experience influence.
The automatic conversion of high-precision parameterized models is realized, the generation efficiency and accuracy are improved, and the application value of the model in product scenarios is ensured.
Smart Images
Figure CN114332377B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to a method for determining an object model and a related device. Background Art
[0002] Scanning a 3D object using lasers and large-scale camera arrays can produce a high-precision 3D model that can realistically display the shape and texture details of the object. However, the scanned 3D model does not have driving parameters, that is, the 3D model cannot be controlled by processing logic, and is a non-parametric model. This makes it difficult to apply such non-parametric models to various product scenarios (such as game interaction scenarios, film and television content scenarios).
[0003] In the related art, the non-parametric model obtained by the above scanning is manually processed by artists and model designers through model adjustment, binding, skinning, and material assignment to obtain a drivable 3D model, that is, a parametric model with driving parameters. Only then can it be applied to product scenarios.
[0004] Manually converting a non-parametric model into a parametric model is not only time-consuming and labor-intensive, but also highly dependent on human experience, resulting in uncontrollable accuracy of the converted parametric model. Summary of the invention
[0005] In order to solve the above technical problems, the present application provides a method for determining an object model, which realizes the automatic conversion of the high-precision non-parametric model obtained by scanning into a parametric model, avoids the influence of human experience, and improves the generation efficiency and accuracy of the high-precision parametric model.
[0006] The embodiments of the present application disclose the following technical solutions:
[0007] In one aspect, an embodiment of the present application provides a method for determining an object model, the method comprising:
[0008] Acquire a non-parametric model obtained by scanning a target three-dimensional object;
[0009] Expanding the model vertices of the initial parameterized model according to the first model vertices number of the non-parametric model to obtain a registered three-dimensional model with a second model vertices number;
[0010] Performing posture alignment of the registered three-dimensional model with the non-parametric model to obtain an aligned three-dimensional model;
[0011] Determining model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model;
[0012] The aligned three-dimensional model is adjusted to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameters.
[0013] On the other hand, an embodiment of the present application provides a device for determining an object model, the device comprising an acquisition unit, an expansion unit, an alignment unit, a determination unit, and an adjustment unit:
[0014] The acquisition unit is used to acquire a non-parametric model obtained by scanning the target three-dimensional object;
[0015] The expansion unit is used to expand the model vertices of the initial parameterized model according to the first model vertices number of the non-parametric model to obtain a registered three-dimensional model with a second model vertices number;
[0016] The alignment unit is used to perform posture alignment of the registered three-dimensional model with the non-parametric model to obtain an aligned three-dimensional model;
[0017] The determining unit is used to determine the model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model;
[0018] The adjustment unit is used to adjust the aligned three-dimensional model to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameters.
[0019] In a possible implementation manner, the alignment unit is further configured to:
[0020] Based on the same key point semantics, respectively annotating a target number of model key points on the registered three-dimensional model and the non-parametric model;
[0021] The registered three-dimensional model is posture-aligned with the non-parametric model according to the model key points to obtain the aligned three-dimensional model.
[0022] In a possible implementation manner, the alignment unit is further configured to:
[0023] Calculate, according to the model key points of the registered three-dimensional model and the non-parametric model, an alignment parameter of the registered three-dimensional model relative to the non-parametric model, wherein the alignment parameter includes at least one of rotation information, translation information, scaling information, and posture information;
[0024] The registered three-dimensional model is posture-aligned with the non-parametric model based on the alignment parameters to obtain the aligned three-dimensional model.
[0025] In a possible implementation manner, the determining unit is further configured to:
[0026] According to the model vertices in the aligned three-dimensional model, vertex matching is performed in the non-parametric model to obtain a model vertex pair, wherein the model vertex pair includes a first model vertex in the aligned three-dimensional model and a second model vertex in the non-parametric model, and the second model vertex is a model vertex in the non-parametric model that is closest to the first model vertex;
[0027] Model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model are determined according to the model vertex pairs.
[0028] In a possible implementation manner, the determining unit is further configured to:
[0029] Determining a key point pair in the model vertex object, a first model vertex in the key point pair being the model key point in the aligned three-dimensional model;
[0030] The model vertex offset parameters are determined according to the key point offsets determined by the key point pairs and the vertex offsets determined by the model vertex pairs.
[0031] In a possible implementation, the expansion unit is further used for:
[0032] Determining the number of to-be-expanded model vertices for the initial parameterized model according to the number of initial model vertices of the initial parameterized model and the number of the first model vertices;
[0033] Based on the normal interpolation parameters, the number of expanded model vertices to be expanded is added to the initial parameterized model through model vertex interpolation to obtain the registered three-dimensional model, and the normal interpolation parameters are used to identify the degree of concavity of the vertex positions of the expanded model vertices in the initial parameterized model relative to the non-parametric model.
[0034] In a possible implementation, the device further includes a generation unit and a migration unit:
[0035] The generating unit is used to generate a corresponding texture image according to the non-parametric model;
[0036] The migration unit is used to migrate the texture image to the target parameterized model to obtain the target parameterized model having the texture image.
[0037] In a possible implementation manner, the device further includes a storage unit:
[0038] The saving unit is used to save the target parameterized model based on a tree structure to convert it into a model format file of the target three-dimensional object, where the model format file is suitable for a game engine.
[0039] In another aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:
[0040] The memory is used to store program code and transmit the program code to the processor;
[0041] The processor is used to determine the object model according to the instructions in the program code.
[0042] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method for determining the object model described in the above aspect.
[0043] It can be seen from the above technical solution that after scanning the target three-dimensional object to obtain a non-parametric model, an initial parametric model with the same object type as the target three-dimensional object is determined, and the initial parametric model with driving parameters is used to expand the model vertices to obtain a registered three-dimensional model with a number of model vertices close to that of the non-parametric model. The registered three-dimensional model is posture-aligned with the non-parametric model to obtain an aligned three-dimensional model, so that the aligned three-dimensional model and the non-parametric model are in the same posture, so that the model vertex offset parameters that identify the aligned three-dimensional model relative to the non-parametric model can be determined, and the target parametric model of the target three-dimensional object is obtained by adjusting the aligned three-dimensional model based on the model vertex offset parameters. The target parametric model not only has an appearance accuracy close to that of the non-parametric model, but also has driving parameters that can be used for driving, thereby realizing the automatic conversion of the high-precision non-parametric model obtained by scanning to a parametric model, avoiding the influence of human experience, and improving the generation efficiency and accuracy of the high-precision parametric model. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 A method flow chart of a method for determining an object model provided in an embodiment of the present application;
[0046] Figure 2 A schematic diagram of model vertex interpolation based on normal direction provided in an embodiment of the present application;
[0047] Figure 3A schematic diagram of marking key points of a model on a registered three-dimensional model and a non-parametric model provided in an embodiment of the present application;
[0048] Figure 4 A schematic diagram of aligning a registered three-dimensional model with a non-parametric model provided in an embodiment of the present application;
[0049] Figure 5 A schematic diagram of adjusting an aligned three-dimensional model to a non-parametric model provided in an embodiment of the present application;
[0050] Figure 6 A schematic diagram of migrating a texture image to a target parameterized model provided in an embodiment of the present application;
[0051] Figure 7 A schematic diagram of converting a non-parametric model of a target human body into an FBX file provided in an embodiment of the present application;
[0052] Figure 8 A device structure diagram of a device for determining an object model provided in an embodiment of the present application;
[0053] Fig. 9 A structural diagram of a terminal device provided in an embodiment of the present application;
[0054] Fig.10 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0055] The embodiments of the present application are described below in conjunction with the accompanying drawings.
[0056] Based on the high-precision three-dimensional model obtained by scanning, in order to apply it to various product scenarios, the relevant technology mainly uses manual methods to adjust and set the non-parametric model before obtaining the corresponding parametric model. The accuracy of the parametric model and the degree of similarity with the non-parametric model are directly related to human experience, which is not only inefficient but also has large fluctuations in accuracy.
[0057] To this end, an embodiment of the present application provides a method for determining an object model, which realizes the automatic conversion of a high-precision non-parametric model obtained by scanning into a parametric model, avoids the influence of human experience, and improves the generation efficiency and accuracy of the high-precision parametric model.
[0058] The object model determination method provided in the embodiment of the present application can be implemented by a computer device, which can be a terminal device or a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
[0059] Figure 1 A method flow chart of a method for determining an object model provided in an embodiment of the present application, the method comprising:
[0060] S101: Acquire a non-parametric model obtained by scanning a target three-dimensional object.
[0061] The non-parametric model is a three-dimensional model without driving parameters, that is, it is unbound and cannot be driven by the driving engine in terms of behavior and action. However, since the non-parametric model is obtained based on scanning the target three-dimensional object, its model shape accuracy is very close to the target three-dimensional object and can realistically show the actual shape and texture details of the target three-dimensional object. If a model with driving parameters can be obtained based on the non-parametric model, it can have a high application value.
[0062] The present application does not limit the type of the target three-dimensional object, which may be various items, people, etc. For example, the target three-dimensional object may be a specific human body, animal, vehicle, mobile phone, etc.
[0063] The present application also does not limit the scanning method for the target three-dimensional object. For example, scanning may be achieved using lasers and large-scale camera arrays.
[0064] S102: Expanding the model vertices of the initial parameterized model according to the first number of model vertices of the non-parametric model to obtain a registered three-dimensional model with a second number of model vertices.
[0065] Since the non-parametric model has a large number of model vertices, the processing flow of directly setting the driving parameters in the non-parametric model will be very complicated. For this reason, the embodiment of the present application adopts a method of obtaining the target parametric model of the target three-dimensional object through an initial parametric model that already has driving parameters.
[0066] To this end, first determine the initial parametric model of the same object type according to the object type of the target three-dimensional object. For example, when the target three-dimensional object is a male body, the initial parametric model is also a three-dimensional model corresponding to the male body. The initial parametric model determined in this way is similar in appearance to the non-parametric model, which helps to fine-tune it into a parametric model with an appearance close to the target three-dimensional object.
[0067] Since the number of model vertices of the initial parametric model is generally much smaller than the number of model vertices of the non-parametric model (i.e., the first number of model vertices), taking the target three-dimensional object as the human body as an example, the non-parametric model has a very high precision and its model vertices are generally hundreds of thousands, while the number of model vertices of the initial parametric model is generally only more than 10,000. Taking the SMPL-X model (a parametric human body model) as an example, the model only includes 10,475 vertices and 20,000 model faces.
[0068] Therefore, in order to make the target parametric model obtained later closer to the non-parametric model or closer to the actual shape of the target three-dimensional object, it is necessary to expand the number of model vertices of the initial parametric model so that the difference between the number of model vertices of the obtained registered three-dimensional model (i.e., the number of second model vertices) and the number of first model vertices of the non-parametric model is small, so as to obtain a registered three-dimensional model with a model accuracy close to that of the non-parametric model according to the initial parametric model. That is, the difference between the number of first model vertices and the number of second model vertices is less than a threshold value, which can be zero (in this case, the number of first model vertices is equal to the number of second model vertices) or a smaller integer.
[0069] S103: Performing posture alignment on the registered three-dimensional model and the non-parametric model to obtain an aligned three-dimensional model.
[0070] S104: Determine model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model.
[0071] Although the obtained registered 3D model is closer to the non-parametric model in terms of the number of model vertices, it is still necessary to adjust the model appearance of the registered 3D model based on the non-parametric model so that this model with driving parameters can reflect the actual shape of the target 3D object or achieve the actual shape that the non-parametric model can reflect.
[0072] To this end, the posture of the registered 3D model needs to be adjusted to achieve alignment with the posture of the non-parametric model, so that the aligned 3D model and the non-parametric model obtained by adjustment are in the same posture. In this case, the model vertex offset can be obtained more accurately by comparing the aligned 3D model and the non-parametric model. The model vertex offset is determined for the vertices between the aligned 3D model and the non-parametric model, and can reflect the differences in position and concavity between the model vertices of the two models.
[0073] S105: adjusting the aligned three-dimensional model to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameters.
[0074] By adjusting the aligned 3D model based on the model vertex offset parameters, the model shape of the aligned 3D model becomes closer and closer to the non-parametric model during the adjustment process, so that the final target parametric model can restore the actual shape of the target 3D object as much as possible and has driving parameters for model driving.
[0075] It can be seen that after scanning the target three-dimensional object to obtain a non-parametric model, an initial parametric model with the same object type as the target three-dimensional object is determined, and the initial parametric model with driving parameters is used to expand the model vertices to obtain a registered three-dimensional model with a number of model vertices close to that of the non-parametric model. The registered three-dimensional model is posture-aligned with the non-parametric model to obtain an aligned three-dimensional model, so that the aligned three-dimensional model and the non-parametric model are in the same posture, so that the model vertex offset parameters that identify the aligned three-dimensional model relative to the non-parametric model can be determined, and the target parametric model of the target three-dimensional object is obtained by adjusting the aligned three-dimensional model based on the model vertex offset parameters. The target parametric model not only has an appearance accuracy close to that of the non-parametric model, but also has driving parameters that can be used for driving, thereby realizing the automatic conversion of the high-precision non-parametric model obtained by scanning to a parametric model, avoiding the influence of human experience, and improving the generation efficiency and accuracy of the high-precision parametric model.
[0076] The embodiment of the present application provides an optional specific model vertex interpolation method, and the newly generated model vertices are distributed along each edge of the existing triangle surface. The model vertex interpolation method can be shown in Formula 1, for example:
[0077] v N+e =0.5(v i +v j ), (i, j)∈ε e (1)
[0078] In Formula 1, N is the original model vertex of the initial parameterized model, e is the expanded model vertex, 0.5 is a set parameter that can be adjusted based on the size of the number of expanded model vertices to be expanded, i and j are two model vertices that form an edge in the initial parameterized model, ε e is the set of edges formed by two model vertices in the initial parameterized model, v i and v j are the positions of i and j in the model.
[0079] The above method is difficult to reflect the changes in vertices along the normal direction of the triangle surface, so the embodiment of the present application introduces a method of model vertex interpolation based on the normal direction.
[0080] That is, in addition to expanding the model vertices of the initial parameterized model by model vertex interpolation in the above manner, in a possible implementation, S102 includes:
[0081] S1021: Determine the number of to-be-expanded model vertices of the initial parameterized model according to the number of initial model vertices of the initial parameterized model and the number of the first model vertices.
[0082] S1022: Based on normal interpolation parameters, the number of expanded model vertices to be expanded is added in the initial parameterized model through model vertex interpolation to obtain the registered three-dimensional model, wherein the normal interpolation parameters are used to identify the degree of concavity of the vertex positions of the expanded model vertices in the initial parameterized model relative to the non-parametric model.
[0083] For details, please refer to Figure 2 As shown, the arrows marked on the model vertices newly generated by interpolation (ie, the expanded model vertices) are used to reflect the corresponding normal interpolation parameters.
[0084] This method can be achieved by formula 2:
[0085] v N+e =0.5(v i +v j )+s e n e , (i, j)∈ε e (2)
[0086] Among them, n e is the vector representation of the normal, sx is the direction of the normal, and represents the degree of concavity reflected by the normal.
[0087] For the non-parametric model and the initial parametric model, when the complete triangular faces of the two overlap, the vertices on the smaller surface of the non-parametric model are located above or below the corresponding surface of the initial parametric model. Figure 2 The new model vertices inserted along the normal direction can represent the displacement on the non-patch plane, so that the registered 3D model can be generated more accurately, that is, the difference in model shape between the generated registered 3D model and the non-parametric model is relatively small. When the target 3D object is a human body, the registered 3D model can also be recorded as a high-quality body model (High Quality Body Model, HQBM).
[0088] In a possible implementation, S103: performing posture alignment of the registered three-dimensional model with the non-parametric model to obtain an aligned three-dimensional model includes:
[0089] S1031: Based on the same key point semantics, label a target number of model key points on the registered three-dimensional model and the non-parametric model respectively.
[0090] S1032: Performing posture alignment of the registered three-dimensional model with the non-parametric model according to the model key points to obtain the aligned three-dimensional model.
[0091] Model key points are feature points used to identify the shape features of the target 3D object. For example, when the target 3D object is a human body, the model key points may include specific points that identify the shape features of the human body, such as the mouth, eyes, nose, hands, and feet. Figure 3 The model on the left is a registered three-dimensional model (target: HQBM), and the model on the right is a non-parametric model (Template), and the points on it are model key points marked by S1031.
[0092] The number and semantics of the key points of the model marked on the registered 3D model and the non-parametric model are the same. For example, the five key points of the model, mouth, left hand, right hand, left foot, and right foot, are marked on the registered 3D model and the non-parametric model. The above is just for the sake of example. In general, the number of marked model key points ranges from dozens to hundreds.
[0093] Since the registered 3D model and the non-parametric model are annotated with model key points with the same key point semantics, these model key points can be used as the basis for alignment. Through posture alignment, the posture of the aligned 3D model obtained based on the registered 3D model is consistent with that of the non-parametric model, for example Figure 3 By aligning the registered 3D model to the non-parametric model, an aligned 3D model with the same posture as the non-parametric model can be obtained, for example Figure 4As shown, the aligned 3D model on the left and the non-parametric model on the right.
[0094] In a possible implementation, S1032 includes:
[0095] According to the model key points of the registered three-dimensional model and the non-parametric model, an alignment parameter of the registered three-dimensional model relative to the non-parametric model is calculated, and the alignment parameter includes at least one of rotation information, translation information, scaling information and posture information.
[0096] The registered three-dimensional model is posture-aligned with the non-parametric model based on the alignment parameters to obtain the aligned three-dimensional model.
[0097] The above method can also belong to the rigid alignment method, as shown in Formula 3:
[0098] s * , R * , T * ,θ * =
[0099] argmin s，R，T，θ ||K tar get -K template (HQBM(s,R,T,θ))|| 2 (3)
[0100] Among them, s is scaling, R is rotation, T is translation, θ is posture, and K target K is the key point position of the registered 3D model. template is the location of the model key points of the non-parametric model.
[0101] Formula 3 identifies the amount of scaling, rotation, translation, and pose adjustment required to align the registered 3D model to the non-parametric model through pose alignment.
[0102] This results in an aligned 3D model that is in the same pose as the non-parametric model.
[0103] In a possible implementation, S104: determining model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model includes:
[0104] S1041: Perform vertex matching in the non-parametric model according to the model vertices in the aligned three-dimensional model to obtain a model vertex pair, wherein the model vertex pair includes a first model vertex in the aligned three-dimensional model and a second model vertex in the non-parametric model, and the second model vertex is the model vertex in the non-parametric model that is closest to the first model vertex.
[0105] S1042: Determine model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model according to the model vertex pairs.
[0106] After the rigid body alignment is completed, the aligned 3D model (also denoted as HQBM) and the non-parametric model are in the same position and orientation. The nearest neighbor matching is performed using the K-Nearest Neighbor (KNN) algorithm to find the dense nearest match between the HQBM and the non-parametric model. The nearest match is a series of pairings, which can be achieved by formula 4:
[0107] S={V i , V k(i)} i∈HQBM (4)
[0108] Among them, V i To align model vertices in a 3D model, V k(i) For the nonparametric model with V i The corresponding model vertices, HQBM is the set of model vertices in the aligned 3D model.
[0109] After the model vertex pairs are determined, the position difference between the first model vertex and the second model vertex in the same model vertex pair can be represented by the corresponding model vertex offset parameters. Based on the position difference identified by each model vertex offset parameter, it can be indicated how to adjust the three-dimensional model to a non-parametric model, so that the adjusted target parametric model is closer to the actual shape of the target three-dimensional model.
[0110] In a possible implementation manner, if the aligned three-dimensional model is obtained by performing posture alignment of the registered three-dimensional model to the non-parametric model according to the model key points, S1042 includes:
[0111] A key point pair in the model vertex object is determined, wherein a first model vertex in the key point pair is the model key point in the aligned three-dimensional model.
[0112] The model vertex offset parameters are determined according to the key point offsets determined by the key point pairs and the vertex offsets determined by the model vertex pairs.
[0113] Among them, the key point pair can be used as the basis for adjusting the aligned 3D model from a detailed perspective, based on the more critical positions in the model: the model key points. The model vertex pair takes the whole as the optimization target and is used as the basis for adjusting the aligned 3D model from the overall level. In this way, in the process of adjusting the aligned 3D model, reference is made from multiple dimensions, which improves the shape accuracy of the adjusted target parameterized model.
[0114] Specifically, the model vertex offset parameters for aligning the 3D model to the non-parametric model can be calculated using Formula 5:
[0115]
[0116] Among them, N sparse is a key point pair, N dense is a model vertex pair, To align the position of the i-th model vertex in the three-dimensional model, is the position of the i-th model vertex in the non-parametric model, D is the initialized D space (generally each element is set to 0), and α, β and γ are weight coefficients.
[0117] The determined D is the detail learning module D. is a learnable vertex offset matrix, which is used to predict the relative offset of each model vertex between the non-parametric model and the aligned 3D model, that is, the model vertex offset parameter. is the real number space, V is the number of rows, and 3 is the number of columns.
[0118] Finally, S105: the method of adjusting the aligned three-dimensional model to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameter can be referred to in Formula 6:
[0119] M=M HQBM +D (6)
[0120] Among them, M is the target parameterized model, M HQBM To align the 3D model. The target parameterized model can be Figure 5 The left figure shows the non-parametric model, and the right figure shows the non-parametric model.
[0121] After the target parameterized model is determined, a texture image may be further added to the target parameterized model for subsequent visualization during the driving process.
[0122] In a possible implementation, after S105, the method further includes:
[0123] Generate a corresponding texture image according to the non-parametric model;
[0124] The texture image is migrated to the target parameterized model to obtain a target parameterized model having the texture image.
[0125] Use the texture space migration algorithm to migrate the texture image of the non-parametric model to the texture image space of the target parametric model. This process uses a texel representation method based on barycentric coordinate interpolation to complete the RGB (red, green, and blue) pixel interpolation from the target parametric model to the non-parametric model within a texel surface. The final output is a complete texture image on the target parametric model. The obtained UV map (a planar representation of the surface of a three-dimensional model, U and V refer to the horizontal and vertical axes of the 2D space) and the target parametric model with a texture image can be obtained through a standard rendering engine. Figure 6 As shown, Figure 6 The upper left corner is the non-parametric model, the lower left corner is the texture image, and the right side is the target parameterized model with texture image.
[0126] In a possible implementation, the method further includes: converting the target parameterized model into a model format file of the target three-dimensional object by saving the model based on a tree structure, wherein the model format file is suitable for a game engine.
[0127] By saving in a tree structure, the model mesh, material and texture image of the target parametric model can be saved in the tree structure, and the model format file obtained in this way can be directly called by the game engine. For example, the model format file can be an FBX file, which is a common CG model format for mainstream game rendering engines such as unity and unreal. This completes the entire automatic binding and driving process from the non-parametric model to the target parametric model.
[0128] Next, we will take the target 3D object as a human body as an example. Figure 7 As shown, including:
[0129] S701: SMPL-X model converted to HQBM (High Quality Human Body Model).
[0130] The SMPL-X model with fewer model vertices can be converted into the HQBM model with more model vertices by interpolating the model vertices based on the normal direction.
[0131] S702: Mark sparse 3D key points and complete rigid registration by HQBM-non-parametric model.
[0132] Through rigid registration, the HQBM is aligned to the non-parametric model so that the HQBM and the non-parametric model are in the same posture and orientation.
[0133] S703: KNN dense vertex matching, optimize the HQBM geometric detail space D, and restore the target template geometric details.
[0134] By optimizing the D space, the model shape of HQBM is made close to that of the non-parametric model.
[0135] S704: Texture migration algorithm, inputting the texture image of the non-parametric model to obtain the texture image in the HQBM space.
[0136] S705: FBX structuring, writing the blendshape, bone position and skinning information in HQBM into the FBX file.
[0137] In the aforementioned Figure 1-Figure 7 Based on the corresponding embodiment, Figure 8 The device structure diagram of a device for determining an object model provided in an embodiment of the present application, wherein the device for determining an object model 800 includes an acquisition unit 801, an expansion unit 802, an alignment unit 803, a determination unit 804, and an adjustment unit 805:
[0138] The acquisition unit 801 is used to acquire a non-parametric model obtained by scanning a target three-dimensional object;
[0139] The expansion unit 802 is used to expand the model vertices of the initial parameterized model according to the first model vertex number of the non-parametric model to obtain a registered three-dimensional model with a second model vertex number;
[0140] The alignment unit 803 is used to perform posture alignment of the registered three-dimensional model with the non-parametric model to obtain an aligned three-dimensional model;
[0141] The determining unit 804 is used to determine the model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model;
[0142] The adjusting unit 805 is used to adjust the aligned three-dimensional model to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameters.
[0143] In a possible implementation manner, the alignment unit is further configured to:
[0144] Based on the same key point semantics, respectively annotating a target number of model key points on the registered three-dimensional model and the non-parametric model;
[0145] The registered three-dimensional model is posture-aligned with the non-parametric model according to the model key points to obtain the aligned three-dimensional model.
[0146] In a possible implementation manner, the alignment unit is further configured to:
[0147] Calculate, according to the model key points of the registered three-dimensional model and the non-parametric model, an alignment parameter of the registered three-dimensional model relative to the non-parametric model, wherein the alignment parameter includes at least one of rotation information, translation information, scaling information, and posture information;
[0148] The registered three-dimensional model is posture-aligned with the non-parametric model based on the alignment parameters to obtain the aligned three-dimensional model.
[0149] In a possible implementation manner, the determining unit is further configured to:
[0150] According to the model vertices in the aligned three-dimensional model, vertex matching is performed in the non-parametric model to obtain a model vertex pair, wherein the model vertex pair includes a first model vertex in the aligned three-dimensional model and a second model vertex in the non-parametric model, and the second model vertex is a model vertex in the non-parametric model that is closest to the first model vertex;
[0151] Model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model are determined according to the model vertex pairs.
[0152] In a possible implementation manner, the determining unit is further configured to:
[0153] Determining a key point pair in the model vertex object, a first model vertex in the key point pair being the model key point in the aligned three-dimensional model;
[0154] The model vertex offset parameters are determined according to the key point offsets determined by the key point pairs and the vertex offsets determined by the model vertex pairs.
[0155] In a possible implementation, the expansion unit is further used for:
[0156] Determining the number of to-be-expanded model vertices for the initial parameterized model according to the number of initial model vertices of the initial parameterized model and the number of the first model vertices;
[0157] Based on the normal interpolation parameters, the number of expanded model vertices to be expanded is added to the initial parameterized model through model vertex interpolation to obtain the registered three-dimensional model, and the normal interpolation parameters are used to identify the degree of concavity of the vertex positions of the expanded model vertices in the initial parameterized model relative to the non-parametric model.
[0158] In a possible implementation, the device further includes a generation unit and a migration unit:
[0159] The generating unit is used to generate a corresponding texture image according to the non-parametric model;
[0160] The migration unit is used to migrate the texture image to the target parameterized model to obtain the target parameterized model having the texture image.
[0161] In a possible implementation manner, the device further includes a storage unit:
[0162] The saving unit is used to save the target parameterized model based on a tree structure to convert it into a model format file of the target three-dimensional object, where the model format file is suitable for a game engine.
[0163] It can be seen that after scanning the target three-dimensional object to obtain a non-parametric model, an initial parametric model with the same object type as the target three-dimensional object is determined, and the initial parametric model with driving parameters is used to expand the model vertices to obtain a registered three-dimensional model with a number of model vertices close to that of the non-parametric model. The registered three-dimensional model is posture-aligned with the non-parametric model to obtain an aligned three-dimensional model, so that the aligned three-dimensional model and the non-parametric model are in the same posture, so that the model vertex offset parameters that identify the aligned three-dimensional model relative to the non-parametric model can be determined, and the target parametric model of the target three-dimensional object is obtained by adjusting the aligned three-dimensional model based on the model vertex offset parameters. The target parametric model not only has an appearance accuracy close to that of the non-parametric model, but also has driving parameters that can be used for driving, thereby realizing the automatic conversion of the high-precision non-parametric model obtained by scanning to a parametric model, avoiding the influence of human experience, and improving the generation efficiency and accuracy of the high-precision parametric model.
[0164] The embodiment of the present application further provides a computer device, which is the computer device described above and may include a terminal device or a server, and the aforementioned object model determination device may be configured in the computer device. The computer device is described below in conjunction with the accompanying drawings.
[0165] If the computer device is a terminal device, see Fig. 9 As shown, the embodiment of the present application provides a terminal device, taking the terminal device as a mobile phone as an example:
[0166] Fig. 9 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig. 9The mobile phone includes: a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490. Those skilled in the art can understand that Fig. 9 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0167] Combine the following Fig. 9 A detailed introduction to the various components of the mobile phone:
[0168] The RF circuit 1410 may be used for receiving and sending signals during information transmission or communication or a call, and in particular, for receiving downlink information from a base station and sending it to the processor 1480 for processing.
[0169] The memory 1420 may be used to store software programs and modules, and the processor 1480 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1420 .
[0170] The input unit 1430 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the mobile phone. Specifically, the input unit 1430 may include a touch panel 1431 and other input devices 1432 .
[0171] The display unit 1440 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1440 may include a display panel 1441, and the display panel 1441 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.
[0172] The mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.
[0173] The audio circuit 1460 , the speaker 1461 , and the microphone 1462 can provide an audio interface between the user and the mobile phone.
[0174] Processor 1480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in memory 1420 and calling data stored in memory 1420.
[0175] The mobile phone also includes a power supply 1490 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1480 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption management.
[0176] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.
[0177] In this embodiment, the processor 1480 included in the terminal device also has the following functions:
[0178] Acquire a non-parametric model obtained by scanning a target three-dimensional object;
[0179] Expanding the model vertices of the initial parameterized model according to the first model vertices number of the non-parametric model to obtain a registered three-dimensional model with a second model vertices number;
[0180] Performing posture alignment of the registered three-dimensional model with the non-parametric model to obtain an aligned three-dimensional model;
[0181] Determining model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model;
[0182] The aligned three-dimensional model is adjusted to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameters.
[0183] If the computer device is a server, the present application embodiment also provides a server, see Fig.10 As shown, Fig.10The structural diagram of the server 1500 provided in the embodiment of the present application, the server 1500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1522 (for example, one or more processors) and a memory 1532, and one or more storage media 1530 (for example, one or more mass storage devices) storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage medium 1530 can be short-term storage or permanent storage. The program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1522 can be configured to communicate with the storage medium 1530 and execute a series of instruction operations in the storage medium 1530 on the server 1500.
[0184] The server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server 2003 or Windows Server 2008. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.
[0185] The steps performed by the server in the above embodiment can be based on Fig.10 The server structure shown.
[0186] In addition, an embodiment of the present application further provides a storage medium, wherein the storage medium is used to store a computer program, and the computer program is used to execute the method provided in the above embodiment.
[0187] The embodiments of the present application also provide a computer program product including instructions, which, when executed on a computer, enables the computer to execute the method provided in the above embodiments.
[0188] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium can be at least one of the following media: read-only memory (English: Read-only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store program codes.
[0189] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.
[0190] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application. Moreover, based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for determining an object model, characterized in that: The method comprises: Acquire a non-parametric model obtained by scanning a target three-dimensional object; Expanding the model vertices of the initial parameterized model according to the first model vertices number of the non-parametric model to obtain a registered three-dimensional model with a second model vertices number; Based on the same key point semantics, respectively annotating a target number of model key points on the registered three-dimensional model and the non-parametric model; Performing posture alignment of the registered three-dimensional model with the non-parametric model according to the model key points to obtain an aligned three-dimensional model; Determining model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model; The aligned three-dimensional model is adjusted to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameters.
2. The method according to claim 1, characterized in that The step of performing posture alignment of the registered three-dimensional model with the non-parametric model according to the model key points to obtain the aligned three-dimensional model comprises: Calculate, according to the model key points of the registered three-dimensional model and the non-parametric model, an alignment parameter of the registered three-dimensional model relative to the non-parametric model, wherein the alignment parameter includes at least one of rotation information, translation information, scaling information, and posture information; The registered three-dimensional model is posture-aligned with the non-parametric model based on the alignment parameters to obtain the aligned three-dimensional model.
3. The method according to claim 1 or 2, characterized in that: The determining of model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model comprises: According to the model vertices in the aligned three-dimensional model, vertex matching is performed in the non-parametric model to obtain a model vertex pair, wherein the model vertex pair includes a first model vertex in the aligned three-dimensional model and a second model vertex in the non-parametric model, and the second model vertex is a model vertex in the non-parametric model that is closest to the first model vertex; Model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model are determined according to the model vertex pairs.
4. The method according to claim 3, characterized in that If the aligned three-dimensional model is obtained by performing posture alignment of the registered three-dimensional model with the non-parametric model according to the model key points, determining the model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model according to the model vertex pairs includes: Determining a key point pair in the model vertex object, a first model vertex in the key point pair being the model key point in the aligned three-dimensional model; The model vertex offset parameters are determined according to the key point offsets determined by the key point pairs and the vertex offsets determined by the model vertex pairs.
5. The method according to claim 1, characterized in that: The method of expanding the model vertices of the initial parameterized model according to the first model vertices of the non-parametric model to obtain a registered three-dimensional model having a second model vertices number includes: Determining the number of to-be-expanded model vertices for the initial parameterized model according to the number of initial model vertices of the initial parameterized model and the number of the first model vertices; Based on the normal interpolation parameters, the number of expanded model vertices to be expanded is added to the initial parameterized model through model vertex interpolation to obtain the registered three-dimensional model, and the normal interpolation parameters are used to identify the degree of concavity of the vertex positions of the expanded model vertices in the initial parameterized model relative to the non-parametric model.
6. The method according to claim 1, characterized in that The method further comprises: Generate a corresponding texture image according to the non-parametric model; The texture image is migrated to the target parameterized model to obtain a target parameterized model having the texture image.
7. The method according to claim 1, characterized in that The method further comprises: The target parameterized model is saved based on a tree structure and converted into a model format file of the target three-dimensional object, where the model format file is suitable for a game engine.
8. A device for determining an object model, characterized in that: The device comprises an acquisition unit, an expansion unit, an alignment unit, a determination unit and an adjustment unit: The acquisition unit is used to acquire a non-parametric model obtained by scanning the target three-dimensional object; The expansion unit is used to expand the model vertices of the initial parameterized model according to the first model vertex number of the non-parametric model to obtain a registered three-dimensional model with a second model vertex number, the three-dimensional object corresponding to the initial parameterized model and the target three-dimensional object belong to the same object type, and the difference between the first model vertex number and the second model vertex number is less than a threshold value; The alignment unit is used to calculate an alignment parameter of the registered three-dimensional model relative to the non-parametric model according to model key points of the registered three-dimensional model and the non-parametric model, wherein the alignment parameter includes at least one of rotation information, translation information, scaling information and posture information; Performing posture alignment of the registered three-dimensional model with the non-parametric model based on the alignment parameters to obtain an aligned three-dimensional model; The determining unit is used to determine the model vertex offset parameters of the aligned three-dimensional model relative to the non-parametric model; The adjustment unit is used to adjust the aligned three-dimensional model to a target parameterized model corresponding to the target three-dimensional object according to the model vertex offset parameters.
9. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the object model determination method described in any one of claims 1 to 7 according to the instructions in the program code.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method for determining the object model according to any one of claims 1 to 7.