Object model determination method and related device

Abstract

The embodiments of the present application disclose an object model determination method and related devices, which can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. Among the multiple object models of different deformation states of the target object, the first model is an object model to which the target cavity model has been spliced. In order to splice the target cavity model into the second model, control points can be screened for the target cavity model spliced into the first model, and associated points with positional association relationships can be determined in the first model based on the control points. Since the first model and the second model are both object models for the same target object, the associated points can be determined in the second model, and the position information of the control points of the target cavity model in the second model can be updated based on the position information of the associated points in the second model, so that the control points can adapt to the deformation state of the second model, thereby realizing the splicing of the target cavity model into the second model and improving the efficiency of improving the object model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an object model determination method and related devices. Background Art

[0002] An object model can be used to represent the virtual image of the object, such as a face model for a human face or a doll model for a doll. When the object model needs to display the deformation of the corresponding object, the cavity model in the object model also needs to change with the deformation of the object model. For example, for a face model, the cavity model includes the oral cavity, nasal cavity, and eye cavity. When creating blendshapes for various facial expressions, the cavity model needs to accurately adapt to the deformation caused by each expression.

[0003] In the related technology of reconstructing object models through 3D scanning (photogrammetry), the cavity of the object is generally obscured and difficult to scan accurately. Therefore, the reconstructed object model generally lacks a cavity model, which needs to be additionally produced by personnel and spliced into the corresponding position of the object model to obtain a complete object model.

[0004] For different deformation states of the object model, such as the deformation target of the human face model, personnel are required to manually adjust the corresponding shape of the cavity model for each deformation target, and then splice it into the corresponding object model. This results in a long time period for making the object model, and the accuracy is also easily affected by human experience. Summary of the Invention

[0005] In order to solve the above technical problems, the present application provides an object model determination method and related devices, which automatically and quickly and accurately splice the target cavity model in the object model without manual adjustment, thereby improving the efficiency of object model improvement.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In one aspect, an embodiment of the present application provides a method for determining an object model, the method comprising:

[0008] Acquire an object model set for a target object, the object model set including a plurality of object models having different deformation states identifying the target object, the plurality of object models including a first model and a second model;

[0009] For the target cavity model that has been spliced into the first model, selecting control points from the cavity feature points of the target cavity model;

[0010] Determining, among the model feature points of the first model, an associated point having a positional association relationship with the control point;

[0011] Update the position information of the corresponding control point using the position information of the associated point in the second model to obtain an updated control point;

[0012] The target cavity model is spliced in the second model according to the updated control points.

[0013] On the other hand, an embodiment of the present application provides an object model determination device, the device comprising an acquisition unit, a screening unit, a determination unit, an update unit, and a splicing unit:

[0014] The acquiring unit is configured to acquire an object model set for a target object, wherein the object model set includes a plurality of object models having different deformation states for identifying the target object, and the plurality of object models include a first model and a second model;

[0015] The screening unit is configured to screen out control points from cavity feature points of the target cavity model that has been spliced into the first model;

[0016] The determining unit is configured to determine, from the model feature points of the first model, an associated point having a positional association relationship with the control point;

[0017] The updating unit is configured to update the position information of the corresponding control point using the position information of the associated point in the second model to obtain an updated control point;

[0018] The splicing unit is used to splice the target cavity model in the second model according to the updated control points.

[0019] In another aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory:

[0020] The memory is used to store program code and transmit the program code to the processor;

[0021] The processor is configured to execute the object model determination method described in the above aspect according to instructions in the program code.

[0022] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the object model determination method described in the above aspect.

[0023] On the other hand, an embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the object model determination method described in the above aspects.

[0024] It can be seen from the above technical solution that for multiple object models with different deformation states of the target object, including a first model and a second model, the first model is an object model to which the target cavity model has been spliced, while the target cavity model has not yet been spliced in the second model. In order to quickly and accurately splice the target cavity model into the second model, control points can be screened out from the cavity feature points included in the target cavity model spliced into the first model, and associated points with positional association relationships can be determined in the model feature points of the first model based on the control points. Since the first model and the second model are both object models for the same target object, although the second model has a different deformation state relative to the first model, they both have model feature points with the same feature point semantics, so the associated points can be determined in the second model, and based on the positional association relationship, the position information of the control points of the target cavity model in the second model can be updated with the position information of the associated points in the second model, so that the control points can adapt to the deformation state of the second model, thereby realizing the splicing of the target cavity model into the second model. For multiple object models of the target object, the target cavity model can be automatically and accurately spliced in other object models based on the target cavity model that has been spliced in the first model, without the need for manual adjustment, thereby improving the efficiency of object model improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 A schematic diagram of a determination scenario of an object model provided in an embodiment of the present application;

[0027] Figure 2 A flow chart of a method for determining an object model provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of a target cavity model provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of control points of an eye cavity model provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of splicing an eye cavity model on a second model based on control points and associated points provided in an embodiment of the present application;

[0031] Figure 6A schematic diagram of a face model with a cavity model spliced together provided in an embodiment of the present application;

[0032] Figure 7 This is an interface diagram of a plug-in provided in an embodiment of the present application;

[0033] Figure 8 A schematic diagram of splicing a target cavity model corresponding to a neutral face model into a face model with a laughing expression, provided in an embodiment of the present application;

[0034] Figure 9 A schematic diagram of a method for determining an object model based on a Laplace deformer provided in an embodiment of the present application;

[0035] Figure 10 A diagram showing the structure of an object model determination device provided in an embodiment of the present application;

[0036] Figure 11 A structural diagram of a terminal device provided in an embodiment of the present application;

[0037] Figure 12 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present application are described below with reference to the accompanying drawings.

[0039] For purposes such as driving and rendering, an object can have multiple object models in different deformation states. In related technologies, for each object model, the shape and parameters of the cavity model must be manually adjusted to match the corresponding object model. The adjusted cavity model can then be spliced into the object model to complete the object model. This manual adjustment method is not only inefficient, but also easily affected by human experience in terms of accuracy.

[0040] To this end, an embodiment of the present application provides an object model determination method, which can automatically and quickly and accurately splice the target cavity model in each object model corresponding to the target object without manual adjustment, thereby improving the efficiency of object model improvement.

[0041] The object model determination method provided in the embodiment of the present application can be implemented by a computer device, which can be a terminal device or a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, etc. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0042] It is understandable that in the specific implementation of this application, the facial models used may involve data related to user information, etc. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0043] First, several terms involved in the embodiments of this application are briefly introduced:

[0044] A virtual environment is a virtual environment displayed (or provided) when an application is running on a terminal. This virtual environment can be a simulation of the real world, a semi-simulated, semi-imaginary three-dimensional environment, or a purely imaginary three-dimensional environment. The virtual environment can be any of a two-dimensional, a 2.5-dimensional, or a three-dimensional virtual environment. The following embodiments illustrate a three-dimensional virtual environment, but this is not limiting. Optionally, the virtual environment can also be used for virtual environment battles between at least two three-dimensional characters.

[0045] A 3D character refers to an animated object in a virtual environment. This animated object can be at least one of a virtual person, a virtual animal, or an animated character. Optionally, when the virtual environment is a 3D virtual environment, the 3D character is a 3D model created using animation skeletal technology. Each 3D character has its own unique shape and volume within the 3D virtual environment and occupies a portion of the space within the 3D virtual environment.

[0046] Face model (or head model): It is a model located on the face (or head) of a three-dimensional character. The face model includes bones and meshes. Bones are used to build a skeleton that supports the image of a three-dimensional character and drives the movement of the three-dimensional character; meshes (also called skins or skin meshes) are polygonal meshes bound between the bones, and there are multiple vertices on the polygonal meshes. For the face model, bones are used to control the position of each vertex during the facial expression or facial movement of the three-dimensional character. That is, the position change of several bones in the skeleton will cause each vertex on the mesh to move.

[0047] When the bones in the face model change position, the required displacement of each vertex on the mesh may be different. In other words, each vertex has its own skin weight, which is used to indicate the contribution of the control point transformation on each bone to the vertex transformation. Optionally, the skin weight of each vertex can be calculated using the bounded biharmonic weight method or the moving least squares method.

[0048] Skeleton-skinned animation: The skeleton in the face model can be divided into multiple layers of parent-child skeletons. Driven by animation keyframe data, the positions of each parent-child skeleton are calculated. Each frame is rendered based on the position of each vertex on the skeleton-controlled mesh, and the continuous change effect is reflected through multiple consecutive frames.

[0049] In applications based on three-dimensional virtual environments, a function is provided for personalizing the facial model of a three-dimensional character (referred to as the face-pinching function).

[0050] like Figure 1 As shown, the terminal device 100 is used as the aforementioned computer device, and the target object is exemplarily the face of user a. When user a makes different facial expressions, a plurality of corresponding face models (including a first model and a second model) are obtained through three-dimensional scanning. None of these face models have the target cavity part, such as the eye cavity and the oral cavity model ( Figure 1 shown in grey).

[0051] After the target cavity model is spliced into the first model through the terminal device, the terminal device can filter out control points from the cavity feature points included in the target cavity model spliced into the first model, and determine corresponding association points in the model feature points of the first model based on the control points.

[0052] Since both the first model and the second model are object models for the same target object, although the second model has a different deformation state relative to the first model, they both have model feature points with the same feature point semantics. Therefore, by determining the correspondence between the control points and the associated points, the terminal device can update the position information of the corresponding control points based on the actual position of the associated points in the second model, so that the control points can adapt to the deformation state of the second model, so that the target cavity model can be aligned to the second model with a different deformation state from the first model through the updated control points, thereby achieving the splicing of the target cavity model and the second model.

[0053] Therefore, for any face model of user a except the first model among the multiple face models of user a, the terminal device can automatically and accurately splice the target cavity model among these face models based on the spliced target cavity model in the first model, without the need for manual adjustment, thereby improving the efficiency of object model improvement.

[0054] Figure 2 This is a flowchart of a method for determining an object model provided in an embodiment of the present application, and is illustrated using a terminal device as an example of the aforementioned computer device. The method includes:

[0055] S201: Acquire an object model set for a target object.

[0056] This application does not limit the type of target object, and it can be any object with a cavity that can change the shape of the cavity by deforming itself, such as a human face, an animal face, a doll, etc.

[0057] Since a target object can have different deformation states, in order to comprehensively reconstruct the target object, a target object has multiple object models for identifying different deformation states of the target object. These multiple object models can be collectively referred to as an object model set. Different object models in the multiple object models identify different deformation states of the target object.

[0058] In a possible implementation, the target object is a human face, and the object model set includes a plurality of human face models with different expression states identifying the human face.

[0059] For example, taking a human face as the target object, different face models of a face can identify different expression states of the face, such as a face model of a laughing expression, a face model of a crying expression, etc.

[0060] However, since each object model is reconstructed by three-dimensional scanning of the target object in different deformation states, it lacks the target cavity model corresponding to the cavity part of the target object, and therefore does not belong to a complete object model.

[0061] The target cavity model is related to the type of the target object. In one possible implementation, the target cavity model spliced into the first model includes at least one of the nasal cavity, oral cavity, or eye cavity. For example, when the target object is a human face, the corresponding target cavity model includes the oral cavity, eye cavity, and nasal cavity on the face. Figure 3 shown.

[0062] In this object model set, two different object models are recorded as the first model and the second model. In this embodiment of the application, the first model is the object model that has been spliced with the target cavity model and is a complete object model. However, the other object models in the object model set, except the first model, have not yet been spliced with the target cavity model and need to be spliced with the adapted target cavity model through the subsequent implementation of S202-S205.

[0063] It should be noted that the present application is not limited to the method of splicing the target cavity model on the first model.

[0064] When the target object is a human face and the object model is a face model, this application does not limit the expression state identified by the first model. For example, it can be a face model that identifies any expression state. For face models, there is an expressionless neutral face model (also a face model). In some cases, other face models that embody expressions are mostly determined based on the neutral face model. Therefore, in order to improve the accuracy of the subsequent automatic splicing of the target cavity model, in one possible implementation, the first model is an expressionless neutral face model and the second model is an expressive face model.

[0065] S202: For the target cavity model that has been spliced into the first model, select control points from the cavity feature points of the target cavity model.

[0066] S203: Determine, from the model feature points of the first model, associated points that have a positional association relationship with the control point.

[0067] The cavity feature points are used to identify the shape of the target cavity model, such as Figure 3 The intersection points of the lines in the oral cavity, eye cavity and nasal cavity are shown as cavity feature points. The cavity feature points have corresponding position information, which is used to identify the positions of the cavity feature points on the target cavity model.

[0068] The control points screened by the terminal device are used to control the cavity feature points of the target cavity model to be aligned with the second model through the updated position information when splicing the target cavity model to the second model.

[0069] In the embodiment of the present application, the position information of the control points is mainly updated by using the associated points determined from the first model. Since the target cavity model has been spliced onto the first model to adapt to the deformation state of the first model, the terminal device can determine the associated points that have a positional association relationship with the control points from the model feature points of the first model for the target cavity model that has been spliced onto the first model.

[0070] The model feature points of the first model are used to identify the shape of the first model. The associated points selected from the model feature points of the first model have a corresponding position identification function with the control points, that is, through the position information and position association relationship of the associated points in the first model, the position of the control point corresponding to the associated point relative to the first model can be more clearly determined.

[0071] Furthermore, since both the first and second models are object models for the same target object, although the second model has a different deformation state relative to the first model, they both possess model feature points with the same feature point semantics. This means that the association points identified in the first model exist in all object models included in the object model set. The terminal device can accurately identify association points in different object models based on the feature point semantics embodied by the association points.

[0072] After an associated point having a positional association relationship with a control point is determined through the first model of the spliced target cavity model, the terminal device can more clearly determine the position of the control point corresponding to the associated point relative to different object models through the position information of the associated point in different object models and the positional association relationship.

[0073] In a possible implementation, S202 includes:

[0074] Determine a seam where the target cavity model is spliced with the first model, where the seam is used to identify a splicing position between the target cavity model and the first model; and determine the control point from cavity feature points at the seam.

[0075] The seam is used to identify the splicing position of the first model and the target cavity model. When the target object is in different deformation states, the seam is likely to cause the shape of the cavity part corresponding to the target cavity model to change. Therefore, the cavity feature point at the seam is determined as the control point. The control point can not only accurately reflect the impact of the deformation state of different object models on the cavity part, but also better change the target cavity model to adapt to the deformation state reflected by the object model.

[0076] For the cavity feature points of the target cavity model at the seam, the terminal device can determine all the cavity feature points at the seam as control points, or can determine some of the cavity feature points at the seam as control points. This application does not limit this.

[0077] like Figure 4 As shown, taking the object model as a human face model and the target cavity model as an eye cavity model as an example, the circular position marked by the black dot is the seam between the eye cavity model and the human face model, the model within the circular position is the eye cavity model, and the model outside the circular position is the human face model ( Figure 4 Only the part of the face model around the eye cavity model is shown), and the determined control points are represented by the black dots.

[0078] In order to enable the terminal device to more accurately identify the location association relationship, in a possible implementation, S203 includes:

[0079] Determine the distance between the model feature point and the control point in the first model; use the model feature point with a distance less than a threshold as the associated point corresponding to the control point, and the control point and the associated point are in one-to-one correspondence.

[0080] by Figure 4 For example, the intersection of the midlines of the face model is the model feature point. Based on the determined control points, the terminal device can determine the distance between each model feature point and the control point. In some scenarios, the closer the model feature point is to the control point, the stronger the positional correlation between the control point and the model feature point. To improve calculation efficiency, it is also possible to calculate only the distance between the model feature points near the seam and the control point.

[0081] By setting a smaller threshold, the model feature points that are closer to the control point can be determined from the model feature points as the associated points corresponding to the control point. Since the position information of the control point relative to different object models needs to be updated later through the position information of the associated points, in one possible implementation, in order to improve the control accuracy, the control points and the associated points are in a one-to-one correspondence. For example, in some scenarios, when the target cavity model is spliced into the first model, the cavity feature points and model feature points at the seam are fused, that is, the cavity feature points and model feature points at the seam are in the same position. Therefore, through the above method, the model feature points that coincide with the position of the control point are selected as the associated points that correspond one-to-one with the control point.

[0082] S204: updating the position information of the corresponding control point using the position information of the associated point in the second model to obtain an updated control point.

[0083] As mentioned above, the model feature points of the first model and the second model for the same target object have the same feature point semantics, so the associated points determined based on the first model can be accurately identified in the second model through the feature point semantics of these associated points.

[0084] Because the first and second models identify different deformation states of the target object, the position information of the associated points in the second model may differ from the same associated points in the first model. Based on the positional relationship between the associated points and the control points, in order to automatically splice the target cavity model into the second model and adapt to the deformation state identified by the second model, the terminal device needs to update the position information of the control points of the target cavity model based on the position information of the associated points in the second model.

[0085] The updating method may be determined based on the positional association relationship identified by the association point.

[0086] For example, when the position association relationship is overlapping, the position information of the control point corresponding to the association point can be updated to the position information of the association point in the second model.

[0087] For example, when the position association relationship is a fixed orientation, angle, distance, etc., the position information of the associated point in the second model can be calculated using the orientation, angle, and distance identified by the position association relationship to determine the new position information, and the position information of the control point corresponding to the associated point can be updated to the new position information.

[0088] For example Figure 5 As shown, the first model and the second model are both face models for different expression states of the same face. The first model is a neutral face model without expression, and the second model is a face model with a smiling expression. Figure 5 In the figure, both the first and second models only show the eye cavity model of the left eye and the partial face model around the left eye. It can be seen that the smiling face expression identified by the second model causes the partial model of the left eye of the second model to change in shape relative to the first model.

[0089] Assumptions Figure 5 In the scene shown, the positional relationship between the control point and the associated point is coincident, then Figure 5 In the first model shown on the left, due to the overlap, the black dots at the seams are the determined control points and associated points. Figure 5 In the second model shown on the right, the shape of the left eye part of the face model changes because the smiley face expression is marked. The positions of these related points are determined in the second model by using the related points determined by the first model, such as Figure 5 The black dots at the seam are shown on the right.

[0090] Since the positional relationship between the control point and the associated point is coincident, the position information of the corresponding control point can be updated according to the position information of the associated point in the second model, and the position information of the control point is updated to the position information of the associated point in the second model. The position identified by the updated control point is Figure 5 Black dots shown on the right.

[0091] S205: splicing the target cavity model in the second model according to the updated control points.

[0092] Since the updated control points can accurately reflect the difference between the deformation state identified by the second model and the deformation state identified by the first model, the target cavity model can be automatically spliced into the second model through the position of the updated control points in the target cavity model and the positioning of the updated control points in the second model.

[0093] For example, in the aforementioned possible implementation, if the control point is a cavity feature point at a seam, the seam of the target cavity model in the second model can be directly located based on the updated control point, so that the target cavity model can be more accurately spliced in the second model through the identified seam. Figure 5 The scene shown on the right side is updated by the position of the control point. The updated control point is at Figure 5 The seam between the second model and the target cavity model.

[0094] It can be seen that for multiple object models with different deformation states for identifying the target object, including the first model and the second model, the first model is an object model to which the target cavity model has been spliced, while the target cavity model has not yet been spliced in the second model. In order to quickly and accurately splice the target cavity model into the second model, control points can be screened out from the cavity feature points included in the target cavity model spliced into the first model, and associated points with positional association relationships can be determined in the model feature points of the first model based on the control points. Since the first model and the second model are both object models for the same target object, although the second model has a different deformation state relative to the first model, they both have model feature points with the same feature point semantics, so the associated points can be determined in the second model. Since there is a positional association relationship between the associated points and the control points, the position information of the control points of the target cavity model in the second model can be updated based on the position information of the associated points in the second model, so that the control points can adapt to the deformation state of the second model, thereby achieving the splicing of the target cavity model into the second model based on the control points. This allows multiple object models of the target object to be automatically and accurately spliced into other object models based on the spliced target cavity model in the first model, without the need for manual adjustment, thereby improving the efficiency of object model improvement.

[0095] In order to better splice the target cavity model into the second model, in addition to accurately locating the position of the target cavity model in the second model through control points, it is also necessary to effectively adjust the position of the cavity feature points of the target cavity model in the second model so that the overall deformation of the spliced target cavity model can fit the deformation state identified by the second model.

[0096] Therefore, in a possible implementation, S205 includes:

[0097] S2051: For the target cavity model spliced into the first model, determine a first feature point spacing parameter corresponding to the cavity feature points.

[0098] The first feature point spacing parameter is used to identify the spacing between the corresponding cavity feature point and the adjacent cavity feature point in the first model, so as to reflect the distance relationship between each cavity feature point (including control points) of the target cavity model spliced in the first model and the adjacent cavity feature points.

[0099] For the first model, one cavity feature point may have a corresponding first feature point spacing parameter.

[0100] S2052: Determine, for the updated control points, a second feature point spacing parameter corresponding to the cavity feature points.

[0101] The second feature point spacing parameter is used to identify the spacing between the corresponding cavity feature point and the adjacent cavity feature point. When calculating the second feature point spacing parameter through the position information of each cavity feature point, the position information of the cavity feature point serving as the control point is the position information obtained by updating in the aforementioned S204, and is relatively fixed when determining the position information of the cavity feature point corresponding to the second model in the subsequent S2053, while the position information of other cavity feature points needs to be adjusted and determined through S2053.

[0102] For the second model, one cavity feature point may have a corresponding second feature point spacing parameter.

[0103] S2053: Determine position information of the cavity feature points corresponding to the second model based on a difference between the first feature point spacing parameter and the second feature point spacing parameter.

[0104] Since the first feature point spacing parameter can reflect the distance relationship between each cavity feature point and its surrounding adjacent cavity feature points when the target cavity model is adapted and spliced into the first model, when the target cavity model needs to be spliced into a second model that belongs to the same target object as the first model, this distance relationship can serve as an effective basis for how to adjust the target cavity model when splicing it into the second model. That is, for target cavity models spliced into different object models, the spacing between a cavity feature point and the adjacent cavity feature points should be relatively similar. This can ensure, to a certain extent, that the target cavity model will not undergo abnormal deformation in different object models, avoiding the unrealistic feeling of the determined complete object model.

[0105] Therefore, based on the guidance of minimizing the difference between the first feature point spacing parameter and the second feature point spacing parameter of each cavity feature point, by adjusting the position information of the cavity feature point that generates the second feature point spacing parameter, the position information of each cavity feature point corresponding to the second model can be determined, that is, the position where the cavity feature point should be when the target cavity model is adapted and spliced to the second model.

[0106] S2054: splicing the target cavity model into the second model according to the position information of the cavity feature points corresponding to the second model.

[0107] After the terminal device determines the position information of the cavity feature point corresponding to the second model through S2053, the terminal device can automatically and accurately splice the target cavity model into the second model, and the deformation of the target cavity model spliced into the second model can conform to the deformation state identified by the second model.

[0108] Next, a method for determining the feature point spacing parameter provided in an embodiment of the present application is described. This method is a possible implementation method, taking any one of the cavity feature points as the target feature point, and taking the first feature point spacing parameter as an example for description.

[0109] Wherein S2051 includes: determining the cavity feature points adjacent to the target feature point in the target cavity model as adjacent points; and determining the first feature point spacing parameter corresponding to the target feature point according to the difference between the average position information of the adjacent points and the position information of the target feature point.

[0110] As mentioned above Figure 3-5 As explained in the description, the cavity feature point is the intersection of the lines in the target cavity model shown, and the line reflects the connection relationship between the cavity feature points. Therefore, the adjacent point corresponding to the target feature point can be determined based on the cavity feature point that has a connection relationship with the target feature point. When changing the shape of the target cavity model, if it is necessary to change the position of the target feature point, the position of its adjacent point will also change with the change of the position of the target feature point, so the adjacent point belongs to the cavity feature point that has the closest positional relationship with the target feature point. Therefore, by determining the first feature point spacing parameter of the target feature point through the adjacent point, the spacing relationship between the target feature point and other cavity feature points in the target cavity model can be more accurately reflected. The first feature point spacing parameter of the target feature point can be obtained by formula 1:

[0111]

[0112] The obtained first feature point spacing parameter can be recorded as Laplace coordinates, which can represent the geometric details of the model. The i-th cavity feature point (for example, the target feature point) of the target cavity model is v i .v i The number of adjacent points is d i , this d i The adjacent points form a set N i , j identifies N i Any adjacent point in N. i The average coordinates of all points in v i The Laplace coordinates L(v i ) is v i The difference between the position information of the points and the average position information of the adjacent points.

[0113] Since the target cavity model has been spliced into the first model, the position information of each cavity feature point in the above formula is a known value when calculating the first feature point spacing parameter using formula 1. When calculating the second feature point spacing parameter using formula 2, only the control points among the cavity feature points have their position information updated by the associated points in the second model. When the position association relationship is overlapping, the position information of the control points is determined as shown in formula 2:

[0114] v′ i =u i ,i∈{m,…,n},m <n (2)

[0115] Among them, v′ i is the updated i-th control point, u i is the associated point corresponding to the i-th control point, the total number of control points is n-m+1, and n is the total number of cavity feature points in the target cavity model. i The position information is taken as v′ i location information.

[0116] The position information of other cavity control points except the control point needs to be calculated based on the difference in S2053.

[0117] Therefore, in one possible implementation, S2053 includes:

[0118] An objective function is constructed based on the difference between the first feature point spacing parameter and the second feature point spacing parameter, and the difference between the position information of the updated control point and the position information of the associated point in the second model; and the position information of the cavity feature point corresponding to the second model is determined by minimizing the objective function.

[0119] This can be achieved through formula 3:

[0120]

[0121] Among them, δ i is the first feature point spacing parameter, which can be understood as the Laplace coordinates of all cavity feature points before being spliced into the second model, L(v′ i ) is the second feature point spacing parameter, which can be understood as the Laplace coordinates of all cavity feature points after being spliced into the second model. Formula 3 is the objective function that needs to be optimized. The first term of the objective function is the change in the Laplace coordinates of all cavity feature points before and after being spliced into the second model, and the second term is the distance between the position where the control point is designated and the target position after being spliced into the second model. Minimize the objective function to obtain the position information of all cavity feature points. The target cavity model identified by the obtained position information retains the geometric details spliced into the first model, and the control points are updated at the designated positions based on the associated points.

[0122] Let’s take a specific application scenario as an example. Assuming that the target object is a human face and the multiple object models are multiple human face models, the control points of the target cavity model and the associated points on the neutral face are determined by the neutral face (first model) to which the target cavity model has been spliced. When it is necessary to splice the target cavity model for other human face models among the multiple human face models, the terminal device updates the position information of the control point to the position information of the associated point on the other human face models, thereby driving the control point to the corresponding position of the other human face models, and then automatically adjusts the shape of the target cavity model according to the above formulas 1-3, so as to accurately splice the target cavity model to the other human face models.

[0123] Thus, multiple face models that represent different expression states are spliced together with the adapted target cavity model to obtain a complete face model. Figure 6 It shows the face models of the upper eye cavity, oral cavity and nasal cavity that have been spliced together using the solution provided in the embodiment of the present application, and different face models indicate different expression states.

[0124] Some application scenarios for the embodiments of this application involve the production of digital human facial models. Simply providing the digital human's expression blendshapes (face models) for various expressions, a neutral face model, and a target cavity model, the target cavity model can be adaptively deformed to match the expression blendshapes for different expressions using the Laplace deformation algorithm. The deformed target cavity model can then be stitched (or spliced) onto the expression blendshape to create a complete expression blendshape. This solution significantly improves the work efficiency of artists and reduces the workload of modelers.

[0125] The embodiment of the present application can be provided to modelers in the form of a Maya plug-in and supports running in a home PC hardware and software environment. It is currently developed and deployed in Windows 10 and above environments. The plug-in can run in Maya 2017, Maya 2018, Maya 2019, and Maya 2020.

[0126] The plugin interface is as follows Figure 7 As shown in the figure, the plug-in name is Laplacian Mesh Deformer. Before starting the Laplacian deformer, as shown in the figure, Figure 9 As shown in the figure, we first determine the control points of the cavity model and the corresponding associated points of the neutral face model. Then, we build a Laplace deformer based on the control points and associated points.

[0127] After opening the Laplace deformer, first click the [1. Select Cavity Model] button to select the cavity model (such as the eye cavity, nasal cavity, and oral cavity) that has been spliced to the neutral face model. Figure 8 The left side shows the cavity model corresponding to the neutral face model. The neutral face model indicates a closed mouth and slightly open eyes. Then click [2. Set Cavity Model] to confirm.

[0128] Then click the button [3. Select face model] and select a face model that needs to be spliced with the cavity model, for example Figure 8 Select the facial model with a laughing expression shown on the right, and then click the [4. Set facial model] button to confirm.

[0129] Finally, click the button [Deform Cavity]. The plug-in will update and drive the cavity model confirmed by the button [2. Set Cavity Model] through the position information of the control point through the above-mentioned embodiment of the present application, and splice the cavity model into the face model determined by the button [4. Set Face Model] to achieve the following Figure 9 The control points are driven by the associated points of the second model to deform the cavity model, thereby obtaining Figure 8 The face model with a laughing expression and a cavity model is shown on the right.

[0130] It should be noted that other types of deformers can also be used in the embodiments of the present application. Under specific artistic requirements, different deformers produce different effects, and it is necessary to constantly select according to the specific situation.

[0131] The object model determination method provided by the embodiment of the present application can be effectively applied to the field of games. For example, by scanning and reconstructing a three-dimensional model of a real person, the embodiment of the present application can quickly splice the target cavity model for the three-dimensional model of the real person with different deformation states. This allows users to control the game character with their own image to play the game without having to wait for too long, thereby improving the user's gaming experience. For example, in a game scene, a user can quickly create a virtual person that matches their own image in the game (for example, through the face-pinching function when creating a game character) through the object model determination solution provided by the embodiment of the present application, so that the virtual person with their own image can be controlled to interact and fight in the game, thereby improving the user's immersion and achieving a sense of accomplishment and integration in the virtual world.

[0132] In the aforementioned Figure 1-9 Based on the corresponding embodiment, Figure 10 This is a diagram showing the structure of an object model determination device provided in an embodiment of the present application. The object model determination device 1000 includes an acquisition unit 1001, a screening unit 1002, a determination unit 1003, an update unit 1004, and a splicing unit 1005:

[0133] The acquiring unit 1001 is configured to acquire an object model set for a target object, wherein the object model set includes a plurality of object models having different deformation states for identifying the target object, and the plurality of object models includes a first model and a second model;

[0134] The screening unit 1002 is configured to screen out control points from cavity feature points of the target cavity model that has been spliced into the first model;

[0135] The determining unit 1003 is configured to determine, from the model feature points of the first model, an associated point having a positional association relationship with the control point;

[0136] The updating unit 1004 is configured to update the position information of the corresponding control point using the position information of the associated point in the second model to obtain an updated control point;

[0137] The splicing unit 1005 is configured to splice the target cavity model in the second model according to the updated control points.

[0138] In a possible implementation, the screening unit is further configured to:

[0139] Determining a joint where the target cavity model is spliced to the first model, where the joint is used to identify a splicing position between the target cavity model and the first model;

[0140] The control points are determined from cavity feature points at the seams.

[0141] In a possible implementation manner, the determining unit is further configured to:

[0142] determining a distance between a model feature point and the control point in the first model;

[0143] The model feature points whose distance is less than the threshold are used as the associated points corresponding to the control points, and the control points and the associated points are in one-to-one correspondence.

[0144] In a possible implementation, the splicing unit is further configured to:

[0145] For the target cavity model spliced into the first model, determining a first feature point spacing parameter corresponding to the cavity feature point, where the first feature point spacing parameter is used to identify a spacing between the corresponding cavity feature point and an adjacent cavity feature point in the first model;

[0146] Determining, for the updated control point, a second feature point spacing parameter corresponding to the cavity feature point, where the second feature point spacing parameter is used to identify a spacing between the corresponding cavity feature point and an adjacent cavity feature point;

[0147] Determining position information of the cavity feature point corresponding to the second model based on a difference between the first feature point spacing parameter and the second feature point spacing parameter;

[0148] The target cavity model is spliced in the second model according to the position information of the cavity feature points corresponding to the second model.

[0149] In a possible implementation, the splicing unit is further configured to:

[0150] constructing an objective function based on a difference between the first feature point spacing parameter and the second feature point spacing parameter, and a difference between the updated position information of the control point and the position information of the associated point in the second model;

[0151] By minimizing the objective function, position information of the cavity feature points corresponding to the second model is determined.

[0152] In a possible implementation, for any target feature point among the cavity feature points, the splicing unit is further configured to:

[0153] Determining a cavity feature point adjacent to the target feature point in the target cavity model as an adjacent point;

[0154] A first feature point spacing parameter corresponding to the target feature point is determined according to a difference between the average position information of the adjacent points and the position information of the target feature point.

[0155] In a possible implementation, the target object is a human face, and the object model set includes a plurality of human face models with different expression states identifying the human face;

[0156] The target cavity model spliced into the first model includes at least one of the nasal cavity, oral cavity or eye cavity.

[0157] In a possible implementation, the first model is an expressionless neutral face model, and the second model is an expressive face model.

[0158] It can be seen that for multiple object models with different deformation states for identifying the target object, including the first model and the second model, the first model is an object model to which the target cavity model has been spliced, while the target cavity model has not yet been spliced in the second model. In order to quickly and accurately splice the target cavity model into the second model, control points can be screened out from the cavity feature points included in the target cavity model spliced into the first model, and associated points with positional association relationships can be determined in the model feature points of the first model based on the control points. Since the first model and the second model are both object models for the same target object, although the second model has a different deformation state relative to the first model, they both have model feature points with the same feature point semantics, so the associated points can be determined in the second model, and based on the positional association relationship, the position information of the control points of the target cavity model in the second model can be updated with the position information of the associated points in the second model, so that the control points can adapt to the deformation state of the second model, thereby realizing the splicing of the target cavity model into the second model. For multiple object models of the target object, the target cavity model can be automatically and accurately spliced in other object models based on the target cavity model that has been spliced in the first model, without the need for manual adjustment, thereby improving the efficiency of object model improvement.

[0159] The present application also provides a computer device, which is the aforementioned computer device and may include a terminal device or a server. The aforementioned object model determination device may be configured in the computer device. The computer device is described below with reference to the accompanying drawings.

[0160] If the computer device is a terminal device, see Figure 11 As shown, the embodiment of the present application provides a terminal device, taking a mobile phone as an example:

[0161] Figure 11 The block diagram shows a partial structure of a mobile phone related to the terminal device provided in the embodiment of the present application. Figure 11 The mobile phone includes components such as a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490. It will be understood by those skilled in the art that Figure 11 The mobile phone structure shown in the figure does not constitute a limitation to the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0162] The following combination Figure 11 A detailed introduction to the various components of a mobile phone:

[0163] RF circuit 1410 can be used to receive and transmit signals during information transmission or calls. Specifically, it receives downlink information from the base station and transmits it to processor 1480 for processing. It also transmits uplink data to the base station. Typically, RF circuit 1410 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, and the like.

[0164] The memory 1420 can be used to store software programs and modules. The processor 1480 executes the various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1420. The memory 1420 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.).

[0165] The input unit 1430 may be configured to receive input numbers or character information, and generate key signal inputs related to user settings and function control of the mobile phone.

[0166] The display unit 1440 may be configured to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1440 may include a display panel 1441 .

[0167] The mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.

[0168] The audio circuit 1460 , the speaker 1461 , and the microphone 1462 can provide an audio interface between the user and the mobile phone.

[0169] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 1470, providing users with wireless broadband Internet access.

[0170] The processor 1480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1420 and calling data stored in the memory 1420.

[0171] The mobile phone also includes a power supply 1490 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1480 through a power management system, thereby realizing functions such as charging, discharging, and power consumption management through the power management system.

[0172] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0173] In this embodiment, the processor 1480 included in the terminal device further has the following functions:

[0174] Acquire an object model set for a target object, the object model set including a plurality of object models having different deformation states identifying the target object, the plurality of object models including a first model and a second model;

[0175] For the target cavity model that has been spliced into the first model, selecting control points from the cavity feature points of the target cavity model;

[0176] Determining, among the model feature points of the first model, an associated point having a positional association relationship with the control point;

[0177] Update the position information of the corresponding control point using the position information of the associated point in the second model to obtain an updated control point;

[0178] The target cavity model is spliced in the second model according to the updated control points.

[0179] If the computer device is a server, this embodiment of the application also provides a server, see Figure 12 As shown, Figure 12 The structural diagram of the server 1500 provided in the embodiment of the present application, the server 1500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1522 (for example, one or more processors) and a memory 1532, and one or more storage media 1530 (for example, one or more mass storage devices) for storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage medium 1530 can be temporary storage or permanent storage. The program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1522 can be configured to communicate with the storage medium 1530 to execute a series of instruction operations in the storage medium 1530 on the server 1500.

[0180] The server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server 2003. TM , Mac OS XTM , Unix TM ,Linux TM , FreeBSD TM etc.

[0181] The steps performed by the server in the above embodiment can be based on Figure 12 The server structure shown.

[0182] In addition, an embodiment of the present application further provides a storage medium, which is used to store a computer program, and the computer program is used to execute the method provided by the above embodiment.

[0183] An embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the method provided in the above embodiment.

[0184] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the above-mentioned storage medium can be at least one of the following media: read-only memory (English: Read-only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0185] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0186] The above is only one specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Moreover, based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for determining an object model, characterized in that: The method comprises: Acquire an object model set for a target object, the object model set including a plurality of object models having different deformation states identifying the target object, the plurality of object models including a first model and a second model, the first model being an object model with a target cavity model spliced thereto, and the second model being an object model without the target cavity model spliced thereto; Determining a joint where the target cavity model is spliced to the first model, where the joint is used to identify a splicing position between the target cavity model and the first model; determining a control point from cavity feature points located at the seam; determining a distance between a model feature point and the control point in the first model; The model feature points whose distance is less than the threshold are used as the associated points corresponding to the control points, and the control points and the associated points are in one-to-one correspondence; Update the position information of the corresponding control point using the position information of the associated point in the second model to obtain an updated control point; The target cavity model is spliced in the second model according to the updated control points.

2. The method according to claim 1, characterized in that The step of splicing the target cavity model in the second model according to the updated control points includes: For the target cavity model spliced into the first model, determining a first feature point spacing parameter corresponding to the cavity feature point, where the first feature point spacing parameter is used to identify a spacing between the corresponding cavity feature point and an adjacent cavity feature point in the first model; Determining, for the updated control point, a second feature point spacing parameter corresponding to the cavity feature point, where the second feature point spacing parameter is used to identify a spacing between the corresponding cavity feature point and an adjacent cavity feature point; Determining position information of the cavity feature point corresponding to the second model based on a difference between the first feature point spacing parameter and the second feature point spacing parameter; The target cavity model is spliced in the second model according to the position information of the cavity feature points corresponding to the second model.

3. The method according to claim 2, characterized in that The determining, based on the difference between the first feature point spacing parameter and the second feature point spacing parameter, position information of the cavity feature point corresponding to the second model includes: constructing an objective function based on a difference between the first feature point spacing parameter and the second feature point spacing parameter, and a difference between the updated position information of the control point and the position information of the associated point in the second model; By minimizing the objective function, position information of the cavity feature points corresponding to the second model is determined.

4. The method according to claim 2, characterized in that For any target feature point among the cavity feature points, determining a first feature point spacing parameter corresponding to the cavity feature point includes: Determining a cavity feature point adjacent to the target feature point in the target cavity model as an adjacent point; A first feature point spacing parameter corresponding to the target feature point is determined according to a difference between the average position information of the adjacent points and the position information of the target feature point.

5. The method according to claim 1, wherein The target object is a human face, and the object model set includes a plurality of human face models with different expression states of the human face; The target cavity model spliced into the first model includes at least one of the nasal cavity, oral cavity or eye cavity.

6. The method according to claim 5, characterized in that The first model is a neutral face model without expression, and the second model is a face model with expression.

7. An object model determination device, characterized in that: The device includes an acquisition unit, a screening unit, a determination unit, an update unit and a splicing unit: The acquisition unit is configured to acquire an object model set for a target object, the object model set including a plurality of object models having different deformation states that identify the target object, the plurality of object models including a first model and a second model, the first model being an object model to which a target cavity model has been spliced, and the second model being an object model not to which the target cavity model has been spliced; The screening unit is configured to screen out control points from cavity feature points of the target cavity model that has been spliced into the first model; The determining unit is configured to determine, from the model feature points of the first model, an associated point having a positional association relationship with the control point; The updating unit is configured to update the position information of the corresponding control point using the position information of the associated point in the second model to obtain an updated control point; The splicing unit is configured to splice the target cavity model in the second model according to the updated control points; The screening unit is also used for: Determining a joint where the target cavity model is spliced to the first model, where the joint is used to identify a splicing position between the target cavity model and the first model; Determining the control point from the cavity feature points at the seam; The determining unit is further configured to: determining a distance between a model feature point and the control point in the first model; The model feature points whose distance is less than the threshold are used as the associated points corresponding to the control points, and the control points and the associated points are in one-to-one correspondence.

8. The device according to claim 7, characterized in that The splicing unit is also used for: For the target cavity model spliced into the first model, determining a first feature point spacing parameter corresponding to the cavity feature point, where the first feature point spacing parameter is used to identify a spacing between the corresponding cavity feature point and an adjacent cavity feature point in the first model; Determining, for the updated control point, a second feature point spacing parameter corresponding to the cavity feature point, where the second feature point spacing parameter is used to identify a spacing between the corresponding cavity feature point and an adjacent cavity feature point; Determining position information of the cavity feature point corresponding to the second model based on a difference between the first feature point spacing parameter and the second feature point spacing parameter; The target cavity model is spliced in the second model according to the position information of the cavity feature points corresponding to the second model.

9. The device according to claim 8, characterized in that The splicing unit is also used for: constructing an objective function based on a difference between the first feature point spacing parameter and the second feature point spacing parameter, and a difference between the updated position information of the control point and the position information of the associated point in the second model; By minimizing the objective function, position information of the cavity feature points corresponding to the second model is determined.

10. The device according to claim 8, characterized in that For any target feature point among the cavity feature points, the splicing unit is further configured to: Determining a cavity feature point adjacent to the target feature point in the target cavity model as an adjacent point; A first feature point spacing parameter corresponding to the target feature point is determined according to a difference between the average position information of the adjacent points and the position information of the target feature point.

11. The device according to claim 7, characterized in that The target object is a human face, and the object model set includes a plurality of human face models with different expression states of the human face; The target cavity model spliced into the first model includes at least one of the nasal cavity, oral cavity or eye cavity.

12. The device according to claim 11, characterized in that The first model is a neutral face model without expression, and the second model is a face model with expression.

13. A computer device, characterized in that: The computer device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the object model determination method according to any one of claims 1 to 6 according to instructions in the program code.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the object model determination method according to any one of claims 1 to 6.

15. A computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the object model determination method according to any one of claims 1 to 6.

Images