Industrial manufacturing method based on digital human basic model and related equipment thereof

Abstract

The invention relates to the technical field of model construction, and provides an industrial manufacturing method based on a digital human basic model and related equipment thereof. The method comprises the following steps: performing target person facial region extraction on an input image to obtain a target region image, performing gender recognition on a target person to determine a basic model type, and performing facial feature point extraction on the target region image by using a coordinate recognition model to obtain a facial feature point coordinate set; then, the extracted facial feature point coordinates are mapped to the basic model in a feature matching mode to generate a first three-dimensional model, and in order to improve the details of the three-dimensional model, target detail enhancement is carried out on the first three-dimensional model according to a preset geometrical shape adjustment strategy to obtain a second three-dimensional model; and finally, performing structure optimization on the second three-dimensional model according to a preset model optimization strategy to generate a third three-dimensional model. According to the method, facial feature extraction and matching and model optimization are automatically carried out, so that the production efficiency and quality of the digital human are improved.

Description

Technical Field

[0001] This application relates to the technical field of model construction, and particularly relates to an industrial production method based on a digital human schema and related equipment. Background Art

[0002] In the construction of three-dimensional digital human models, in the film and television and game industries, realistic three-dimensional character models can bring a more real and immersive experience, enhancing the visual effects of works and the sense of participation of the audience. Secondly, in the applications of virtual reality (VR) and augmented reality (AR) technologies, these models can achieve a more interactive and personalized user experience and are widely used in fields such as education, training, and healthcare. In addition, in advertising and commercial displays, three-dimensional character models can be used for product displays, virtual try-ons, etc., improving marketing effects and customer satisfaction.

[0003] Existing digital human models need to construct a basic model from head to toe and map a large number of feature points to the basic model to achieve the fusion of human characteristics and the model. Therefore, there is a large amount of redundant feature data in existing digital human products, resulting in low optimization of the digital human model, making the generated three-dimensional model transition less natural, and causing slow loading speed and low running stability of existing digital human models on different devices. Summary of the Invention

[0004] In view of this, this application provides an industrial production method based on a digital human schema and related equipment to solve the problems of low automation and optimization in the construction of digital human models.

[0005] The first aspect of this application provides an industrial production method based on a digital human schema, and the method includes:

[0006] Extract the facial region of the target person from the input image to obtain the target region image;

[0007] Perform gender recognition based on the target region image to obtain a basic model type label, and select a preset basic model according to the basic model type label;

[0008] Extract facial feature point coordinates from the target region image according to a preset coordinate recognition model to obtain a set of facial feature point coordinates;

[0009] Map the set of facial feature point coordinates to the basic model according to a preset feature matching method to obtain a first three-dimensional model;

[0010] Perform target detail enhancement on the first three-dimensional model according to a preset geometric shape adjustment strategy to obtain a second three-dimensional model;

[0011] Optimize the structure of the second 3D model according to a preset model optimization strategy to obtain a third 3D model.

[0012] In an optional embodiment, the mapping of the set of facial feature point coordinates to the base model according to a preset feature matching method to obtain a first 3D model includes:

[0013] Identify the standard feature points in the base model, and compare the standard feature points with the set of facial feature point coordinates to extract the target facial feature point coordinates;

[0014] Perform preliminary alignment of the target facial feature point coordinates with the corresponding standard feature points to obtain preliminary corresponding data;

[0015] According to a preset affine transformation model and the preliminary corresponding data, perform preliminary alignment of the set of facial feature point coordinates with the base model to obtain a set of preliminary alignment coordinate points;

[0016] Perform thin plate spline deformation calculation on the set of preliminary alignment coordinate points to obtain the surface mapping parameters of each preliminary alignment coordinate point;

[0017] Perform one-to-one mapping of the standard feature points and the facial feature point coordinates according to the set of preliminary alignment coordinate points and the surface mapping parameters to obtain the first 3D model.

[0018] In an optional embodiment, the target detail enhancement of the first 3D model according to a preset geometric shape adjustment strategy to obtain a second 3D model includes:

[0019] Perform preliminary adjustment on the first 3D model according to a preset sparse matrix and the set of facial feature point coordinates to obtain a primary deformation model;

[0020] Perform key area identification on the primary deformation model to obtain a set of target key area coordinates;

[0021] Obtain deformation feature localization data according to a preset Laplacian deformation model and the set of target key area coordinates;

[0022] Adjust the primary deformation model according to the deformation feature localization data to obtain an advanced deformation model;

[0023] Perform facial contour and depth optimization on the advanced deformation model according to a preset geometric optimization model to obtain the second 3D model.

[0024] In an optional embodiment, the structure optimization of the second 3D model according to a preset model optimization strategy to obtain a third 3D model includes:

[0025] Identify key regions of the second 3D model and optimize the topological wiring of the key regions to obtain a set of facial topological structures;

[0026] Perform local smoothing on the second 3D model according to the set of facial topological structures to obtain a facial geometry model;

[0027] Merge geometric faces of the facial geometry model according to a preset facial optimization strategy to obtain a face number optimized model;

[0028] Perform vertex data analysis on the face number optimized model to obtain simplified target vertex pairs, and perform vertex simplification according to the simplified target vertex pairs to obtain a vertex optimized model;

[0029] Perform resolution level division on the vertex optimized model to obtain a third 3D model applied to different resolutions.

[0030] In an optional implementation manner, the merging geometric faces of the facial geometry model according to a preset facial optimization strategy to obtain a face number optimized model includes:

[0031] Step 51: Perform patch attribute analysis on the facial geometry model to obtain a first set of attribute information for each patch;

[0032] Step 52: Obtain a first geometric error between each vertex and its adjacent patches according to the first set of attribute information and a preset 3D network simplification calculation formula;

[0033] Step 53: When the first geometric error is greater than a preset error threshold, calculate a second geometric error between adjacent vertices according to the first geometric error, and screen out merging target vertex pairs according to the second geometric error;

[0034] Step 54: Perform geometric face merging according to the merging target vertex pairs to generate an optimized facial geometry model, and update the first set of attribute information according to the optimized facial geometry model;

[0035] Repeat steps 52 to 54 until the first geometric error is equal to the error threshold to obtain the face number optimized model.

[0036] In an optional implementation manner, the performing vertex data analysis on the face number optimized model to obtain simplified target vertex pairs, and performing vertex simplification according to the simplified target vertex pairs to obtain a vertex optimized model includes:

[0037] Step 61: Obtain the number of vertices of each patch in the face number optimization model, and compare the number of vertices of each patch with the corresponding vertex number threshold. When the number of vertices is greater than the vertex number threshold, determine the patch as the target simplification patch;

[0038] Step 62: Perform vertex attribute analysis on the target simplification patch to obtain the second attribute information set of each vertex;

[0039] Step 63: Perform vertex data analysis according to the second attribute information set to obtain the target simplification vertex set;

[0040] Step 64: Perform simplification evaluation on each vertex in the target simplification vertex set to obtain the simplified target vertex pairs;

[0041] Step 65: Merge adjacent vertices according to the target simplification vertex pairs, and update the face number optimization model according to the merged vertices;

[0042] Repeat steps 61 to 65 until the number of vertices of each patch in the face number optimization model is equal to the corresponding vertex number threshold to obtain the vertex optimization model.

[0043] In an alternative embodiment, after performing step 65, the method further includes:

[0044] Identify the key areas of the updated face number optimization model to evaluate the geometric shapes of the key areas;

[0045] When it is detected that the geometric shape details of the key area are lost, restore the patches corresponding to the key area according to the simplified target vertex pairs in step 64, and prohibit performing step 61 on the patches.

[0046] The second aspect of the present application provides an industrial production device based on a digital human schema, and the device includes:

[0047] An area extraction module, configured to extract the facial area of the target person from the input image to obtain a target area image;

[0048] A schema selection module, configured to perform gender recognition according to the target area image to obtain a basic model type label, and select a preset basic model according to the basic model type label;

[0049] A feature extraction module, configured to extract facial feature point coordinates according to a preset coordinate recognition model for the target area image to obtain a facial feature point coordinate set;

[0050] A mapping construction module, configured to map the set of facial feature point coordinates to the base model according to a preset feature matching method, so as to obtain a first three-dimensional model;

[0051] A detail enhancement module, configured to perform target detail enhancement on the first three-dimensional model according to a preset geometric shape adjustment strategy, so as to obtain a second three-dimensional model;

[0052] A structure optimization module, configured to perform structure optimization on the second three-dimensional model according to a preset model optimization strategy, so as to obtain a third three-dimensional model.

[0053] A third aspect of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the industrial production method based on the digital human base model as described above are implemented.

[0054] A fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the industrial production method based on the digital human base model as described above are implemented.

[0055] In summary, the present application at least includes the following beneficial technical effects:

[0056] 1. Through automated image processing, facial feature extraction, three-dimensional model mapping, and optimization, the process of traditional manual modeling is simplified, and the production efficiency is significantly improved. And through the preset feature matching and optimization strategies, a high-precision three-dimensional model that conforms to the characteristics of the target person can be quickly generated.

[0057] 2. Through technologies such as geometric shape adjustment and Laplacian deformation, the detail accuracy of the model can be improved, and the visual effect of the digital human is enhanced. At the same time, the optimization process of the model ensures the structural rationality and efficiency of the digital human model, avoiding waste of computing resources.

[0058] 3. Through face number optimization and vertex optimization, unnecessary patches and vertices are reduced, the computing performance is optimized, and the application requirements under different resolutions can be adapted, making the digital human model lighter and facilitating the fast and stable loading and running of the digital human model on different devices. Description of the Drawings

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 It is a flowchart of an industrial production method based on a digital human schema provided by an embodiment of the present application;

[0061] Figure 2 It is a schematic structural diagram of a male basic model provided by an embodiment of the present application;

[0062] Figure 3 It is a schematic structural diagram of a female basic model provided by an embodiment of the present application;

[0063] Figure 4 It is a functional module diagram of an industrial production device based on a digital human schema provided by an embodiment of the present application;

[0064] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0065] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0066] As Figure 1 shown, it is a flowchart of an industrial production method based on a digital human schema provided by an embodiment of the present application. The industrial production method based on a digital human schema provided by an embodiment of the present application includes the following steps.

[0067] Step S1: Extract the facial area of the target person from the input image to obtain the target area image.

[0068] Among them, the input image should be a portrait image. After obtaining the input image, it is sent to a preprocessing module for normalization processing to remove noise and irrelevant image elements, improving the accuracy of subsequent face recognition. After normalization, the image enters the face recognition module for face area recognition. The face recognition module usually adopts a convolutional neural network in deep learning. Through training a large number of images with labeled face areas, the convolutional neural network, a deep learning model, can learn how to recognize face features in the image, including main face areas such as eyes, nose, and mouth. Specifically, the network first extracts low-level features (e.g., edges, textures) through convolutional layers, then reduces the dimension and compresses these features through pooling layers, and finally outputs the bounding box position of the face through fully connected layers. After face area detection, the output result contains the face box coordinates in the image, which are used to represent the boundary of the target person's face area. Subsequently, through region segmentation technology, the extraction of the face area is further optimized to separate the face area from the background, and finally a separate target area image is formed. At this time, the obtained target area image only contains the face area.

[0069] Step S2: Perform gender recognition on the target area image to obtain a basic model type label, and select a preset basic model according to the basic model type label.

[0070] The face area image is sent to a gender recognition model, which usually adopts another convolutional neural network for gender classification. The model learns how to distinguish between men and women through male and female face data in the training set, and predicts the gender label of the input image by outputting a probability value. For example, if the output value is higher than the set threshold, it is determined to be male, otherwise female. After extracting the face area in the input image, the gender classifier goes through a series of convolutional and pooling operations and finally outputs a gender label ("male" or "female"). According to this gender label, the system will select a preset basic model that matches the gender from Figure 2 and Figure 3 the basic models shown. The basic model is a 3D digital human model with different face topologies and feature point configurations. For an input male image, the system selects a male basic model, which usually has a more angular chin and nose bridge; for an input female image, the system selects a female basic model, which usually has a more gentle facial contour.

[0071] Step S3: Extract facial feature points from the target area image according to a preset coordinate recognition model to obtain a set of facial feature point coordinates.

[0072] Among them, the facial feature points include, but are not limited to, facial organ parts such as the corners of the eyes, the tip of the nose, the corners of the mouth, the eyebrows, etc. Usually, an algorithm based on deep learning is adopted, combined with traditional image processing techniques for precise positioning of the feature points. The core technology of this process is a facial key point detection model based on a deep convolutional neural network. Such models are trained through a large number of labeled facial images and can accurately identify and locate the positions of facial key points. The network usually uses multiple convolutional layers to extract local features of the image and outputs the two-dimensional coordinates of each key point through a regression algorithm. The model learns the geometric relationships and position features of each part of the face through the face data in the training set, so as to identify the position of each feature point in the input image.

[0073] In an optional embodiment, in order to improve the accuracy and robustness of feature point extraction, some optimization algorithms are usually also used. For example, the 68-point face calibrator in Dlib or other multi-task learning models are used to refine the accurate positions of each facial feature point. For each input image, the extraction of facial feature points can be represented by the following mathematical formula:

[0074]

[0075] Among them, is the facial feature point coordinate set, n is the total number of feature points, ( ) is the coordinate of the i-th feature point, is the extraction function for the coordinate of the i-th feature point. Specifically, is a regression function used to predict the position of each feature point based on the input data of the image (i.e., the pixel values of the facial region image). The regression function is trained through the features of each layer in the deep convolutional neural network to calculate the most likely feature point coordinates for each image pixel. The feature point coordinates represent the precise values of the positions of the corners of the eyes, the tip of the nose, the corners of the mouth, etc. in the input image, forming the facial feature point coordinate set.

[0076] Step S4, map the facial feature point coordinate set to the base model according to a preset feature matching method to obtain a first three-dimensional model.

[0077] Among them, the base model includes, but is not limited to, a set of preset standard feature points used to represent the main features of the face. In order to identify the standard feature points, a preset structured grid or point cloud model is used to mark the standard feature points. The identification of the standard feature points is to automatically detect and identify the positions of the feature points in the model with the help of deep learning. The identification process of the standard feature points will not be elaborated here.

[0078] After obtaining the standard feature points, a point-to-point matching algorithm (e.g., nearest neighbor search, rigid registration algorithm) or a feature-based matching method is used to compare the facial feature point coordinate set with the feature points of the standard model. Both the point-to-point matching algorithm and the feature-based matching method utilize a distance metric (e.g., Euclidean distance) to find the most matching point pairs and determine the mapping relationship between each facial feature point and the standard feature point, thereby extracting the target facial feature point coordinates.

[0079] The goal of the initial alignment is to roughly align the standard feature points and the facial feature points so that they are relatively close in spatial position. The system uses a preset affine transformation model to achieve the initial alignment between the image and the base model. Among them, affine transformation is a linear transformation that preserves the relationship of parallel lines, including but not limited to operations such as translation, rotation, and scaling. Specifically, the affine transformation is completed through an affine transformation matrix A and a translation vector t, thereby mapping a point p in one coordinate system to a point p′ in another coordinate system. The affine transformation can be represented by the following mathematical formula:

[0080]

[0081] Among them, is the target facial feature point coordinate, is the transformed feature point coordinate, A is a 3x3 affine transformation matrix, and t is the translation vector. The matrix of the affine transformation contains information such as rotation and scaling, while the translation vector adjusts the overall translation position. By minimizing the distance between the standard feature points and the target feature points, the affine transformation matrix A can be calculated by the least squares method or other optimization methods.

[0082] It should be understood that the thin plate spline deformation algorithm is a non-linear deformation method, which further finely aligns these points by minimizing the curvature change of the target facial feature points. The core goal of the thin plate spline deformation algorithm is to minimize the difference between the deformed coordinate points and the coordinate points of the base model, while ensuring that the entire deformation process is as smooth as possible and avoiding unnatural shape changes. The mathematical formula of the thin plate spline deformation is as follows:

[0083]

[0084] Among them, is the target position after deformation, is the thin plate spline basis function, is the standard feature point in the base model, is the weight coefficient of each feature point, is the second-order gradient of the basis function, is a parameter for adjusting smoothness, and n is the number of feature points of the base model. The surface mapping parameters calculated by the thin plate spline deformation algorithm can be used to further map the standard feature points and the facial feature point coordinates one by one. This mapping process ensures that each standard feature point is accurately mapped to the corresponding position of the target facial feature point while maintaining the natural transition of facial features.

[0085] Finally, the preliminary aligned coordinate point set and the surface mapping parameters are used to map the standard feature points and the facial feature point coordinates one by one, so as to automatically perform face pinching fusion on the image and the base model according to the mapping relationship to obtain the first three-dimensional model.

[0086] Step S5: Perform target detail enhancement on the first three-dimensional model according to a preset geometric shape adjustment strategy to obtain a second three-dimensional model.

[0087] It should be understood that the contour in the first three-dimensional model is relatively sharp, and it is necessary to optimize the positions of facial feature points through geometric shape adjustment to make the overall contour of the first three-dimensional model refined and achieve a smooth transition of the internal geometric shape of the model. The first three-dimensional model is preliminarily adjusted according to a preset sparse matrix and the facial feature point coordinate set. By constructing a sparse matrix to represent the geometric relationship between facial feature points and performing preliminary alignment on these feature points, the geometric structure of the overall facial area of the first three-dimensional model is optimized, so as to obtain a refined model of the overall facial geometry (i.e., the primary deformation model).

[0088] When obtaining the primary deformation model, it is necessary to further refine the geometric shape in the key areas of the three-dimensional model, so that the organs in the three-dimensional model are more detailed and three-dimensional. By classifying according to the spatial position relationship of facial feature points, the key areas of the primary deformation model are identified to obtain a target key area coordinate set. The target key area coordinate set includes, but is not limited to, the feature point coordinates representing areas such as eyes, nose, and mouth in the model.

[0089] After obtaining the target key area coordinate set, a preset Laplacian deformation model is used to further process the primary deformation model to obtain deformation feature localization data. The Laplacian deformation algorithm optimizes the local areas in the facial geometry to ensure the smooth transition of the curves in these areas and avoid unnatural geometric changes. The goal of Laplacian deformation is to make adjustments according to the deformation feature localization data (such as the offset of feature points). The formula is as follows:

[0090]

[0091] Among them, is the displacement of feature point i, is the adjustment factor of Laplacian deformation, They are the coordinates of feature point i and adjacent feature point j respectively. is the neighborhood set of feature point i. By minimizing the offset of points in the local area to optimize the smoothness of the key area of the primary deformation model and reduce unnatural protrusions or depressions, a high-level deformation model for further geometric adjustment is obtained.

[0092] In order to further refine the facial structure, the high-level deformation model is adjusted according to the deformation feature positioning data to obtain a more accurate facial contour and depth change. In this application, a preset geometric optimization model (geometric optimization algorithm) is used to optimize the facial contour and depth of the high-level deformation model to obtain a second three-dimensional model. Among them, a preset geometric optimization algorithm is adopted in the geometric optimization model to adjust the facial contour and depth. This process is completed through the following geometric optimization formula:

[0093]

[0094] Among them, is the geometric optimization energy, is the weight coefficient, is the position of point i in the current model, is the position of the target geometric point, and N is the number of points in the model. By minimizing this energy function, the geometric shape of the model is optimized, making the facial contour and depth more in line with the natural facial structure. Geometric optimization mainly focuses on the fine adjustment of facial features to ensure the true presentation of the three-dimensional sense and contour of the face. The geometric optimization model continuously updates the details of the facial model through an iterative method, especially in the facial depth area (such as eye sockets, cheekbones, nasal bridges, etc.). During the optimization process, the three-dimensional sense of the model is enhanced by adjusting the depth of facial feature points to ensure the natural transition between the facial contour, shape, and various regions.

[0095] Step S6: Structure the second three-dimensional model according to a preset model optimization strategy to obtain a third three-dimensional model.

[0096] It should be understood that the model optimization strategy includes but is not limited to topological wiring optimization, face number optimization, vertex simplification, and resolution level division, so as to ensure a good balance among the details, geometric structure, and performance of the final digital human three-dimensional model.

[0097] The key regions of the second 3D model are identified through a preset deep learning model, and the key points of the face are located and their spatial relationships are determined to achieve the identification of key regions. After the identification of key regions is completed, the topological structure of the face region is optimized through topological wiring to ensure reasonable transition and natural connection of facial features. Specifically, a preset sparse matrix deformation algorithm or the Laplacian deformation algorithm is used to smoothly adjust the face to avoid unnatural geometric distortions in the face model. For the specific process, please refer to step S5 and will not be elaborated here. By optimizing the topological structure, the geometric structure of the face region can be made more natural.

[0098] Local smoothing is performed on the second 3D model according to the optimized face topological structure set (i.e., the key region after topological wiring optimization) to reduce the noise in the face region and repair the possible loss of details due to topological wiring adjustment. The spatial positions of the patches are adjusted through a preset thin plate spline deformation algorithm to ensure that the overall geometric shape of the face remains natural and smooth. At the same time, the thin plate spline deformation algorithm can effectively keep the surface smooth by minimizing the energy function and remove the local geometric errors caused by topological optimization. This process is completed through the following formula:

[0099]

[0100] where, is the deformation function, is the second-order partial derivative of the deformation function, is the optimized region. By minimizing this energy, the thin plate spline deformation algorithm ensures that the geometric shape of the face region is smooth and continuous.

[0101] A detailed patch attribute analysis is performed on the face geometry model (i.e., the second 3D model after local smoothing) to obtain the first attribute information set of each patch. Among them, the first attribute information set contains the geometric information of the patch, such as the normal, area, and boundary length of the patch. Specifically, the geometric attributes of each patch play a role in describing the shape and adjacent relationship of the patch in the first attribute information set, forming the basic data structure of the face geometry.

[0102] After the first-step patch attribute analysis, according to the obtained first attribute information set and the preset 3D network simplification calculation formula, the first geometric error between each vertex and its adjacent patches needs to be calculated next. The first geometric error is used to measure the geometric differences between vertices and between vertices and patches in space. The first geometric error can be calculated through the following formula:

[0103]

[0104] where, The first geometric error for vertex v The set of all faces adjacent to vertex v For face The area of For face The normal vector of The position vector of vertex v For face The centroid position of. The first geometric error is evaluated by calculating the distance between each face And the projection of the normal vector of the vertex v on the adjacent face. The contributions of all adjacent faces are summed up to form the total error of the vertex

[0105] After obtaining the first geometric error, if the error is greater than the preset error threshold, it is necessary to further calculate the second geometric error between adjacent vertices in order to screen out the target vertex pairs for merging. The second geometric error refers to evaluating whether two vertices are suitable for merging by calculating the geometric difference between them. To ensure that the error after merging will not be too large, the second geometric error can be calculated by the following formula

[0106]

[0107] Where Is the second geometric error between two adjacent vertices On the adjacent faces. By comparing the geometric errors of two vertices in the adjacent faces, we can evaluate whether they should be merged. In the case of a small error difference, vertex merging will result in a small shape change, thus ensuring the coherence and accuracy of the facial geometric model

[0108] When the target vertex pairs for merging are screened out, geometric face merging is performed on the vertex pairs according to the second geometric error. The process of vertex merging usually combines the positions of two vertices into a new position, and the new vertex position is determined by minimizing the geometric error after merging. Specifically, the new vertex position after merging can be calculated by the following formula

[0109]

[0110] Where Is the new vertex position after merging For vertex The set of all adjacent faces For face The area of For face The normal vector of The position vector of vertex v For face The centroid position of

[0111] After merging, the position of the new vertex is determined by optimizing the geometric error before and after merging to ensure that the facial geometry does not change too much as much as possible. After the new vertex is merged, all the faces related to the original vertex will be reconnected to the newly merged vertex, and the properties such as the normal vector and area of the face will be updated according to the merged geometry. After completing the geometric face merging, the first attribute information set needs to be updated to ensure that this information set reflects the properties of the merged faces. This update process includes but is not limited to updating the normal vector, area, and other geometric properties of the merged faces to provide accurate geometric data for subsequent steps.

[0112] By iteratively executing the process of the first geometric error calculation - screening of target vertex pairs for merging - face merging, the facial geometry is continuously optimized until the first geometric error meets the preset error threshold, and finally a face number optimized model is obtained. At this time, the details of the facial geometry are retained, while the number of faces is greatly reduced, thus achieving the optimization purpose.

[0113] After obtaining the face number optimized model, it is necessary to simplify the vertices on each face in the model. While reducing the redundant vertices on each face, it does not affect the face structure of the model after face number optimization. By obtaining the number of vertices of each face in the face number optimized model, the number of vertices of each face is compared with the corresponding vertex number threshold. When the number of vertices of a face exceeds this threshold, this face is determined as the target simplified face.

[0114] After determining the target simplified face, the attribute analysis of the vertices in each target simplified face is carried out to obtain the second attribute information set of each vertex. The second attribute information set includes but is not limited to the geometric position of the vertex, the normal vector, and the geometric relationship of adjacent faces. Through the second attribute information set, it is possible to judge which vertices have a lower contribution in the geometric form within the face and can be considered for merging. In particular, when multiple vertices are close to each other in space, they can be regarded as redundant vertices. After analysis, the redundant vertices can be selected for simplification.

[0115] After obtaining the second attribute information set, data analysis is carried out on the vertices corresponding to the second attribute information set to obtain the target simplified vertex set. The simplified target vertex set is screened according to the geometric importance and redundancy of the vertices. By calculating the geometric importance and influence of the vertices, it is possible to identify which vertices are not important in the geometric form and which vertices are redundant. It should be understood that redundant vertices are vertices that are relatively close in space and do not affect the overall geometric form of the face. By simplifying the redundant vertices, the number of vertices on the face can be reduced, and the shape and details of the face are less affected. The geometric importance I(v) of each vertex can be calculated by the following formula:

[0116]

[0117] Among them, is the geometric importance of the vertex ; is the set of all faces adjacent to vertex v, is the face ; is the face ; is the position information of vertex v, is the face ; By calculating the geometric importance of each vertex, an importance value can be assigned to each vertex, and the target vertices for simplification can be selected according to the geometric importance value.

[0118] After obtaining the set of target vertices for simplification, evaluate the simplification effect of each pair of target vertices to determine whether they can be merged. The decision of merging is made by calculating the geometric error of the merged vertex pair. The geometric error is mainly based on the impact on the geometric shape of the face after merging. If the error after merging is small and the merging does not significantly change the shape or details of the face, then vertex merging can be performed. The calculation of the geometric error can be through the following formula:

[0119]

[0120] Among them, is the geometric error after merging vertex ; is the vertex ; is the face ; is the face ; is the position of the new vertex after merging, is the face ; By calculating the geometric error of the new vertex after merging through the above formula, it is ensured that the geometric impact of the merging operation on the face is as small as possible to avoid destroying the structure of the face during the simplification process. When the evaluation process confirms that the impact of vertex merging on the geometric shape is minimal and meets the merging conditions, vertex merging can be performed. The position of the new vertex after merging is calculated by minimizing the geometric error. The calculation of the position of the new vertex during the vertex simplification process is the same as that during the face number optimization process. For details, please refer to the process of calculating the position of the new vertex during the face number optimization process, which will not be elaborated here. The new vertex after merging replaces the original two vertices, and the relevant faces will be reconnected to the new vertex, thereby updating the geometric information of the face.

[0121] The final vertex optimization model is obtained by iteratively performing the vertex simplification process until the number of vertices of each patch is equal to the corresponding vertex number threshold.

[0122] In an alternative embodiment, to avoid over-simplification of the details in the critical area due to vertex simplification, which may lead to a decrease in model quality. After each execution of vertex simplification, the method further includes:

[0123] Each patch is evaluated based on the geometric properties of the patch, such as curvature, normal direction, area, etc., and the facial model is divided into different regions according to the geometric properties of the patch. The critical area generally has a higher geometric complexity, with a larger surface curvature or a higher density of its patches. Thus, the critical areas are identified in the updated face number optimized model.

[0124] After identifying the critical area, the geometric shape of the critical area is evaluated to determine whether the details in the critical area have been over-simplified or lost during the vertex simplification process. Determine whether the details in the critical area have been over-simplified or lost during the vertex simplification process. Specifically, the following calculation formula can be used to quantify the geometric error of each critical area and check the degree of detail loss:

[0125]

[0126] Where, is the change in the geometric shape within the critical area R, is the area of the patch is the area of the patch is the normal vector of the patch is the normal vector of the patch is the original vertex position of the critical area before simplification, is the position of the merged vertex. By comparing the geometric errors before and after simplification, it can be determined whether the area has been over-simplified.

[0127] When it is detected that the geometric shape details of the critical area are lost (i.e., exceeds the preset threshold), it indicates that the details of this area have been affected and need to be restored. The patch is repaired using the simplified target vertices used in the vertex merging process. That is, the originally simplified vertices will be re-added to the patch, and the patch will no longer perform the vertex simplification operation in step 61.

[0128] The vertex optimization model is divided into resolution levels according to different resolution requirements to obtain a third 3D model applicable to different resolutions, so as to adapt to different devices and application scenarios. The process of level division can generate multiple versions from high resolution to low resolution by simplifying the model's mesh at different levels. Each version corresponds to a different level of detail, ensuring efficient loading and rendering under different devices or different requirements. Resolution level division not only ensures that the model has sufficient details at high resolution, but also guarantees that it can maintain sufficient geometric structure at low resolution, enabling the model to run smoothly on various devices.

[0129] This application is applied to the field of model construction technology. By extracting the target person's facial area from the input image to obtain the target area image, the basic model type is determined by gender recognition of the target person, and the facial feature point coordinate set is obtained by using the coordinate recognition model to extract facial feature points from the target area image. Then, the extracted facial feature point coordinates are mapped to the basic model through feature matching to generate the first 3D model. To improve the details of the 3D model, the first 3D model is enhanced with target details according to the preset geometric shape adjustment strategy to obtain the second 3D model. Finally, the second 3D model is structurally optimized according to the preset model optimization strategy to generate the third 3D model. Through an efficient automated process, accurate facial feature extraction, meticulous model optimization, and flexible adaptability, this application significantly improves the quality and efficiency of 3D digital human model production, enabling the digital human model to meet the requirements of rapid loading and stable operation on different devices.

[0130] As Figure 4 shown, it is a functional module diagram of an industrial production device based on a digital human base model provided by an embodiment of this application.

[0131] In some embodiments, the industrial production device 2 based on the digital human base model may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the industrial production device 2 based on the digital human base model can be stored in the memory of the server and executed by at least one processor to execute (see details in Figure 1 the description) the functions of the industrial production method based on the digital human base model.

[0132] In this embodiment, the industrial production device 2 based on the digital human schema can be divided into multiple functional modules according to the functions it performs. The functional modules may include: a region extraction module 21, a schema selection module 22, a feature extraction module 23, a mapping construction module 24, a detail enhancement module 25, a structure optimization module 26, and an optimization verification module 27. The module referred to in the present invention means a series of computer program segments that can be executed by at least one processor and can complete fixed functions, and are stored in the memory. In this embodiment, the functions of each module will be described in detail in the subsequent embodiments.

[0133] The region extraction module 21 is used to extract the facial region of the target person from the input image to obtain the target region image.

[0134] The schema selection module 22 is used to perform gender recognition on the target region image to obtain the basic model type label, and select a preset basic model according to the basic model type label.

[0135] The feature extraction module 23 is used to extract facial feature point coordinates from the target region image according to a preset coordinate recognition model to obtain a facial feature point coordinate set.

[0136] The mapping construction module 24 is used to map the facial feature point coordinate set to the basic model according to a preset feature matching method to obtain the first three-dimensional model.

[0137] In an alternative embodiment, the mapping construction module 24 is specifically used for:

[0138] Identifying the standard feature points in the basic model, and comparing the standard feature points with the facial feature point coordinate set to extract the target facial feature point coordinates;

[0139] Preliminarily aligning the target facial feature point coordinates with the corresponding standard feature points to obtain preliminary corresponding data;

[0140] According to a preset affine transformation model and the preliminary corresponding data, preliminarily aligning the facial feature point coordinate set with the basic model to obtain a preliminary alignment coordinate point set;

[0141] Performing thin plate spline deformation calculation on the preliminary alignment coordinate point set to obtain the surface mapping parameters of each preliminary alignment coordinate point;

[0142] According to the preliminary alignment coordinate point set and the surface mapping parameters, performing one-to-one mapping on the standard feature points and the facial feature point coordinates to obtain the first three-dimensional model.

[0143] The detailed enhancement module 25 is configured to perform target detailed enhancement on the first three-dimensional model according to a preset geometric shape adjustment strategy to obtain a second three-dimensional model.

[0144] In an alternative embodiment, the detailed enhancement module 25 is specifically configured to:

[0145] Perform preliminary adjustment on the first three-dimensional model according to a preset sparse matrix and the facial feature point coordinate set to obtain a primary deformation model;

[0146] Identify key regions of the primary deformation model to obtain a target key region coordinate set;

[0147] Obtain deformation feature positioning data according to a preset Laplacian deformation model and the target key region coordinate set;

[0148] Adjust the primary deformation model according to the deformation feature positioning data to obtain a high-level deformation model;

[0149] Perform facial contour and depth optimization on the high-level deformation model according to a preset geometric optimization model to obtain the second three-dimensional model.

[0150] The structure optimization module 26 is configured to perform structure optimization on the second three-dimensional model according to a preset model optimization strategy to obtain a third three-dimensional model.

[0151] In an alternative embodiment, the structure optimization module 26 is specifically configured to:

[0152] Identify key regions of the second three-dimensional model and perform topological routing optimization on the key regions to obtain a facial topological structure set;

[0153] Perform local smoothing on the second three-dimensional model according to the facial topological structure set to obtain a facial geometry model;

[0154] Perform geometric surface merging on the facial geometry model according to a preset facial optimization strategy to obtain a face number optimization model;

[0155] Perform vertex data analysis on the face number optimization model to obtain simplified target vertex pairs, and perform vertex simplification according to the simplified target vertex pairs to obtain a vertex optimization model;

[0156] Perform resolution level division on the vertex optimization model to obtain third three-dimensional models applied to different resolutions.

[0157] In an alternative embodiment, the structure optimization module 26 is further configured to:

[0158] Step 51: Analyze the patch attributes of the facial geometry model to obtain the first attribute information set of each patch;

[0159] Step 52: Obtain the first geometric error between each vertex and its adjacent patches according to the first attribute information set and a preset 3D network simplification calculation formula;

[0160] Step 53: When the first geometric error is greater than a preset error threshold, calculate the second geometric error between adjacent vertices according to the first geometric error, and screen out the merging target vertex pairs according to the second geometric error;

[0161] Step 54: Perform geometric patch merging according to the merging target vertex pairs to generate an optimized facial geometry model, and update the first attribute information set according to the optimized facial geometry model;

[0162] Repeat steps 52 to 54 until the first geometric error is equal to the error threshold to obtain the face number optimized model.

[0163] In an optional embodiment, the structure optimization module 26 is further configured to:

[0164] Step 61: Obtain the number of vertices of each patch in the face number optimized model, compare the number of vertices of each patch with a corresponding vertex number threshold, and determine the patch as a target simplified patch when the number of vertices is greater than the vertex number threshold;

[0165] Step 62: Analyze the vertex attributes of the target simplified patch to obtain the second attribute information set of each vertex;

[0166] Step 63: Perform vertex data analysis according to the second attribute information set to obtain a target simplified vertex set;

[0167] Step 64: Perform a simplification evaluation on each vertex in the target simplified vertex set to obtain the simplified target vertex pairs;

[0168] Step 65: Merge adjacent vertices according to the target simplified vertex pairs, and update the face number optimized model according to the merged vertices;

[0169] Repeat steps 61 to 65 until the number of vertices of each patch in the face number optimized model is equal to the corresponding vertex number threshold to obtain the vertex optimized model.

[0170] In an optional embodiment, the industrial production device 2 based on the digital human base model further includes an optimization verification module 27. After performing step 65, the optimization verification module 27 is specifically configured to:

[0171] Perform key area recognition on the updated face number optimization model to evaluate the geometric shape of the key area;

[0172] When it is detected that the geometric shape details of the key area are lost, restore the patches corresponding to the key area according to the simplified target vertices in step 64, and prohibit performing step 61 on the patches.

[0173] It should be understood that the various variations and specific embodiments in the methods provided in the above embodiments are equally applicable to the industrial production device based on the digital human base model in this embodiment. Through the foregoing detailed description of the industrial production method based on the digital human base model, those skilled in the art can clearly know the implementation method of the industrial production device based on the digital human base model in this embodiment. For the sake of brevity of the specification, it will not be elaborated herein.

[0174] As Figure 5 shown, it is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0175] In a preferred embodiment of the present invention, the electronic device 3 may include, but is not limited to: a memory 31, at least one processor 32, and at least one communication bus 33.

[0176] Those skilled in the art should understand that Figure 5 the structure of the electronic device 3 shown does not constitute a limitation on the embodiments of the present invention. The electronic device 3 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0177] In some embodiments, the electronic device 3 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits, programmable gate arrays, digital processors, and embedded devices, etc.

[0178] It should be noted that the electronic device 3 is only an example, and other existing or future possible electronic products that can be adapted to the present application should also be included in the protection scope of the present application and are included herein by reference.

[0179] In some embodiments, a computer program is stored in the memory 31, and when the computer program is executed by the at least one processor 32, all or part of the steps in the industrial production method based on the digital human schema are implemented. The memory 31 includes a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc memories, magnetic disk memories, tape memories, or any other computer-readable medium capable of carrying or storing data. Further, the computer-readable storage medium mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, and the like.

[0180] In some embodiments, the at least one processor 32 is the control core (Control Unit) of the electronic device 3, connecting various components of the entire electronic device 3 through various interfaces and circuits. By running or executing the programs or modules stored in the memory 31, and calling the data stored in the memory 31, various functions of the electronic device 3 are executed and data is processed. For example, when the at least one processor 32 executes the computer program stored in the memory 31, all or part of the steps in the industrial production method based on the digital human schema described in the embodiments of the present application are implemented; or all or part of the functions of the industrial production device based on the digital human schema are implemented. The at least one processor 32 may be composed of integrated circuits. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same or different functions packaged together, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0181] In some embodiments, the at least one communication bus 33 is arranged to implement the connection communication between the memory 31 and the at least one processor 32, etc. Although not shown, the electronic device 3 may further include a power supply (such as a battery) for powering each component. Preferably, the power supply may be logically connected to the at least one processor 32 through a power management device, so as to implement functions such as management of charging, discharging, and power consumption management through the power management device. The power supply may further include any components such as one or more DC or AC power supplies, a recharge device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device 3 may further include various sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be elaborated herein.

[0182] The integrated units implemented in the form of software function modules as described above may be stored in a computer-readable storage medium. The above software function modules are stored in a storage medium and include several instructions for causing an electronic device (which may be a personal computer, an electronic device, or a network device, etc.) or a processor to execute a part of the methods described in the various embodiments of the present application.

[0183] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0184] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units. They may be located in one place or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereby. Therefore, all equivalent changes made according to the structure, shape, and principle of the present application should be covered within the protection scope of the present application.

Claims

1. An industrial production method based on a digital human schema, characterized in that, The method includes: Extracting the facial region of the target person from the input image to obtain a target region image; Performing gender recognition on the target region image to obtain a basic model type label, and selecting a preset basic model according to the basic model type label; Extracting facial feature point coordinates from the target region image according to a preset coordinate recognition model to obtain a facial feature point coordinate set; Mapping the facial feature point coordinate set to the basic model according to a preset feature matching method to obtain a first three-dimensional model; Performing target detail enhancement on the first three-dimensional model according to a preset geometric shape adjustment strategy to obtain a second three-dimensional model; Performing structural optimization on the second three-dimensional model according to a preset model optimization strategy to obtain a third three-dimensional model.

2. The industrial production method based on the digital human schema according to claim 1, characterized in that, The step of mapping the facial feature point coordinate set to the basic model according to a preset feature matching method to obtain a first three-dimensional model includes: Identifying standard feature points in the basic model, and comparing the standard feature points with the facial feature point coordinate set to extract target facial feature point coordinates; Performing preliminary alignment of the target facial feature point coordinates with the corresponding standard feature points to obtain preliminary corresponding data; According to a preset affine transformation model and the preliminary corresponding data, performing preliminary alignment of the facial feature point coordinate set with the basic model to obtain a preliminary alignment coordinate point set; Performing thin plate spline deformation calculation on the preliminary alignment coordinate point set to obtain surface mapping parameters for each preliminary alignment coordinate point; According to the preliminary alignment coordinate point set and the surface mapping parameters, performing one-by-one mapping of the standard feature points and the facial feature point coordinates to obtain the first three-dimensional model.

3. The industrial production method based on the digital human schema according to claim 1, wherein, The step of performing target detail enhancement on the first three-dimensional model according to a preset geometric shape adjustment strategy to obtain a second three-dimensional model includes: Performing preliminary adjustment on the first three-dimensional model according to a preset sparse matrix and the facial feature point coordinate set to obtain a primary deformation model; Performing key region recognition on the primary deformation model to obtain a target key region coordinate set; Obtaining deformation feature localization data according to a preset Laplace deformation model and the target key region coordinate set; Adjusting the primary deformation model according to the deformation feature localization data to obtain an advanced deformation model; Performing facial contour and depth optimization on the advanced deformation model according to a preset geometric optimization model to obtain the second three-dimensional model.

4. The industrial production method based on the digital human schema according to claim 1, characterized in that, The step of performing structural optimization on the second three-dimensional model according to a preset model optimization strategy to obtain a third three-dimensional model includes: Performing key region recognition on the second three-dimensional model, and optimizing the topological wiring of the key region to obtain a facial topological structure set; Performing local smoothing on the second three-dimensional model according to the facial topological structure set to obtain a facial geometric shape model; Performing geometric surface merging on the facial geometric shape model according to a preset facial optimization strategy to obtain a face number optimized model; Perform vertex data analysis on the face number optimization model to obtain simplified target vertex pairs, and perform vertex simplification according to the simplified target vertex pairs to obtain a vertex optimization model; Perform resolution level division on the vertex optimization model to obtain a third 3D model applied to different resolutions.

5. The industrial production method based on the digital human schema according to claim 4, wherein The geometric face merging of the facial geometry model according to a preset facial optimization strategy to obtain a face number optimization model includes: Step 51: Perform patch attribute analysis on the facial geometry model to obtain a first attribute information set for each patch; Step 52: Obtain a first geometric error between each vertex and its adjacent patches according to the first attribute information set and a preset 3D network simplification calculation formula; Step 53: When the first geometric error is greater than a preset error threshold, calculate a second geometric error between adjacent vertices according to the first geometric error, and filter out merged target vertex pairs according to the second geometric error; Step 54: Perform geometric face merging according to the merged target vertex pairs to generate an optimized facial geometry model, and update the first attribute information set according to the optimized facial geometry model; Repeat steps 52 to 54 until the first geometric error is equal to the error threshold to obtain the face number optimization model.

6. The industrial production method based on the digital human schema according to claim 4, wherein The performing vertex data analysis on the face number optimization model to obtain simplified target vertex pairs, and performing vertex simplification according to the simplified target vertex pairs to obtain a vertex optimization model includes: Step 61: Obtain the number of vertices of each patch in the face number optimization model, and compare the number of vertices of each patch with a corresponding vertex number threshold. When the number of vertices is greater than the vertex number threshold, determine the patch as a target simplified patch; Step 62: Perform vertex attribute analysis on the target simplified patch to obtain a second attribute information set for each vertex; Step 63: Perform vertex data analysis according to the second attribute information set to obtain a target simplified vertex set; Step 64: Perform a simplification evaluation on each vertex in the target simplified vertex set to obtain the simplified target vertex pairs; Step 65: Perform adjacent vertex merging according to the target simplified vertex pairs, and update the face number optimization model according to the merged vertices; Repeat steps 61 to 65 until the number of vertices of each patch in the face number optimization model is equal to the corresponding vertex number threshold to obtain the vertex optimization model.

7. The industrial production method based on the digital human schema according to claim 6, wherein After performing step 65, the method further includes: Identifying key regions of the updated face number optimization model to evaluate the geometric shapes of the key regions; When it is detected that the geometric shape details of the key region are lost, restore the patches corresponding to the key region according to the simplified target vertex pairs in step 64, and prohibit performing step 61 on the patches.

8. An industrial production device based on a digital human schema, characterized in that, The device includes: A region extraction module, configured to extract the facial region of the target person from the input image to obtain a target region image; A base model selection module, configured to perform gender recognition on the target region image to obtain a base model type label, and select a preset base model according to the base model type label; A feature extraction module, configured to extract facial feature point coordinates from the target region image according to a preset coordinate recognition model to obtain a facial feature point coordinate set; A mapping construction module, configured to map the facial feature point coordinate set to the base model according to a preset feature matching method to obtain a first three-dimensional model; A detail enhancement module, configured to perform target detail enhancement on the first three-dimensional model according to a preset geometric shape adjustment strategy to obtain a second three-dimensional model; A structure optimization module, configured to perform structure optimization on the second three-dimensional model according to a preset model optimization strategy to obtain a third three-dimensional model.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the industrial production method based on the digital human base model according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the industrial production method based on the digital human base model according to any one of claims 1 to 7 are implemented.

Images