3D digital human intelligent face pinching system based on photos

Abstract

The invention provides a photo-based 3D digital human intelligent face pinching system, which comprises a digital human generation model, a human face segmentation model, a human face recognition model and face alignment, and is characterized in that a working method of the system comprises the following steps of: randomly generating a digital human image and corresponding face pinching parameters through the face pinching system, and taking the data as a data set; then training a deep convolutional generative adversarial neural network as a digital human generation model; based on a Res-Net50 deep residual network, a Helen face data set is used for continuous pre-training, and the Helen face data set is used as a face segmentation model of the system. According to the invention, through the digital human generation model, the human face segmentation model, the human face recognition model and the face alignment, the user can input the photo to carry out intelligent face pinching, the learning cost of the user is reduced, the digital human customization demand of the user can be quickly met, and the user can quickly customize the digital human of a related business scene, so that the user experience is improved. And the application range of the digital human is greatly expanded.

Description

Technical Field

[0001] The present invention relates to the technical field of face pinching systems, and in particular to a photo-based 3D digital human intelligent face pinching system. Background Art

[0002] With the development of games, digital twins and other technologies in recent years, three-dimensional digital human technology has become increasingly mature, which has led to the emergence of many unique 3D digital human products on the market, including intelligent digital human products combined with large model technology. As the number of users of 3D digital human products continues to grow, the customization needs of each user continue to increase, but the existing 3D digital human products have the following two problems: First, the customization cost is high. Most 3D digital humans are built manually, which makes it difficult to quickly and cost-effectively solve the customization needs of users; second, a small number of 3D digital human products have a face-pinching system, but there are many face-pinching parameters, and users have a certain learning cost, which also limits the applicability of 3D digital humans.

[0003] To this end, the present invention provides a photo-based 3D digital human intelligent face pinching system. Summary of the invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a photo-based 3D digital human intelligent face pinching system to solve the problems raised in the above-mentioned background technology. The present invention allows users to input photos for intelligent face pinching, which reduces the user's learning cost and can quickly solve the user's digital human customization needs. Users can quickly customize digital humans for related business scenarios, greatly enhancing the application of digital humans; users can quickly get started and can modify digital humans at any time, reducing the user's usage cost and trial and error cost.

[0005] In order to achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions: a photo-based 3D digital human intelligent face pinching system, including a digital human generation model, a face segmentation model, a face recognition model and face alignment, characterized in that the system works as follows: S1. Randomly generate digital human images and corresponding face-pinching parameters through the face-pinching system, use these data as a data set, and then train a deep convolutional generative adversarial neural network as a digital human generation model; S2, based on the Res-Net50 deep residual network, use the Helen face dataset for further pre-training as the face segmentation model of this system; S3. Use LightCNN-29-V2 to implement a large number of facial feature recognition functions, so that the input image can generate a feature vector for face matching, and use this model as the face recognition model of this system; S4. Use the dlib face key point detection and alignment library to process the input image and align the face image through facial key point detection; S5. We specially formulate relevant loss functions for face segmentation model and face recognition model, and use the back-propagation iterative loss function to guide the generative model to predict continuous facial parameters. S6. After the training is completed, the user selects the gender of the digital human, sets basic parameters, and then uploads the photo on the interactive interface; S7. The face pinching parameters are output through the neural network, and the face pinching system generates a corresponding 3D digital human according to the face pinching parameters.

[0006] Furthermore, the face pinching parameters include multiple parameters such as face, eyes, nose, mouth, ears, etc.

[0007] Furthermore, the randomly generated digital human images include at least 20,000 images.

[0008] Furthermore, the Helen face dataset is used for further pre-training to separate the face area in the image from the background.

[0009] Furthermore, face alignment is used to make the face image more standardized, aiming to combine the real photo input by the user with the input image to be iterated below.

[0010] Furthermore, facial parameters are optimized to maximize the similarity between the digital human image and the real face.

[0011] Furthermore, when the user inputs a photo into the 3D digital human intelligent face-pinching system, after alignment and iteration, a set of corresponding face-pinching parameters will be output to the system, and the user's customized digital human will be successfully generated.

[0012] Furthermore, the following requirements apply to photos uploaded by users: the picture must contain facial information of the person, and must be a frontal, unobstructed, evenly lit picture without glasses, and with no exposed teeth.

[0013] Furthermore, the face-pinching system can output parameter information of the picture based on the picture uploaded by the user, and the user can fine-tune the corresponding parameters.

[0014] Furthermore, the response time of the entire system is within 20 seconds.

[0015] Beneficial effects of the present invention: The present invention provides a photo-based 3D digital human intelligent face pinching system, comprising a digital human generation model; a face segmentation model; a face recognition model; and face alignment.

[0016] Through the digital human generation model, face segmentation model, face recognition model and face alignment, users can input photos for intelligent face pinching, which reduces the user's learning cost and can quickly solve the user's digital human customization needs. Users can quickly customize digital humans for related business scenarios, which greatly enhances the application of digital humans. The response time of the entire system is within 20 seconds, and the response speed is fast. Users can get started quickly and can modify the digital human at any time, reducing the user's usage cost and trial and error cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a photo-based 3D digital human intelligent face pinching system of the present invention; DETAILED DESCRIPTION

[0018] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0019] See also Figure 1 The present invention provides a technical solution: a photo-based 3D digital human intelligent face pinching system, a digital human generation model, a face segmentation model, a face recognition model and a face alignment, characterized in that the system works as follows: S1. Randomly generate digital human images and corresponding face-pinching parameters through the face-pinching system, use these data as a data set, and then train a deep convolutional generative adversarial neural network as a digital human generation model; S2, based on the Res-Net50 deep residual network, use the Helen face dataset for further pre-training as the face segmentation model of this system; S3. Use LightCNN-29-V2 to implement a large number of facial feature recognition functions, so that the input image can generate a feature vector for face matching, and use this model as the face recognition model of this system; S4. Use the dlib face key point detection and alignment library to process the input image and align the face image through facial key point detection; S5. We specially formulate relevant loss functions for face segmentation model and face recognition model, and use the back-propagation iterative loss function to guide the generative model to predict continuous facial parameters. S6. After the training is completed, the user selects the gender of the digital human, sets basic parameters, and then uploads the photo on the interactive interface; S7. The face pinching parameters are output through the neural network, and the face pinching system generates a corresponding 3D digital human according to the face pinching parameters.

[0020] In this embodiment, the face pinching parameters include multiple parameters such as face, eyes, nose, mouth, ears, etc.

[0021] Specifically, users can change and save the image of the digital human in real time by adjusting multiple parameters such as face, eyes, nose, mouth, and ears. They can also generate digital human information in one step through the 3D digital human intelligent face pinching system, directly generating multiple parameters such as face, eyes, nose, mouth, and ears, eliminating the need for manual adjustment by users and reducing user learning costs.

[0022] The randomly generated digital human images include at least 20,000 images.

[0023] Specifically, the face pinching system is used to randomly generate at least 20,000 pictures of random digital human images and corresponding face pinching parameters. These data are then used as a data set to train a deep convolutional generative adversarial neural network as a digital human generation model for 3D digital human generation.

[0024] Use the Helen face dataset for further pre-training to separate the face area in the image from the background.

[0025] Specifically, based on the Res-Net50 deep residual network, the Helen face dataset is used for further pre-training to enable it to have excellent face segmentation capabilities, that is, to separate the face area in the image from the background, as the face segmentation model of this system.

[0026] Face alignment is used to standardize face images, aiming to combine the real photos input by the user with the input map to be iterated below.

[0027] Specifically, the dlib facial key point detection and alignment library is used to process the input image. Through facial key point detection, the face image can be aligned to make the face image more standardized; the purpose is to make the real photo input by the user conform to the input image to be iterated below.

[0028] By optimizing facial parameters and maximizing the similarity between the digital human image and the real face.

[0029] Specifically, relevant loss functions are specially formulated for the face segmentation model and the face recognition model. The loss function is iterated through backpropagation to guide the generative model to predict continuous facial parameters, optimize the facial parameters and maximize the similarity between the created digital human image and the real face.

[0030] Furthermore, the user inputs a photo into the 3D digital human intelligent face-pinching system, and after alignment and iteration, a set of corresponding face-pinching parameters will be output to the system, and the user's customized digital human will be successfully generated. The user's uploaded photo includes the following requirements: the picture must contain the person's facial information, and it must be a frontal face, unobstructed, evenly lit, without glasses, and without missing teeth. The face-pinching system can output the picture's parameter information based on the picture uploaded by the user, and the user can fine-tune the corresponding parameters. The response time of the entire system is within 20 seconds.

[0031] Workflow: The user selects the gender of the digital person to be generated and sets basic parameters. Then the user uploads a picture through the interactive interface. The picture needs to contain the facial information of the person and needs to be a frontal face, unobstructed, evenly lit, without glasses, and without missing teeth. Otherwise, it will have a certain impact on the generation result. Finally, the trained neural network will output a set of face-pinching parameters, and the face-pinching system will generate the corresponding 3D digital person for the user, thereby reducing the user's time cost of learning the face-pinching steps and the time cost of the manual face-pinching process, allowing users to get started quickly and efficiently, thereby reducing the cost of trial and error during use.

[0032] In particular, since the images of digital humans may be highly similar, users can make manual adjustments to enable the digital humans to have their own unique personal characteristics and can fine-tune the corresponding values ​​based on the image.

[0033] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A photo-based 3D digital human intelligent face pinching system, including a digital human generation model, a face segmentation model, a face recognition model and face alignment, characterized in that: The system works as follows: S1. Randomly generate digital human images and corresponding face-pinching parameters through the face-pinching system, use these data as a data set, and then train a deep convolutional generative adversarial neural network as a digital human generation model; S2, based on the Res-Net50 deep residual network, use the Helen face dataset for further pre-training as the face segmentation model of this system; S3. Use LightCNN-29-V2 to implement a large number of facial feature recognition functions, so that the input image can generate a feature vector for face matching, and use this model as the face recognition model of this system; S4. Use the dlib face key point detection and alignment library to process the input image and align the face image through facial key point detection; S5. We specially formulate relevant loss functions for face segmentation model and face recognition model, and use the back-propagation iterative loss function to guide the generative model to predict continuous facial parameters. S6. After the training is completed, the user selects the gender of the digital human, sets basic parameters, and then uploads the photo on the interactive interface; S7. The face pinching parameters are output through the neural network, and the face pinching system generates a corresponding 3D digital human according to the face pinching parameters.

2. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: The face pinching parameters include face, eyes, nose, mouth, ears and other parameters.

3. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: The randomly generated digital human images include at least 20,000 images.

4. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: Use the Helen face dataset for further pre-training to separate the face area in the image from the background.

5. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: Face alignment is used to standardize face images, aiming to combine the real photos input by the user with the input map to be iterated below.

6. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: By optimizing facial parameters and maximizing the similarity between the digital human image and the real face.

7. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: The user inputs a photo into the 3D digital human intelligent face-pinching system, which, after alignment and iteration, outputs a set of corresponding face-pinching parameters to the system, and the user's customized digital human is successfully generated.

8. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: The following requirements apply to photos uploaded by users: the picture must contain the person's facial information and must be a frontal, unobstructed, evenly lit picture without glasses and without showing any teeth.

9. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: The face-pinching system can output the parameter information of the picture based on the picture uploaded by the user, and the user can fine-tune the corresponding parameters.

10. The photo-based 3D digital human intelligent face pinching system according to claim 1, characterized in that: The response time of the entire system is within 20 seconds.

Images