Business card production method, system, medium and device based on virtual digital human technology

Abstract

The present invention provides a business card production method, system, medium and device based on virtual digital human technology, which relates to the field of artificial intelligence. The method includes: uploading a face image to a virtual digital human production platform; generating a virtual digital human based on the face image based on the virtual digital human technology; generating a dynamic portrait based on the face image based on the dynamic portrait generation technology; generating a virtual digital human business card by combining the virtual digital human and the dynamic portrait; using speech synthesis technology to convert the user's text information into speech information, adding it to the virtual digital human business card, and producing a virtual digital human business card with speech function. The present invention can reduce the production and update costs of virtual digital human business cards and meet the personalized needs of users.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a business card making method, system, medium and equipment based on virtual digital human technology. Background Art

[0002] At present, traditional digital business card technology already exists and has been widely used. Traditional digital business card technology refers to the technology that converts paper business cards into digital form, which is convenient for transmission and sharing on mobile phones, computers and other devices. This technology usually uses QR code, NFC, Bluetooth and other technologies to store digital business cards in the device, which is convenient for users to view and share at any time. Digital business cards can contain basic information of individuals or companies, such as name, position, phone number, email address, company name, address, etc., and can also add personal or corporate introductions, product or service information, etc. The advantages of traditional digital business card technology are that it is convenient and fast, and can be transmitted and shared anytime and anywhere, while also reducing environmental pollution and paper use.

[0003] Traditional digital business card technology transmits and shares digital business cards stored in the device through QR codes, NFC, Bluetooth and other technologies during use, making it convenient for users to view and share at any time. When using a mobile phone to share digital business cards, it can be achieved by installing the corresponding Android App or applet. For example, WeChat, Alipay, QQ and other applications provide digital business card functions, and users can share their digital business cards with other users through these applications. In addition, there are also some special digital business card management applications, such as CamCard, CamCard, etc. These applications can help users manage and organize digital business cards, and provide some additional functions, such as OCR recognition, scanning business cards, etc. Using these applications can make the management of digital business cards more convenient and quick, and can also improve the readability and interactivity of digital business cards.

[0004] However, the existing technology of traditional digital business card production has many shortcomings and deficiencies. Figure 1 Create a flow chart for traditional digital business cards, such as Figure 1 As shown in the figure, the recognition accuracy is not high during the scanning and recognition stage. During the scanning and text recognition process, due to the large differences in business card formats, fonts, colors, etc., coupled with the influence of factors such as light and scanners, it is difficult to ensure recognition accuracy. This will result in incomplete, missing or erroneous information, such as the lack of certain key information or recognition errors, which will affect the accuracy and practicality of the business card. Traditional digital business cards lack personalization and interactivity based on their production process: Since they are based on existing paper business cards, they cannot provide richer and more vivid introductions and communication methods, and it is difficult to meet the personalized needs of users.

[0005] In addition, the production and design costs of traditional digital business cards are high, requiring certain professional skills and software support, and the production and design costs are high for non-professionals. In addition, traditional digital business cards need to be printed and produced, which will produce environmental pollution such as waste paper and waste ink, and also consume resources and costs. Once the traditional digital business cards are produced, they are difficult to modify and update. If there is new information or demand, they need to be re-produced, which is costly and time-consuming. Summary of the invention

[0006] The purpose of the present invention is to provide a business card production method, system, medium and device based on virtual digital human technology to solve the problems of high production and update costs, low recognition accuracy and difficulty in meeting users' personalized needs in traditional digital business cards.

[0007] To achieve the above object, the present invention provides the following solution.

[0008] A business card making method based on virtual digital human technology includes the following steps.

[0009] Upload the facial image to the virtual digital human production platform.

[0010] Based on the virtual digital human technology, a virtual digital human is generated according to the facial image.

[0011] Based on the dynamic portrait generation technology, a dynamic portrait is generated according to the face image.

[0012] The virtual digital human and the dynamic portrait are combined to generate a virtual digital human business card.

[0013] By using speech synthesis technology, the user's text information is converted into voice information and added to the virtual digital person business card to create a virtual digital person business card with voice function; the text information includes name, position, unit and address.

[0014] Optionally, uploading the facial image to the virtual digital human production platform also includes: taking facial images of the user from multiple angles; the multi-angle facial images include a frontal photo, a photo with eyes open, a photo with eyes closed, a photo with mouth open, a photo with mouth closed, a photo of the face tilted 45 degrees to the left, a photo of the face tilted 45 degrees to the right, and a photo of the face nodding and looking up.

[0015] Optionally, based on the virtual digital human technology, a virtual digital human is generated according to the facial image, specifically comprising: performing template mapping on the facial images at multiple angles, and using a deep learning algorithm to extract facial detail features in the facial images at each angle; the facial detail features include an identification of the facial state; calling a facial splicing template of the corresponding gender, aligning and splicing facial detail features at multiple angles, and generating a facial detail image containing multiple angles; the facial detail image includes facial contours, facial expressions, eye shapes, eye colors, eye sizes, and the relative positions of eyes and face; determining facial key points according to the facial detail image; the facial key points include the shape and size of various facial organs, the relative positions of various facial organs and face, and the hair and hair accessories contained in the face; the organs include a nose, a mouth, and a chin; based on the facial key points, converting the facial detail image into three-dimensional facial reconstruction data, and constructing a three-dimensional facial model using a three-dimensional reconstruction algorithm; adding a skeletal system and animation control to the three-dimensional facial model to generate a virtual digital human model; optimizing the virtual digital human model to generate a virtual digital human.

[0016] Optionally, a face stitching template corresponding to the gender is called to align and stitch facial detail features at multiple angles to generate a face detail image containing multiple angles, specifically including: judging the gender of the user corresponding to the face image; when the user gender is male, calling the male face stitching template to align and stitch facial detail features at multiple angles to generate a face detail image containing multiple angles; when the user gender is female, calling the female face stitching template to align and stitch facial detail features at multiple angles to generate a face detail image containing multiple angles; sending the face detail image to the user; if the user is satisfied, determining that the face detail image is correct; if the user is not satisfied, reselecting the face stitching template.

[0017] Optionally, based on the facial key points, the facial detail image is converted into three-dimensional facial reconstruction data, and a three-dimensional facial model is constructed using a three-dimensional reconstruction algorithm, specifically including: based on the facial key points, the facial detail image is converted into point cloud data; the point cloud data is three-dimensional facial reconstruction data; the point cloud data of different angles and positions are aligned to generate aligned point cloud data; the aligned point cloud data is converted into gridding to generate a gridding result; and based on the aligned point cloud data and the gridding result, a three-dimensional facial model is constructed using a three-dimensional reconstruction algorithm.

[0018] Optionally, the virtual digital human model is optimized to generate a virtual digital human, specifically comprising: performing texture optimization on the virtual digital human model using a synthesis algorithm based on texture mapping to generate a texture-optimized virtual digital human model; performing topological optimization on the morphological structure of the texture-optimized virtual digital human model using a topological reconstruction algorithm based on topology preservation and a topological reconstruction algorithm based on morphology to generate a virtual digital human.

[0019] Optionally, based on virtual digital human technology, a virtual digital human is generated according to the face image, and then the method further includes: rendering the virtual digital human to generate a virtual human dynamic pattern.

[0020] A business card making system based on virtual digital human technology includes the following modules.

[0021] The image upload module is used to upload the face image to the virtual digital human production platform.

[0022] The virtual digital human generation module is used to generate a virtual digital human according to the face image based on the virtual digital human technology.

[0023] The dynamic portrait generation module is used to generate a dynamic portrait according to the face image based on the dynamic portrait generation technology.

[0024] The virtual digital person business card generation module is used to generate a virtual digital person business card by combining the virtual digital person and the dynamic portrait.

[0025] The module for making a virtual digital person business card with voice function is used to convert the user's text information into voice information by using speech synthesis technology, add it to the virtual digital person business card, and make a virtual digital person business card with voice function; the text information includes name, position, unit and address.

[0026] An electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned business card making method based on virtual digital human technology.

[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned business card making method based on virtual digital human technology.

[0028] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the embodiments of the present invention produce virtual digital human business cards based on the user's facial image, without the need to introduce paper business cards, avoiding the problem of text recognition, and adding voice functions. When the virtual digital human business card needs to be updated, the user's text information is converted into voice information and directly added to the virtual digital human business card, reducing the production and update costs of the virtual digital human business card. In addition, the embodiments of the present invention generate virtual digital humans and dynamic portraits based on virtual digital human technology and dynamic portrait generation technology, and then generate virtual digital human business cards to meet the personalized needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 Create a flow chart for traditional virtual digital human business cards.

[0031] Figure 2 This is a flow chart of the business card production method based on virtual digital human technology.

[0032] Figure 3 This is a technical association diagram.

[0033] Figure 4 This is a data interaction flow chart of the business card production method based on virtual digital human technology.

[0034] Figure 5 Create a functional diagram of the platform for virtual digital humans.

[0035] Figure 6 This is the image processing flow chart.

[0036] Figure 7 Flowchart of face image stitching.

[0037] Figure 8 Schematic diagram of the dimensionality reduction effect of face splicing data.

[0038] Fig. 9 Generate algorithm flow chart for 3D digital human.

[0039] Fig.10 This is a schematic diagram of a virtual digital person business card.

[0040] Fig.11 This is the TTS technology flow chart. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The purpose of the present invention is to provide a business card production method, system, medium and device based on virtual digital human technology, which can reduce the production and update costs of virtual digital human business cards and meet the personalized needs of users.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Embodiment 1: Figure 2-Figure 4 As shown, the present invention provides a business card making method based on virtual digital human technology, which includes the following steps.

[0045] Step 1: Upload the face image to the virtual digital human production platform.

[0046] In actual application, the facial image is uploaded to the virtual digital human production platform, which also includes: taking facial images of the user from multiple angles; the multi-angle facial images include frontal photos, photos with eyes open, photos with eyes closed, photos with mouth open, photos with mouth closed, photos of the face tilted 45 degrees to the left, photos of the face tilted 45 degrees to the right, and photos of the face nodding and looking up.

[0047] Step 2: Based on the virtual digital human technology, a virtual digital human is generated according to the facial image.

[0048] In practical applications, based on the virtual digital human technology, a virtual digital human is generated according to the face image, specifically including: performing template mapping on the face images at multiple angles, and using a deep learning algorithm to extract face detail features in the face images at each angle; the face detail features include an identifier of the face state; calling a face splicing template corresponding to the gender, aligning and splicing face detail features at multiple angles, and generating a face detail image containing multiple angles; the face detail image includes facial contours, facial expressions, eye shapes, eye colors, eye sizes, and the relative positions of eyes and face; determining face key points according to the face detail image; the face key points include the shape and size of various facial organs, the relative positions of various facial organs and face, and the hair and hair accessories contained in the face; the organs include a nose, a mouth, and a chin; based on the face key points, converting the face detail image into face three-dimensional reconstruction data, and constructing a face three-dimensional model using a three-dimensional reconstruction algorithm; adding a skeletal system and animation control to the face three-dimensional model to generate a virtual digital human model; optimizing the virtual digital human model to generate a virtual digital human.

[0049] In actual applications, a face stitching template corresponding to the gender is called to align and stitch facial detail features at multiple angles to generate a face detail image containing multiple angles, which specifically includes: judging the gender of the user corresponding to the face image; when the user gender is male, calling the male face stitching template to align and stitch facial detail features at multiple angles to generate a face detail image containing multiple angles; when the user gender is female, calling the female face stitching template to align and stitch facial detail features at multiple angles to generate a face detail image containing multiple angles; sending the face detail image to the user; if the user is satisfied, determining that the face detail image is correct; if the user is not satisfied, reselecting the face stitching template.

[0050] In practical applications, based on the facial key points, the facial detail image is converted into three-dimensional reconstruction data of the face, and a three-dimensional model of the face is constructed using a three-dimensional reconstruction algorithm, which specifically includes: based on the facial key points, the facial detail image is converted into point cloud data; the point cloud data is three-dimensional reconstruction data of the face; the point cloud data of different angles and positions are aligned to generate aligned point cloud data; the aligned point cloud data is converted into gridding processing to generate a gridding result; based on the aligned point cloud data and the gridding result, a three-dimensional model of the face is constructed using a three-dimensional reconstruction algorithm.

[0051] In practical applications, the virtual digital human model is optimized to generate a virtual digital human, specifically including: using a synthesis algorithm based on texture mapping to perform texture optimization on the virtual digital human model to generate a texture-optimized virtual digital human model; using a topology reconstruction algorithm based on topology preservation and a topology reconstruction algorithm based on morphology to perform topology optimization on the morphological structure of the texture-optimized virtual digital human model to generate a virtual digital human.

[0052] In practical applications, based on the virtual digital human technology, a virtual digital human is generated according to the face image, and then the method further includes: rendering the virtual digital human to generate a virtual human dynamic pattern.

[0053] Step 3: Based on the dynamic portrait generation technology, a dynamic portrait is generated according to the face image.

[0054] Step 4: Generate a virtual digital human business card by combining the virtual digital human and the dynamic portrait.

[0055] Step 5: Use speech synthesis technology to convert the user's text information into voice information, add it to the virtual digital person business card, and create a virtual digital person business card with voice function; the text information includes name, position, unit and address.

[0056] The technical terms involved in the present invention are explained as follows.

[0057] 1. Virtual digital human technology: Virtual digital human technology is a virtual digital human generated by a computer. It can simulate the appearance, movements, voice and other characteristics of a person through digital technology to achieve interaction and communication with the user. Using virtual digital human technology, the present invention can add a virtual digital human to the virtual digital human business card, making the business card more vivid and interesting.

[0058] 2. Dynamic portrait generation technology: Dynamic portrait generation technology is a technology based on machine learning and deep learning, which can automatically generate a dynamic portrait with realism and smoothness based on a photo. Using dynamic portrait generation technology, the present invention can automatically generate a dynamic portrait from a selfie photo provided by the user, making the business card more vivid and interesting.

[0059] 3. Conversion technology from flat dynamic portrait to 3D dynamic virtual human: This invention combines virtual digital human technology with dynamic portrait generation technology to propose a new virtual digital human business card production solution. The user only needs to provide a photo, and the system can automatically generate a dynamic portrait with realism and smoothness, and add it to the virtual digital human business card, realizing the technology of generating dynamic business cards from a selfie photo.

[0060] 4. Speech synthesis technology: Adding speech synthesis technology, namely text-to-speech (TTS) technology, to virtual digital business cards can make business cards more vivid and intelligent. TTS technology can convert text content into natural language speech output, allowing users to obtain information by listening to voice, which is more convenient and easy; automatically broadcast voice introductions, and users can click the voice button on the business card to listen to the voice introduction of the business card owner, enhancing the interactivity and personalization of the business card.

[0061] The present invention adopts the combination of the above four technologies to achieve a high degree of personalization and vividness of virtual digital human business cards, making them more attractive and practical. At the same time, due to the use of digital technology, the present invention also avoids the problems of environmental pollution and waste of resources in the production of traditional virtual digital human business cards, and has better environmental protection and sustainability.

[0062] The present invention corresponds to the development of a "quick cloud-based virtual digital human production platform", which is a virtual digital human production platform.

[0063] In the actual processing, Figure 5 As shown, the business card making method based on virtual digital human technology of the present invention includes the following steps: image processing, generating a virtual digital human, generating a virtual digital human business card, and generating a virtual digital human business card with voice function.

[0064] 1) If Figure 6 As shown, image processing.

[0065] The main function of the smartphone mobile program is to realize the photo taking and uploading program. The main working steps are divided into the following three steps.

[0066] S1. Design an App or mini program on your mobile phone to realize the photo taking function for the person whose business card you want to make.

[0067] Users can start the camera through the front-end applet, and after taking photos, they can be saved in the device for subsequent photo editing.

[0068] S2. After taking photos, crop the photos according to the 600dpi resolution and 2-inch standard, and provide photo editing and processing functions such as beautification, cropping, color adjustment, etc. Specific functions include: (1) Photo cropping: Photos can be cropped into 2-inch size personal photos as needed.

[0069] (2) Color adjustment: You can adjust the brightness, contrast, saturation and other parameters of the photo to achieve better results.

[0070] (3) Add special effects: You can add various special effects and filters, such as black and white, retro, warm and cool tones, etc., to make the photos more interesting and personalized.

[0071] S3. The photos taken or edited by the user are uploaded to the Kuaishangyun virtual digital human production platform.

[0072] In terms of acquiring facial photos, users use smartphones to take photos of their faces at multiple angles in a set order of prompts. The facial photos at multiple angles include a frontal photo of the face, photos of eyes open and eyes closed, photos of mouth open and mouth closed, photos of the face tilted 45 degrees to the left, photos of the face tilted 45 degrees to the right, and photos of the face nodding and looking up, a total of 9 photos. The 9 photos can be processed in the next step to obtain a complete, user-editable 3D facial image.

[0073] After the user completes the shooting, the data will be uploaded to the "KuaiShangCloud Virtual Digital Human Production Platform" and enter the virtual digital human production process.

[0074] 2) Generate a virtual digital human.

[0075] Kuaishangyun Virtual Human Production Platform is mainly responsible for completing photo editing, virtual human SDK calling, virtual human generation, rendering and push. The main detailed production process can be broken down into Figure 4 The detailed four components include: image processing, generation of virtual digital humans, generation of virtual digital human business cards and integration of virtual digital humans with natural language processing technology.

[0076] After the 9 photos taken by the user are qualified, the conversion process of the virtual digital human begins. The photo-to-virtual digital human conversion process is the core algorithm required by this technology.

[0077] Before starting, you need to perform image preprocessing based on convolutional neural networks on the photos uploaded by the mobile phone, which mainly includes the following contents.

[0078] Step 1: Template mapping is performed on 9 photos containing facial information, including frontal photos, photos with eyes open and closed, photos with mouth open and closed, photos of the face tilted 45 degrees to the left, photos of the face tilted 45 degrees to the right, and photos of the face nodding and looking up. That is, the 9 photos correspond to 9 image processors respectively, and the details of the face are extracted from the 9 photos. The convolutional neural network algorithm (CNN) in the deep learning algorithm is used to extract facial detail features. The convolutional neural network model obtained by training this algorithm on a large-scale data set can automatically learn the fed face photos and extract discriminative facial features.

[0079] The process of training the data set using the CNN convolutional neural network algorithm is as follows.

[0080] S1: Prepare training data: prepare a large amount of face photo data, such as the 9 face photos from different angles mentioned above, and also prepare photos that do not contain or do not have face data as a control group. The combination of the above two photo sets is called the training data set.

[0081] S2: Select model type: Select the appropriate deep learning model type based on the required task and all available data. Since this task is image classification, select the convolutional neural network (CNN) model.

[0082] S3: Configure the model: During the model configuration phase, the model's hyperparameters need to be adjusted so that it fits the data as accurately as possible during the training process. These hyperparameters include learning rate, batch size, hidden layer size; these hyperparameters need to be repeatedly tested and adjusted during the model training process to obtain the best model performance.

[0083] S4: Training model: In the model training stage, the dataset in S1 is fed to the CNN convolutional neural network model to learn enough facial features and patterns.

[0084] S5: Evaluate the model: In the model evaluation phase, the accuracy of the model is evaluated using a test data set to determine whether the model needs to be further adjusted. The test set is data used to test the trained model. The test set here can be used to test a group of 9 face photos with faces from different angles. The evaluation indicators for the generated model include recognition accuracy, recall rate, and F1 score.

[0085] In summary, after the above steps, 9 photos will be processed to obtain multiple facial detail features containing facial features at various angles.

[0086] Step 2: Align and stitch the facial detail features obtained in step 1 to form a facial detail image containing multiple angles.

[0087] Among them, the facial detail features obtained in step 1 are used to identify and mark the facial states in 9 pictures using a convolutional neural network algorithm, providing a data basis for step 2; "identification and marking" is to extract the differences in the same face at different angles and movements (opening and closing the mouth and opening and closing the eyes) from the 9 pictures, with the aim of extracting data of this face at different angles.

[0088] When using the face data obtained in step 1 to stitch together data from different angles, it is necessary to first determine whether the face is male or female, and then call the face stitching templates of the corresponding gender respectively, and correspond the facial detail features of the 9 dimensions obtained in step 1 to the face templates to complete the stitching process.

[0089] The "face stitching template" is a pre-set face model, which contains multiple features of a single gender's face, such as face shape, mouth shape, nose shape, eye features and ear features. By selecting and matching the facial features and then mapping the facial detail features to the model, the basic face data can be initially generated.

[0090] The following process will be called when stitching 9 processed photos.

[0091] After the process starts, the facial details of step 1 are called in, and the gender characteristics are determined based on the personal information data entered by the user; when it is determined that the digital person to be generated is male, the male face splicing template will be directly called to splice the facial details; when it is determined that the digital person to be generated is female, the female face splicing template will be directly called to splice the facial details. After the facial details are spliced, the spliced ​​effect data will be sent to the user's mobile phone, and the user will be asked if he is satisfied. If the user is not satisfied, the face template will be reselected and regenerated. If the user is satisfied, the process ends. Figure 7 shown.

[0092] After the above process is completed, a detailed face feature mosaic image of the same gender as the user and with facial features highly similar to the user's face can be formed. This image contains all the features of the user's face, including facial contour, facial expression, eye shape, and its color, size, and relative position to the surrounding face; in addition, the shape, size, and relative position of the nose, mouth, and chin to the surrounding face; and some hair on the face, such as eyebrows, male beards, and female hair accessories. All of the above are key points of the face. After determining the information of this key point, the basis for 3D reconstruction of the face is available.

[0093] Step 3: 3D reconstruction.

[0094] After determining the key points of the face, a 3D reconstruction algorithm can be used to convert the face detail map into a 3D face model. The 3D reconstruction algorithm can usually be divided into the following steps.

[0095] (1) Point cloud generation: Convert the facial detail image into point cloud data, that is, convert each pixel in the two-dimensional image into a three-dimensional coordinate point.

[0096] (2) Point cloud registration: The point cloud data is registered, that is, the point cloud data at different angles and positions are matched to obtain a more accurate 3D shape. The point cloud registration uses the Microsoft Little Iceland SDK interface and the PointNet point cloud registration algorithm based on deep learning.

[0097] (3) Meshing: Convert the registered point cloud data into a triangular mesh for subsequent operations such as texture mapping and animation production. Meshing can use the Delaunay triangulation algorithm of the XiaoIce platform or other meshing algorithms.

[0098] (4) Model reconstruction: Based on the registered point cloud data and meshing results, call the Microsoft XiaoIce SDK interface and the 3D reconstruction algorithm to reconstruct the model.

[0099] In point cloud data processing, the principal component analysis (PCA) algorithm is used to combine with the deep learning algorithm to reduce the dimension of the relatively rough 3D face image. The data dimension reduction is to retain the most important features of the high-dimensional data and remove noise and unimportant features, so as to achieve the purpose of improving data processing speed. In this step, the principal component analysis algorithm is used to reduce the dimension of the data at the seams in the relatively rough 3D face image, so as to make this part of the data uniform. The local effect is as follows Figure 8 described.

[0100] Figure 8 In the figure, the two faces whose dimensions are reduced are A and B. Since the two reduced faces are composed of two originally independent faces, there will be a splicing boundary after forced splicing. The boundary is the gap between faces A and B. In order to eliminate the splicing boundary, the two faces need to be reduced in dimension. The right side is the effect after dimensionality reduction, and the splicing boundary is eliminated.

[0101] After the PCA algorithm is called, the Linear Discriminant Analysis (LDA) algorithm is called to fuse with the deep learning algorithm: to extract discriminative facial features. The LDA algorithm is used to project the features to maximize the distance between classes and minimize the distance within classes, thereby improving the discriminative ability of the features. The Local Binary Patterns (LBP) algorithm is called again to enhance the expression of texture features of multi-angle face 3D images. The LBP algorithm is used to extract texture from local areas of the image, and the LBP features are fused with the deep learning features to comprehensively utilize texture and depth features.

[0102] Step 4: Send the 3D face model in step 3 to the digital human processing AI model. The digital human processing AI model will add a skeletal system and animation control to the 3D face model based on the 3D face model, including skeletal binding, joint adjustment and animation binding operations on the virtual digital human model, so that the digital human can have the ability to change movements and expressions.

[0103] The digital human processing AI model also processes the material and texture of the generated digital human, adding human skin texture effects, hairstyle texture effects, and half-body clothing texture effects to the face of the virtual human model, and finally generates a 3D model of the virtual human. The 3D model of the virtual human is a changeable 3D model, such as Fig. 9 shown.

[0104] In the virtual digital human model (i.e. Fig. 9 After the digital human model in is generated, it needs to be optimized, which mainly includes the following contents.

[0105] (1) Texture optimization (i.e., digital human model optimization 1): This invention uses the texture mapping-based synthesis algorithm provided by the XiaoIce interface to optimize it to make it more realistic and natural. At the same time, the texture image is smoothed using an algorithm based on Gaussian filtering to reduce its noise and irregularity to make it more natural and smooth.

[0106] (2) Topology optimization (i.e., digital human model optimization 2): The present invention uses the mature topology-preserving topology reconstruction algorithm and the morphology-based topology reconstruction algorithm provided by the XiaoIce interface to optimize the topological structure of the model, reconstruct the topological structure of the three-dimensional face model, and achieve mesh optimization and repair.

[0107] Step 5: Rendering effect. Send the virtual human 3D model generated in step 4 to the renderer. The renderer is the Unreal Engine5 3D engine. The Unreal Engine5 3D engine can provide highly realistic rendering capabilities for the virtual human, that is, it can make the virtual human look very real. The facial features of the virtual human can be rendered under the illumination of multi-angle light, thereby forming multiple detailed virtual human dynamic patterns.

[0108] 3) Generate a virtual digital person business card.

[0109] In terms of speech synthesis, the "Quick Cloud Virtual Human Production Platform" can call the virtual human production interface of Microsoft XiaoIce, and generate a virtual digital human corresponding to the voice and mouth shape according to the introduction words of the business card, and send it to the mobile phone program to synthesize a complete virtual digital human business card. The introduction words are part of the text information entered by the user through the mobile phone.

[0110] The steps for generating a virtual digital person business card are as follows.

[0111] S1: The initial graphic of the business card.

[0112] After the user inputs the text information of the personal introduction, the mobile APP or mini program sends it to the "Quick Cloud Virtual Digital Human Production Platform". The platform will make a high-definition picture in the "*.bmp" format with the personal introduction text information provided by the user and the generated virtual digital human. This picture is the produced business card, and the main information of the business card includes the user's name, position, unit name, and unit address.

[0113] S2. Merge with the generated virtual digital human.

[0114] The virtual digital human model generated in the above steps can be attached to the coordinate position of the "*.bmp" image in S1 after intercepting a portrait at an angle: (x0, y0) (x1, y1), (x2, y2), (x3, y3), so as to merge the virtual digital human and the BMP image of personal information, and then generate a personal business card based on the virtual digital human. Fig.10 shown.

[0115] 4) Generate a virtual digital human business card with voice function.

[0116] The natural language processing technology is integrated with the generated digital human to give the digital human voice, lip shape, and texture rendering functions, which is mainly completed in the following four steps.

[0117] In terms of speech synthesis, the "Quick Cloud Virtual Human Production Platform" uses a deep learning-based text-to-speech neural network model algorithm; the algorithm model combines convolutional neural networks (CNN) and recurrent neural networks (RNN) to convert text into the original sound waveform of the corresponding text, and can directly generate high-quality, natural and smooth speech waveforms from text. The algorithm flow of the text-to-speech is as follows: Fig.11 described.

[0118] S1. TTS technology: The text information of the personal introduction needs to be input into the "Quick Cloud Virtual Digital Human Production Platform" through the mobile phone program after the face photo is taken. After the virtual digital human model is completed, the text of the personal introduction will be converted into voice information that is consistent with the text information and contains emotional information through the TTS technology method. At present, the commonly used TTS technology is the FastSpeech method, a speech synthesis method based on deep learning. This method achieves high-quality and natural conversion from text to speech waveform by training a neural network model.

[0119] S2, Automatic Speech Recognition (ASR): Convert the generated sound back into text for lip matching. Kuaishangyun Virtual Human Production Platform technology uses the Transformer structure of deep learning methods to convert audio signals into text sequences.

[0120] S3, Lip Sync: Generate corresponding lip animation for the virtual digital person based on the sound and the converted text. The method adopted by the present invention is to map the audio features and the personal introduction text information in S1 to the lip animation using the deep learning generative adversarial network (GANs).

[0121] S4. Virtual digital character rendering: Render the virtual digital human after lip syncing. Kuaishangyun Virtual Digital Human Production Platform technology uses Unreal 3D modeling, texture mapping, lighting and rendering to achieve this.

[0122] The virtual digital human display on the mobile phone is mainly used to realize the preview, display and interaction of the virtual digital human.

[0123] Embodiment 2: In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a business card making system based on virtual digital human technology is provided below.

[0124] A business card making system based on virtual digital human technology includes: an image uploading module for uploading a face image to a virtual digital human making platform.

[0125] The virtual digital human generation module is used to generate a virtual digital human according to the face image based on the virtual digital human technology.

[0126] The dynamic portrait generation module is used to generate a dynamic portrait according to the face image based on the dynamic portrait generation technology.

[0127] The virtual digital person business card generation module is used to generate a virtual digital person business card by combining the virtual digital person and the dynamic portrait.

[0128] The module for making a virtual digital person business card with voice function is used to convert the user's text information into voice information by using speech synthesis technology, add it to the virtual digital person business card, and make a virtual digital person business card with voice function; the text information includes name, position, unit and address.

[0129] The present invention includes a mobile phone APP or applet and a "Quick Cloud Virtual Digital Human Production System" program located on a remote server.

[0130] The mobile phone APP or applet can complete the user's photo shooting and editing, and assist the user to complete the entry of personal introduction information. After the server-side data is calculated, the generated virtual digital person business card can be sent to the mobile phone APP or applet.

[0131] The "Quick Cloud Virtual Digital Human Production System" on the remote server side has the ability to convert from a person's photo to a virtual digital human. Based on a variety of neural network algorithms, the virtual digital human production system can convert the person's photo into a point cloud map, and then generate a digital human 3D model from the point cloud. The deep learning model is then used to add texture and bones to the digital human 3D model generated in the previous step, and continue to optimize it to generate a virtual digital human.

[0132] The "Quick Cloud Virtual Digital Human Production System" on the remote server also has the ability to convert the user's text introduction into voice. By using a variety of text-to-speech algorithms, the generated voice information is embedded into the generated digital human and business card image, thereby completing the business card production of the virtual digital human.

[0133] Use business card virtual digital people within the enterprise to improve employee work efficiency and communication efficiency. Specifically, business card virtual digital people can be used as an enterprise-level application to provide the following functions.

[0134] 1. Unified communications: Business card virtual digital people can serve as a unified communications platform for employees, providing multiple communication methods such as voice, video, and text. Employees can send and receive various information such as emails, instant messages, voice mails, etc. through business card virtual digital people, so as to communicate and collaborate with colleagues faster.

[0135] 2. Personal assistant: Business card virtual digital people can provide employees with personal assistant functions, such as reminding employees of their schedules, sending reminders, setting personal goals, etc. These functions can help employees better manage their time and tasks and improve work efficiency.

[0136] 3. Information query: Business card virtual digital people can provide information query services for employees, such as querying the company's internal knowledge base, personnel information, project information, etc. Employees can quickly obtain the required information through business card virtual digital people, saving time and energy.

[0137] 4. Resource sharing: Business card virtual digital people can serve as a platform for sharing resources among employees, such as sharing documents, sharing codes, sharing experiences, etc. These resources can be accessed and shared through business card virtual digital people, thereby improving the efficiency of collaboration and communication among employees.

[0138] 5. Personalized customization: Business card virtual digital people can be customized according to the individual needs of employees, such as recommending specific tools and applications based on their job responsibilities and preferences. This can help employees better adapt to the work environment and improve work efficiency and job satisfaction.

[0139] Business card virtual people can also be integrated with other systems within the enterprise, such as email systems, calendar systems, project management systems, etc., to achieve more efficient workflow and data sharing.

[0140] The application of business card virtual digital people within the enterprise has great potential, which can improve the work efficiency and communication efficiency of employees, and at the same time improve the efficiency and competitiveness of the enterprise. Technical personnel can realize this application scenario by understanding knowledge and skills such as natural language processing technology, machine learning algorithms and database management. At the same time, they also need to pay close attention to data privacy and security issues and take corresponding measures to protect the information and data of the enterprise.

[0141] Embodiment 3: An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned business card making method based on virtual digital human technology.

[0142] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned business card making method based on virtual digital human technology.

[0143] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0144] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A business card making method based on virtual digital human technology, characterized in that: include: Upload the facial image to the virtual digital human production platform; Based on the virtual digital human technology, generating a virtual digital human according to the face image specifically includes: Performing template mapping on multi-angle facial images, and using a deep learning algorithm to extract facial detail features in the facial images at each angle; the facial detail features include an identification of the facial state; Calling a face stitching template corresponding to the gender, aligning and stitching the face detail features at multiple angles, and generating a face detail image including the face contour, facial expression, eye shape, eye color, eye size, and the relative position of the eye and the face; Determining facial key points according to the facial detail image; the facial key points include the shape and size of various facial organs, the relative position of various facial organs and the face, and the hair and hair accessories contained in the face; the organs include the nose, mouth and chin; Based on the facial key points, the facial detail image is converted into facial 3D reconstruction data, and a 3D facial model is constructed using a 3D reconstruction algorithm; Adding a skeleton system and animation control to the three-dimensional face model to generate a virtual digital human model; Optimizing the virtual digital human model to generate a virtual digital human; Based on the dynamic portrait generation technology, a dynamic portrait is generated according to the face image; the dynamic portrait generation technology is a technology based on machine learning and deep learning, which automatically generates a dynamic portrait with realism and smoothness according to a photo; Combining the virtual digital human and the dynamic portrait to generate a virtual digital human business card; Using speech synthesis technology, the user's text information is converted into voice information and added to the virtual digital person business card to create a virtual digital person business card with voice function; when the virtual digital person business card needs to be updated, the user's text information is converted into voice information and directly added to the virtual digital person business card; the text information includes name, position, unit and address; The virtual digital human production platform combines the personal introduction text information provided by the user and the generated virtual digital human into a picture. The generated virtual digital human model captures a portrait at an angle and attaches it to the coordinate positions of the picture: (x0, y0), (x1, y1), (x2, y2), (x3, y3), thereby integrating the virtual digital human and the picture of the personal introduction text information, and then generating a personal business card based on the virtual digital human; In point cloud data processing, the PCA algorithm is called to merge with the deep learning algorithm to reduce the dimension of the rough 3D face image. After the PCA algorithm is called, the LDA algorithm is called to merge with the deep learning algorithm to extract discriminative facial features. The LDA algorithm is used to project the facial features to maximize the inter-class distance and minimize the intra-class distance, thereby improving the discriminative ability of the features. The LBP algorithm is then called to enhance the expression of texture features of multi-angle 3D face images. The LBP algorithm is used to extract texture from local areas of the image, and the LBP features are merged with the deep learning features to comprehensively utilize texture and depth features.

2. The business card making method based on virtual digital human technology according to claim 1 is characterized in that: Upload the face image to the virtual digital human production platform, which previously also included: Capture facial images of the user at multiple angles; the facial images at multiple angles include a frontal shot, a shot with eyes open, a shot with eyes closed, a shot with mouth open, a shot with mouth closed, a shot of the face tilted 45 degrees to the left, a shot of the face tilted 45 degrees to the right, and a shot of the face nodding and looking up.

3. The business card making method based on virtual digital human technology according to claim 2 is characterized in that: Call the face stitching template of the corresponding gender, align and stitch the facial details from multiple angles, and generate a face detail image containing multiple angles, including: Determine the gender of the user corresponding to the facial image; When the user is male, the male face stitching template is called to align and stitch the facial details from multiple angles to generate a facial detail image containing multiple angles; When the user's gender is female, the female face stitching template is called to align and stitch the facial details from multiple angles to generate a facial detail image containing multiple angles; Sending the face detail image to the user; If the user is satisfied, it is determined that the face detail image is correct; If the user is not satisfied, reselect the face stitching template.

4. The business card making method based on virtual digital human technology according to claim 2 is characterized in that: Based on the facial key points, the facial detail image is converted into facial 3D reconstruction data, and a 3D facial model is constructed using a 3D reconstruction algorithm, specifically including: Based on the facial key points, the facial detail image is converted into point cloud data; the point cloud data is three-dimensional reconstruction data of the face; Register the point cloud data at different angles and positions to generate registered point cloud data; Convert the registered point cloud data into a grid and generate a grid result; A three-dimensional face model is constructed using a three-dimensional reconstruction algorithm according to the registered point cloud data and the gridding result.

5. The business card making method based on virtual digital human technology according to claim 2 is characterized in that: Optimizing the virtual digital human model to generate a virtual digital human specifically includes: Optimizing the texture of the virtual digital human model using a synthesis algorithm based on texture mapping to generate a virtual digital human model with optimized texture; The morphological structure of the texture-optimized virtual digital human model is topologically optimized by using a topology reconstruction algorithm based on topology preservation and a topology reconstruction algorithm based on morphology to generate a virtual digital human.

6. The business card making method based on virtual digital human technology according to claim 2 is characterized in that: Based on the virtual digital human technology, a virtual digital human is generated according to the face image, and then the following steps are further included: The virtual digital human is rendered to generate a dynamic pattern of the virtual human.

7. A business card making system based on virtual digital human technology, characterized in that: include: An image uploading module is used to upload the face image to the virtual digital human production platform; A virtual digital human generation module, used to generate a virtual digital human according to the face image based on the virtual digital human technology; Based on the virtual digital human technology, generating a virtual digital human according to the face image specifically includes: Performing template mapping on multi-angle facial images, and using a deep learning algorithm to extract facial detail features in the facial images at each angle; the facial detail features include an identification of the facial state; Calling a face stitching template corresponding to the gender, aligning and stitching the face detail features at multiple angles, and generating a face detail image including the face contour, facial expression, eye shape, eye color, eye size, and the relative position of the eye and the face; Determining facial key points according to the facial detail image; the facial key points include the shape and size of various facial organs, the relative position of various facial organs and the face, and the hair and hair accessories contained in the face; the organs include the nose, mouth and chin; Based on the facial key points, the facial detail image is converted into facial 3D reconstruction data, and a 3D facial model is constructed using a 3D reconstruction algorithm; Adding a skeleton system and animation control to the three-dimensional face model to generate a virtual digital human model; Optimizing the virtual digital human model to generate a virtual digital human; A dynamic portrait generation module, for generating a dynamic portrait according to the face image based on a dynamic portrait generation technology; the dynamic portrait generation technology is a technology based on machine learning and deep learning, which automatically generates a dynamic portrait with realism and smoothness according to a photo; A virtual digital person business card generation module, used to generate a virtual digital person business card by combining the virtual digital person and the dynamic portrait; A virtual digital person business card making module with voice function is used to convert the user's text information into voice information by using speech synthesis technology, add it to the virtual digital person business card, and make a virtual digital person business card with voice function; when the virtual digital person business card needs to be updated, the user's text information is converted into voice information and directly added to the virtual digital person business card; the text information includes name, position, unit and address; The virtual digital human production platform combines the personal introduction text information provided by the user and the generated virtual digital human into a picture. The generated virtual digital human model captures a portrait at an angle and attaches it to the coordinate positions of the picture: (x0, y0), (x1, y1), (x2, y2), (x3, y3), thereby integrating the virtual digital human and the picture of the personal introduction text information, and then generating a personal business card based on the virtual digital human; In point cloud data processing, the PCA algorithm is called to merge with the deep learning algorithm to reduce the dimension of the rough 3D face image. After the PCA algorithm is called, the LDA algorithm is called to merge with the deep learning algorithm to extract discriminative facial features. The LDA algorithm is used to project the facial features to maximize the inter-class distance and minimize the intra-class distance, thereby improving the discriminative ability of the features. The LBP algorithm is then called to enhance the expression of texture features of multi-angle 3D face images. The LBP algorithm is used to extract texture from local areas of the image, and the LBP features are merged with the deep learning features to comprehensively utilize texture and depth features.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the business card making method based on the virtual digital human technology as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: It stores a computer program, and when the computer program is executed by a processor, the business card making method based on virtual digital human technology as described in any one of claims 1 to 6 is implemented.

Images