Personalized digital human generation and customization platform based on multi-dimensional feature fusion

Abstract

The invention relates to the technical field of digital human generation, in particular to a multi-dimensional feature fusion personalized digital human generation and customization platform, which comprises a data processing and fusion module, the data processing and fusion module is in coupling connection with a personalized feature extraction module, the personalized feature extraction module is in coupling connection with a digital human generation and rendering module, and the digital human generation and rendering module is in coupling connection with the personalized feature extraction module. The digital human generation and rendering module is in coupling connection with an interaction and customization module, the interaction and customization module is in coupling connection with a security and privacy protection module, and the security and privacy protection module is in coupling connection with a cost optimization and efficiency improvement module. According to the multi-dimensional feature fusion personalized digital human generation and customization platform, the problems of insufficient maturity, data problem, lack of security and privacy protection, poor man-machine interaction experience, high cost and long manufacturing period of the multi-dimensional feature fusion personalized digital human generation and customization platform in the actual use process in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital human generation, and in particular to a personalized digital human generation and customization platform integrating multi-dimensional features. Background Art

[0002] The personalized digital human generation and customization platform with multi-dimensional feature fusion is an artificial intelligence system that integrates multiple advanced technologies. It aims to provide users with highly personalized digital human generation and customization services. The personalized digital human generation and customization platform with multi-dimensional feature fusion generates digital human images with unique personality and high fidelity by integrating feature information of multiple dimensions (such as facial features, voice features, action features, etc.) and using advanced algorithms and technical means. This type of platform not only supports users to customize digital humans, but also provides rich interactive functions, allowing users to interact with the generated digital humans in real time. The platform can collect, process and integrate feature information from different dimensions, such as facial contours, expressions, voice characteristics, action habits, etc., to provide comprehensive and rich data support for digital human generation. Users can customize digital humans according to their own needs, including appearance, voice, action style, etc. The platform provides an intuitive and easy-to-use customization interface and rich options, allowing users to easily create digital human images that suit their preferences. Using advanced computer graphics, deep learning and other technical means, the platform can generate digital human images with high fidelity. These digital people not only look similar to real humans, but also show natural and smooth expressions in movements, expressions, and voices. The platform provides a wealth of interactive functions, allowing users to interact with the generated digital people in real time. Whether it is chatting, entertainment or other application scenarios, digital people can provide users with a real and natural interactive experience, and are responsible for collecting, cleaning, and integrating multi-dimensional feature data from different sources to ensure data quality and consistency. Advanced algorithms are used to extract personalized feature information from the fused multi-dimensional feature data, providing key data support for digital human generation.

[0003] However, in actual use, the personalized digital human generation and customization platform with multi-dimensional feature fusion in the existing technology lacks maturity, suffers from data problems, lacks security and privacy protection, has a poor human-computer interaction experience, is costly, and has a long production cycle. To address this problem, a personalized digital human generation and customization platform with multi-dimensional feature fusion is provided. Summary of the invention

[0004] The purpose of the present invention is to provide a personalized digital human generation and customization platform with multi-dimensional feature fusion to solve the problems raised in the above background technology. To achieve the above purpose, the present invention provides the following technical solution: a personalized digital human generation and customization platform with multi-dimensional feature fusion, comprising a data processing and fusion module, the data processing and fusion module is coupled to a personalized feature extraction module, the personalized feature extraction module is coupled to a digital human generation and rendering module, the digital human generation and rendering module is coupled to an interaction and customization module, the interaction and customization module is coupled to a security and privacy protection module, and the security and privacy protection module is coupled to a cost optimization and efficiency improvement module.

[0005] Preferably, the data processing and fusion module includes a data acquisition unit, the data acquisition unit is coupled to a data cleaning unit, the data cleaning unit is coupled to a data integration unit, the data integration unit is coupled to a data fusion unit, and the data fusion unit is coupled to a data storage and management unit.

[0006] Preferably, the algorithm adopted by the data fusion unit includes one of a weighted fusion algorithm, a Kalman filter algorithm, and a Bayesian network algorithm.

[0007] Preferably, the weighted fusion algorithm includes:

[0008] Step 1: Expand the edge pixels of the input full-color image Phigh, and calculate the gradient map of the expanded image. The gradient value corresponding to the pixel point (i, j) is the partial derivative of the pixel point (i, j) in the x direction and the partial derivative of the pixel point (i, j) in the y direction f(i, j) is the gray value at any pixel point (i, j) of the full-color image;

[0009] Step 2: Determine the filter window size according to the calculated gradient map. The specific steps are as follows: ① Process the pixel part corresponding to the full-color image in G(i, j) to obtain ② Determine the basic parameters according to coef and then determine the size of the filter window N=2d+1 according to the basic parameters d;

[0010] Step 3: With pixel (i, j) as the center, in a window of size N*N, define the filter weight matrix where the weight values ​​s and t at any position are integers and are in the range of [-d, d];

[0011] Step 4: Use the filter weight matrix to perform gradient weighted smoothing filtering on the full-color image and calculate the filter output value of the pixel point (i, j). Where Phigh(i, j) represents the pixel value of the pixel point (i, j) in the full-color image; substitute Pmean into the SFIM fusion algorithm to calculate the fused image. Plow represents a multispectral image of any band, and Pmean represents the image after filtering the full-color image Phigh.

[0012] Preferably, the Kalman filter algorithm includes:

[0013] Step 1: Establish a state vector model and a measurement vector model of the filtering system;

[0014] Step 2: After the system is initialized, the Kalman filter prediction value is used as the sample regression fitting value to calculate the goodness of fit determination coefficient and correction coefficient;

[0015] Step 3, correcting the process noise covariance matrix according to the correction coefficient;

[0016] Step 4: Calculate the Kalman gain, the state estimate and the estimation error covariance according to the transmission matrix, the measurement matrix and the corrected process noise covariance matrix in the state vector model and the measurement vector model.

[0017] Preferably, the state vector model and the measurement vector model are respectively:

[0018] x k =A k-1 x k-1 +w k-1

[0019] z k =H k x k +v k

[0020] Among them, X K is the state vector at time K; z k is the measurement vector at time k; A k-1 is the state transfer matrix from time k-1 to time k, i.e., the transmission matrix; H k is the measurement matrix at time k; w k and v k represent the process noise sequence and measurement noise sequence at time k respectively.

[0021] Preferably, the system initialization specifically includes:

[0022]

[0023] Among them, X0 is the initial value of the state vector; Estimate initial values ​​for the posterior state; Estimate the initial value of the covariance for the posterior state; is the initial value of the measured data mean, RSS0 is the initial value of the residual sum of squares, and TSS0 is the initial value of the total sum of squares.

[0024] Preferably, the Kalman filter prediction value is used as the sample regression fitting value, and the goodness of fit determination coefficient and correction coefficient are calculated as follows:

[0025]

[0026] Preferably, the digital human generation and rendering module includes a digital human modeling unit, the digital human modeling unit is coupled to a detail adjustment and optimization unit, the detail adjustment and optimization unit is coupled to an action generation and driving unit, the action generation and driving unit is coupled to an expression generation and rendering unit, the expression generation and rendering unit is coupled to a rendering engine unit, and the rendering engine unit is coupled to a real-time interaction processing unit.

[0027] Preferably, the real-time interaction processing unit is used to adjust the digital human's posture or expression.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] In this case, the data processing and fusion module is used to collect, clean and integrate multi-dimensional feature data from different sources, such as facial features, voice features, action features, etc. The efficient data processing and fusion technology is used to ensure the quality and consistency of the multi-dimensional feature data, providing a solid foundation for the subsequent generation of personalized digital humans. The personalized feature extraction module uses advanced algorithms to extract personalized feature information from the fused multi-dimensional feature data. Through precise feature extraction, it is ensured that the generated digital human can accurately reflect the user's personalized needs, such as specific facial features, voice style, etc. The digital human generation and rendering module generates a highly realistic digital human image based on the extracted personalized feature information, and performs rendering processing to present high-quality visual effects. Through advanced generation and rendering technology, the rapid generation and high-quality presentation of personalized digital humans are achieved to meet the user's requirements for the appearance and performance of digital humans. In order to meet the requirements of current forces, the interactive and customization module provides an interface for users to interact with the generated digital human, and allows users to customize the digital human according to their own needs. Through the intuitive and easy-to-use interactive interface, the user's sense of control and participation in the digital human is enhanced; at the same time, the personalized customization function can meet the user's diverse needs for the appearance, movement, sound, etc. of the digital human. The security and privacy protection module protects the security and privacy of user data to prevent data leakage and abuse. Through encryption technology, permission management and other means, it ensures the security and privacy of user data during collection, storage, processing and transmission, enhances the user's trust in the platform, and solves the problems of insufficient maturity, data problems, lack of security and privacy protection, poor human-computer interaction experience, high cost and long production cycle in the actual use of the personalized digital human generation and customization platform with multi-dimensional features fusion in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A system block diagram of a personalized digital human generation and customization platform with multi-dimensional feature fusion according to the present invention;

[0031] Figure 2 This is a system block diagram of the digital human generation and rendering module of the personalized digital human generation and customization platform with multi-dimensional feature fusion of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without creative work are within the scope of protection of the present invention.

[0033] See also Figure 1 to Figure 2The present invention provides a technical solution: a personalized digital human generation and customization platform with multi-dimensional feature fusion, including a data processing and fusion module, the data processing and fusion module is coupled with a personalized feature extraction module, the personalized feature extraction module is coupled with a digital human generation and rendering module, the digital human generation and rendering module is coupled with an interaction and customization module, the interaction and customization module is coupled with a security and privacy protection module, the security and privacy protection module is coupled with a cost optimization and efficiency improvement module, the data processing and fusion module is used to collect, clean and integrate multi-dimensional feature data from different sources, such as facial features, voice features, action features, etc., through efficient data processing and fusion technology, the quality and consistency of the multi-dimensional feature data are ensured, and a solid foundation is provided for subsequent personalized digital human generation, the personalized feature extraction module uses advanced algorithms to extract personalized feature information from the fused multi-dimensional feature data, and through accurate feature extraction, it is ensured that the generated digital human can accurately reflect the personalized needs of the user, such as specific facial features, voice style, etc., through the digital human generation and rendering module based on the extracted personalized feature information , generate a digital human image with high fidelity, and render it to present high-quality visual effects. Through advanced generation and rendering technology, the rapid generation and high-quality presentation of personalized digital humans are realized, meeting the user's requirements for the appearance and expressiveness of digital humans. Through the interaction and customization module, an interface for users to interact with the generated digital humans is provided, and users are allowed to customize the digital humans according to their own needs. Through an intuitive and easy-to-use interactive interface, the user's sense of control and participation in the digital humans is enhanced; at the same time, the personalized customization function can meet the user's diverse needs for the appearance, movement, sound, etc. of the digital humans. The security and privacy protection module protects the security and privacy of user data to prevent data leakage and abuse. Through encryption technology, permission management and other means, the security and privacy of user data in the process of collection, storage, processing and transmission are ensured, and the user's trust in the platform is enhanced. The problem of insufficient maturity, data problems, lack of security and privacy protection, poor human-computer interaction experience, high cost and long production cycle in the actual use of the personalized digital human generation and customization platform with multi-dimensional feature fusion in the existing technology is solved.

[0034] In this embodiment, the data processing and fusion module includes a data acquisition unit, which is responsible for collecting multi-dimensional feature data, such as facial features, voice features, action features, etc. from various data sources. These data sources may include cameras, microphones, sensors, etc., to ensure that the platform can obtain comprehensive and rich feature data and provide sufficient raw materials for subsequent data processing and fusion. The data acquisition unit is coupled with a data cleaning unit to clean and preprocess the collected raw data, including removing noise, filling missing values, correcting erroneous data and other operations, to improve the quality and reliability of the data, and to provide an accurate data basis for subsequent data analysis and fusion. The data cleaning unit is coupled with a data integration unit to integrate the cleaned data according to certain rules and standards to form a unified data format and structure, which is convenient for subsequent data analysis and processing and improves the efficiency and accuracy of data processing. The data integration unit is coupled with a data fusion unit to use advanced algorithms and technical means to fuse data from different dimensions and sources to generate comprehensive and consistent multi-dimensional feature data, which provides rich and accurate data support for the subsequent generation of personalized digital people and ensures that the generated digital people can accurately reflect the personalized needs of users. The data fusion unit is coupled with a data storage and management unit to store and manage the fused multi-dimensional feature data, including data storage, backup, recovery and access control operations, to ensure the security and availability of data and provide stable data support for subsequent data analysis and application.

[0035] In this embodiment, the algorithm adopted by the data fusion unit includes one of the weighted fusion algorithm, the Kalman filter algorithm, and the Bayesian network algorithm. The weighted fusion algorithm is a simple and effective method. It assigns different weights to the data from different data sources and then performs weighted averaging to obtain the fused data. The selection of weights is usually based on factors such as the credibility, importance or accuracy of the data source. The weighted fusion algorithm is widely used in the data fusion unit, especially when processing multi-dimensional feature data. The weights can be assigned according to the importance and credibility of different features, so as to obtain a more accurate and reliable fusion result. The Kalman filter algorithm is a recursive filter algorithm that can estimate the state of the system in a noisy linear dynamic system. The algorithm continuously corrects the state estimation value of the system through two steps of prediction and update, so as to obtain more accurate results. In the data fusion unit, the Kalman filter algorithm can be used to fuse data from different time points, especially when processing dynamically changing multi-dimensional feature data, which can effectively improve the accuracy and real-time performance of the data. The Bayesian network algorithm is a data fusion method based on the probabilistic graph model. It constructs a Bayesian network to represent the dependency between variables and uses the Bayesian theorem to calculate the posterior probability distribution of variables. In the data fusion unit, the Bayesian network algorithm can be used to fuse multi-dimensional feature data from different data sources with uncertainty. By constructing a Bayesian network and calculating the posterior probability distribution, a more accurate and reliable fusion result can be obtained.

[0036] In this embodiment, the weighted fusion algorithm includes:

[0037] Step 1: Expand the edge pixels of the input full-color image Phigh, and calculate the gradient map of the expanded image. The gradient value corresponding to the pixel point (i, j) is the partial derivative of the pixel point (i, j) in the x direction and the partial derivative of the pixel point (i, j) in the y direction f(i, j) is the gray value at any pixel point (i, j) of the full-color image;

[0038] Step 2: Determine the filter window size according to the calculated gradient map. The specific steps are as follows: ① Process the pixel part corresponding to the full-color image in G(i, j) to obtain ② Determine the basic parameters according to coef and then determine the size of the filter window N=2d+1 according to the basic parameters d;

[0039] Step 3: With pixel (i, j) as the center, in a window of size N*N, define the filter weight matrix where the weight values ​​s and t at any position are integers and are in the range of [-d, d];

[0040] Step 4: Use the filter weight matrix to perform gradient weighted smoothing filtering on the full-color image and calculate the filter output value of the pixel point (i, j). Where Phigh(i, j) represents the pixel value of the pixel point (i, j) in the full-color image; substitute Pmean into the SFIM fusion algorithm to calculate the fused image. Plow represents a multispectral image of any band, and Pmean represents the image after filtering the full-color image Phigh.

[0041] In this embodiment, the Kalman filter algorithm includes:

[0042] Step 1: Establish a state vector model and a measurement vector model of the filtering system;

[0043] Step 2: After the system is initialized, the Kalman filter prediction value is used as the sample regression fitting value to calculate the goodness of fit determination coefficient and correction coefficient;

[0044] Step 3, according to the correction coefficient, correct the process noise covariance matrix;

[0045] Step 4: Calculate the Kalman gain, the state estimate and the estimation error covariance according to the transmission matrix, the measurement matrix and the corrected process noise covariance matrix in the state vector model and the measurement vector model.

[0046] In this embodiment, the state vector model and the measurement vector model are:

[0047] x k =A k-1 x k-1 +w k-1

[0048] z k =H k x k +v k

[0049] Among them, X K is the state vector at time K; z k is the measurement vector at time k; A k-1 is the state transfer matrix from time k-1 to time k, i.e., the transmission matrix; H k is the measurement matrix at time k; w k and v k represent the process noise sequence and measurement noise sequence at time k respectively.

[0050] In this embodiment, the system initialization is specifically as follows:

[0051]

[0052] Among them, X0 is the initial value of the state vector; Estimate initial values ​​for the posterior state; Estimate the initial value of the covariance for the posterior state; is the initial value of the measured data mean, RSS0 is the initial value of the residual sum of squares, and TSS0 is the initial value of the total sum of squares.

[0053] In this embodiment, the Kalman filter prediction value is used as the sample regression fitting value, and the goodness of fit determination coefficient and correction coefficient are calculated as follows:

[0054]

[0055] In this embodiment, the digital human generation and rendering module includes a digital human modeling unit, which uses computer graphics and 3D modeling technology to build a basic model of the digital human based on the fused multi-dimensional feature data. This includes the face, body, clothing and other parts of the digital human, and the technologies used include 3D scanning, parametric modeling, image-based modeling, etc. These technologies can help quickly and accurately generate the basic model of the digital human based on the feature data. The digital human modeling unit is coupled with a detail adjustment and optimization unit, which adjusts and optimizes the details of the initially constructed digital human model to make it more in line with the user's personalized needs and expectations. The technologies involved include texture mapping, material adjustment, lighting processing, shadow effects, etc. These technologies can enhance the realism and detail performance of the digital human. The detail adjustment and optimization unit is coupled with an action generation and driving unit, which generates a corresponding action sequence for the digital human based on the action data input or preset by the user, and drives the digital human model to perform actions. The technologies used include motion capture, keyframe animation, physical simulation, etc. These technologies can help digital humans achieve natural and smooth movements. The movement generation and driving unit is coupled with the expression generation and rendering unit. According to the fused facial expression feature data, the corresponding expression animation is generated for the digital human, and rendering is performed to present high-quality visual effects. The technologies used include facial expression capture, muscle simulation, expression mapping, etc. These technologies can help digital humans achieve realistic facial expressions. The expression generation and rendering unit is coupled with the rendering engine unit, which integrates the digital human model, movement, expression and other elements, and performs rendering to generate the final digital human video or image. The rendering engines used include Unity, Unreal Engine, etc. These rendering engines provide powerful rendering capabilities and rich special effects support, which can help generate high-quality digital human visual content. The rendering engine unit is coupled with the real-time interaction processing unit. During the digital human generation and rendering process, the user's real-time interaction request, such as adjusting the digital human's posture and expression, is processed and immediately fed back to the rendering result. The technologies used include real-time rendering technology, interactive design, etc. These technologies can help improve the user's interactive experience and satisfaction.

[0056] In this embodiment, the real-time interaction processing unit is used to adjust the digital human's posture or expression.

[0057] The above shows and describes the basic principles, main features and advantages of the present invention. Technical personnel in this industry should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A personalized digital human generation and customization platform with multi-dimensional feature fusion, characterized by: It includes a data processing and fusion module, which is coupled to a personalized feature extraction module, which is coupled to a digital human generation and rendering module, which is coupled to an interaction and customization module, which is coupled to a security and privacy protection module, and which is coupled to a cost optimization and efficiency improvement module.

2. The personalized digital human generation and customization platform with multi-dimensional feature fusion according to claim 1, characterized in that: The data processing and fusion module includes a data acquisition unit, the data acquisition unit is coupled to a data cleaning unit, the data cleaning unit is coupled to a data integration unit, the data integration unit is coupled to a data fusion unit, and the data fusion unit is coupled to a data storage and management unit.

3. The personalized digital human generation and customization platform with multi-dimensional feature fusion according to claim 2 is characterized by: The algorithm adopted by the data fusion unit includes one of a weighted fusion algorithm, a Kalman filter algorithm, and a Bayesian network algorithm.

4. The personalized digital human generation and customization platform based on multi-dimensional feature fusion according to claim 3 is characterized by: The weighted fusion algorithm includes: Step 1: Expand the edge pixels of the input full-color image Phigh, and calculate the gradient map of the expanded image. The gradient value corresponding to the pixel point (i, j) is the partial derivative of the pixel point (i, j) in the x direction and the partial derivative of the pixel point (i, j) in the y direction f(i, j) is the gray value at any pixel point (i, j) of the full-color image; Step 2: Determine the filter window size according to the calculated gradient map. The specific steps are as follows: ① Process the pixel part corresponding to the full-color image in G(i, j) to obtain ② Determine the basic parameters according to coef and then determine the size of the filter window N=2d+1 according to the basic parameters d; Step 3: With pixel (i, j) as the center, in a window of size N*N, define the filter weight matrix where the weight values ​​s, t at any position are integers and are in the range of [-d, d]; Step 4: Use the filter weight matrix to perform gradient weighted smoothing filtering on the full-color image and calculate the filter output value of the pixel point (i, j). Where Phigh(i, j) represents the pixel value of the pixel point (i, j) in the full-color image; substitute Pmean into the SFIM fusion algorithm to calculate the fused image. Plow represents a multispectral image of any band, and Pmean represents the image after filtering the full-color image Phigh.

5. The personalized digital human generation and customization platform based on multi-dimensional feature fusion according to claim 3 is characterized by: The Kalman filter algorithm includes: Step 1: Establish a state vector model and a measurement vector model of the filtering system; Step 2: After the system is initialized, the Kalman filter prediction value is used as the sample regression fitting value to calculate the goodness of fit determination coefficient and correction coefficient; Step 3, correcting the process noise covariance matrix according to the correction coefficient; Step 4: Calculate the Kalman gain, the state estimate and the estimation error covariance according to the transmission matrix, the measurement matrix and the corrected process noise covariance matrix in the state vector model and the measurement vector model.

6. The personalized digital human generation and customization platform based on multi-dimensional feature fusion according to claim 5 is characterized by: The state vector model and the measurement vector model are respectively: x k =A k-1 x k-1 +w k-1 z k =H k x k +v k Among them, X K is the state vector at time K; z k is the measurement vector at time k; A k-1 is the state transfer matrix from time k-1 to time k, i.e., the transmission matrix; H k is the measurement matrix at time k; w k and v k They represent the process noise sequence and measurement noise sequence at time k respectively.

7. The personalized digital human generation and customization platform based on multi-dimensional feature fusion according to claim 5, characterized in that: The system initialization is specifically as follows: Among them, X0 is the initial value of the state vector; Estimate initial values ​​for the posterior state; Estimate the initial value of the covariance for the posterior state; is the initial value of the measured data mean, RSS0 is the initial value of the residual sum of squares, and TSS0 is the initial value of the total sum of squares.

8. The personalized digital human generation and customization platform with multi-dimensional feature fusion according to claim 1 is characterized by: The Kalman filter prediction value is used as the sample regression fitting value, and the goodness of fit determination coefficient and correction coefficient are calculated as follows:

9. The personalized digital human generation and customization platform with multi-dimensional feature fusion according to claim 1, characterized in that: The digital human generation and rendering module includes a digital human modeling unit, the digital human modeling unit is coupled to a detail adjustment and optimization unit, the detail adjustment and optimization unit is coupled to an action generation and driving unit, the action generation and driving unit is coupled to an expression generation and rendering unit, the expression generation and rendering unit is coupled to a rendering engine unit, and the rendering engine unit is coupled to a real-time interaction processing unit.

10. The multi-dimensional feature fusion personalized digital human generation and customization platform according to claim 9, characterized in that: The real-time interaction processing unit is used to adjust the digital human's posture or expression.

Images