A virtual digital human management method, device, equipment and storage medium

Abstract

The present invention provides a management method, device, equipment and storage medium for virtual digital humans, which are applied to the field of data processing technology. The present application obtains target user information and a training sample set; pre-processes the training sample set to generate a training sample set with identification information; performs feature extraction processing on the training sample set with identification information to generate an original feature vector; processes the original feature vector based on a preset processing rule to generate a training set and a verification set; obtains an initial digital human driving model that matches the identification information; trains the initial digital human driving model based on the training set and the verification set to generate a target digital human driving model; processes the target user information to generate the target user's attribute information; processes the target user's attribute information based on the target digital human driving model to generate the target digital human information, wherein the target digital human information is used to characterize the information reply to the target user based on the target digital human.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a management method, device, equipment and storage medium for a virtual digital human. Background Art

[0002] Virtual digital human is a multi-modal intelligent human-computer interaction technology that integrates computer vision, speech recognition, speech synthesis, natural language processing, terminal display and other technologies to create a highly anthropomorphic virtual image that interacts and communicates with people like a real person. The main shortcomings of existing virtual digital human and intelligent delivery systems include:

[0003] 1. Poor interactive experience: Virtual digital people have deficiencies in speech recognition, natural language processing, and speech synthesis, resulting in delays and unsmooth conversations. The recommendation algorithm of the smart shopping system has low accuracy, affecting user experience and sales conversion.

[0004] 2. Difficulty in technology integration and poor adaptability: The system needs to integrate multiple technologies but faces compatibility and integration challenges, resulting in high maintenance costs and computing power requirements. The virtual digital human system lacks adaptability and stability in different application scenarios.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of this application is to provide a management method, device, equipment and storage medium for virtual digital humans, which at least to some extent overcome the problems existing in the prior art. In order to create a digital human that can reply to users, it is necessary to collect and pre-process user data, extract features such as inverse document frequency values ​​and emotional features, train and optimize the digital human model, and finally generate digital human information that can understand and respond to user needs.

[0007] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present invention.

[0008] According to one aspect of the present application, a method for managing a virtual digital human is provided, comprising: obtaining target user information and a training sample set, wherein the target user information comprises speech data, body movement data and facial expression data of the target user within a preset time period, and the training sample set comprises other user information matching the target user information; preprocessing the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize that the other user data is in an abnormal state; performing feature extraction processing on the training sample set with the identification information to generate an original feature vector, wherein the original feature vector comprises an initial text word frequency, an initial inverse document frequency value and an initial emotional feature. The initial emotional feature is generated based on the voice data, body movement data and facial expression data of other users; the original feature vector is processed based on the preset processing rules to generate a training set and a verification set; an initial digital human driving model matching the identification information is obtained; the initial digital human driving model is trained based on the training set and the verification set to generate a target digital human driving model; the target user information is processed to generate the attribute information of the target user; the attribute information of the target user is processed based on the target digital human driving model to generate the target digital human information, wherein the target digital human information is used to characterize the information reply to the target user based on the target digital human.

[0009] In one embodiment of the present application, feature extraction processing is performed on the training sample set with identification information to generate an original feature vector, including: the method also includes a calculation formula for calculating the initial emotion feature, and the calculation formula is:

[0010] ;

[0011] ;

[0012] ;

[0013] ;

[0014] ;

[0015] in, They are input gate, forget gate and output gate. is the cell state, i.e. the initial emotional characteristics, is the hidden state, and are weight and bias parameters, is the sigmoid function, is the hyperbolic tangent function, is the feature weight matrix of the input gate, is the hidden state at the previous time step, is the input of the current time step, is the bias term of the input gate, It's the Gate of Forgetfulness. is the feature weight matrix of the forget gate, is the bias term of the forget gate, is the feature weight matrix of the output gate, is the bias term of the output gate, are candidate memory cells, is the weight matrix of the candidate memory cell state, is the bias term of the candidate memory cell state, The cell state at the previous time step.

[0016] In one embodiment of the present application, the original feature vector is processed based on a preset processing rule to generate a training set and a validation set, including:

[0017] Obtaining a preset judgment matrix, feature weight information matching the original feature vector, and some engine information;

[0018] Processing the original feature vector and feature weight information matching the original feature vector to generate a target feature vector, wherein the target feature vector includes a word frequency, an inverse document frequency value, and a sentiment feature;

[0019] Processing the target feature vector and a number of engine information to generate an engine selection result;

[0020] Generate a training set and a validation set based on the engine selection result;

[0021] The method also includes a calculation formula for calculating feature weight information, and the calculation formula is:

[0022] ;

[0023] in, is an element in the preset judgment matrix, is the number of criteria, is the feature weight information;

[0024] The method also includes a calculation formula for calculating the target feature vector, and the calculation formula is:

[0025] ;

[0026] Among them, word frequency , inverse document frequency value and emotional characteristics , and their weights are , and .

[0027] In one embodiment of the present application, the target feature vector and some engine information are processed to generate an engine selection result, including: the method also includes a calculation formula for calculating the probability of selecting the target engine based on the target feature vector, and the calculation formula is: ;in, It is in the characteristics Select Engine The probability of It is a feature In the engine The conditional probability under It's the engine The prior probability of It is a feature The prior probability of .

[0028] In one embodiment of the present application, the initial digital human driving model is trained based on the training set and the verification set to generate a target digital human driving model, including: training the initial digital human driving model based on the training set to generate a prediction result label; predicting the initial digital human driving model based on the verification set to generate a target result label; processing the prediction result label and the target result label to generate a model loss function; processing the model loss function to generate a model adjustment parameter; processing the initial digital human driving model based on the model adjustment parameter to generate a target digital human driving model; the method also includes a calculation formula for calculating the model loss function, and the calculation formula is: in, is the model loss function, is the target result label, is the predicted result label, N is the total number of samples, and i is the index of the sample.

[0029] In one embodiment of the present application, the target user information is processed to generate attribute information of the target user, including: obtaining real-time data processing information of the target server, wherein the real-time data processing information of the target server is used to characterize the data processing progress information of the target server; performing data cleaning processing on the target user information to generate real-time physiological parameter information of the target user and to-do information of the target user; processing the real-time physiological parameter information of the target user and the to-do information of the target user based on the data processing progress information of the target server to generate priority information of the to-do information of the target user; generating the attribute information of the target user based on the priority information of the to-do information of the target user, wherein the attribute information of the target user is used to characterize the real-time to-do information of the target user.

[0030] In one embodiment of the present application, the target user's attribute information is processed based on the target digital human driving model to generate target digital human information, including: processing the target user's attribute information based on the target digital human driving model to generate a data processing result that matches the target user's implementation to-do information; processing the data processing result based on the target digital human driving model to generate the digital human's text information, the digital human's motion information and the digital human's expression information; processing the digital human's text information, the digital human's motion information and the digital human's expression information to generate the target digital human information.

[0031] Another aspect of the present application is a management device for virtual digital humans, characterized in that it includes: an acquisition module for acquiring target user information and a training sample set, wherein the target user information includes voice data, body movement data, and facial expression data of the target user within a preset time period, and the training sample set includes other user information that matches the target user information; acquiring an initial digital human driving model that matches the identification information; a processing module for preprocessing the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize that other user data is in an abnormal state; performing feature extraction processing on the training sample set with identification information to generate an original feature vector, wherein the original The feature vector includes an initial text word frequency, an initial inverse document frequency value (TF-IDF value) and an initial emotional feature, wherein the initial emotional feature is generated based on the voice data, body movement data and facial expression data of other users; the original feature vector is processed based on a preset processing rule to generate a training set and a verification set; the initial digital human driving model is trained based on the training set and the verification set to generate a target digital human driving model; the target user information is processed to generate the attribute information of the target user; the attribute information of the target user is processed based on the target digital human driving model to generate the target digital human information, wherein the target digital human information is used to characterize the information reply to the target user based on the target digital human.

[0032] According to another aspect of the present application, an electronic device is characterized in that it includes: a first processor; and a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the above-mentioned virtual digital human management method by executing the executable instructions.

[0033] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a second processor, the above-mentioned virtual digital human management method is implemented.

[0034] According to another aspect of the present application, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a third processor, the computer program implements the above-mentioned virtual digital human management method.

[0035] The present application provides a method, device, equipment and storage medium for managing virtual digital humans. The server collects the user's voice, body movement and facial expression data, constructs training samples, cleans the data and extracts features, such as text word frequency, TF-IDF value and emotional features. These features are used to create training sets and validation sets to train the digital human driving model. Through model training, parameters are optimized to generate a target digital human driving model that can accurately reflect user needs. Finally, user information is processed and combined with the model to generate digital human information that can effectively respond to users, ensuring that the digital human can provide personalized and accurate services.

[0036] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart showing a method for managing a virtual digital human provided by an embodiment of the present application is shown;

[0038] Figure 2 A schematic diagram of the structure of a virtual digital human management device provided by an embodiment of the present application is shown;

[0039] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;

[0040] Figure 4 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0042] Combine the following Figure 1 To describe the management method of virtual digital people according to the exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0043] In one implementation, the present application also proposes a management method, apparatus, device and storage medium for a virtual digital human. Figure 1The following schematically shows a flow chart of a method for managing a virtual digital human according to an embodiment of the present application. Figure 1 As shown, the method is applied to a server, comprising:

[0044] S101, obtaining target user information and training sample sets.

[0045] In one embodiment, the target user information includes the target user's voice data, body movement data, and facial expression data within a preset time period. The analysis of the target user information involves the comprehensive processing of the voice data, body movement data, and facial expression data to determine the content that the target user wants to express, that is, to determine the content that the target user wants to express, including the corresponding voice, movement, and expression information matching therewith.

[0046] For the processing of target user information, a method similar to text classification can be used to convert speech, action and expression data into a format suitable for Bayesian classifier processing. For example, feature extraction technology can be used to convert speech into spectrograms and body movements and facial expressions into quantifiable feature vectors. These feature vectors can then be used as input to train a naive Bayesian classifier to identify and predict user intent.

[0047] The construction of the training sample set needs to include other user information that matches the target user information to ensure the generalization ability of the model. During the training process, the model can be optimized by adjusting the hyperparameters of the Bayesian classifier (such as the Laplace smoothing coefficient in the multinomial naive Bayes). In terms of model evaluation and tuning, techniques such as cross-validation and grid search can be used to find the best model parameters. In addition, the performance of the model can be evaluated by indicators such as confusion matrix, precision, and recall. Through comprehensive analysis of the target user information, the naive Bayesian classifier can be used to improve the recognition and prediction accuracy of user intent.

[0048] S102: pre-process the training sample set to generate a training sample set with identification information.

[0049] In one embodiment, when processing multimodal data of target user information, the preprocessing step is crucial, including cleaning and standardization of voice data, body movement data, and facial expression data to ensure the quality of the data. For the generation of identification information of abnormal states, first, the data needs to be thoroughly cleaned to remove noise and outliers, including using filtering algorithms or noise reduction algorithms to process voice and video data. Use statistical methods or machine learning models to identify outliers in the data. For example, Z-scores, IQR (interquartile range) or machine learning-based isolation forests, DBSCAN, LOF (local outlier factor) and other algorithms can be used.

[0050] Construct features that can characterize abnormal states, such as the volume of sound, the amplitude of body movements, the degree of distortion of facial expressions, etc. These features can be used as inputs to the model to help identify abnormal states. In anomaly detection, Bayesian theory can be used to combine the detection results of multiple algorithms to improve the accuracy of anomaly detection. For example, if multiple algorithms determine that a sample is abnormal, then the probability that the sample is a true anomaly will be very high. Learn the pattern of normal states through training sample sets, and then use this model to identify sample points that are significantly different from the normal pattern. This can be achieved by training naive Bayes classifiers or other machine learning models. In multimodal learning, special attention should be paid to how to handle the fusion and application of different modal data. For example, in multimodal learning of images and texts, CNN can be used to extract image features, RNN or Transformer can be used to process text data, and then the features can be fused.

[0051] Through the above method, the training sample set can be effectively preprocessed and a training sample set with identification information can be generated to characterize that other user data is in an abnormal state, which provides a solid foundation for subsequent model training and anomaly detection.

[0052] S103, performing feature extraction processing on the training sample set with identification information to generate an original feature vector.

[0053] In one implementation, the original feature vector includes an initial text word frequency, an initial inverse document frequency value (TF-IDF value), and an initial sentiment feature, where the initial sentiment feature is generated based on voice data, body movement data, and facial expression data of other users.

[0054] Among them, by counting the number of occurrences of each word in the input text, the initial text word frequency can be obtained. Specifically, ;in, Indicates a term, Represents a document, Representing terms In the documentation The number of times it appears in Representation Document The total number of words in .

[0055] The inverse document frequency value measures the importance of a word in all documents. It is reflected by calculating the rarity of the word in all documents. Specifically, in, Represents the total number of documents, Indicates that it contains a term The number of documents.

[0056] The feature extraction of initial sentiment features is an important task in natural language processing (NLP), which can help us understand the sentiment tendency and sentiment intensity in text data. This application can complete the extraction of sentiment features in the following ways.

[0057] The sentiment of a text is assessed using a predefined list of positive and negative words. For example, words such as "good" and "excellent" may be labeled as positive, while words such as "bad" and "terrible" are labeled as negative. A machine learning model is trained to automatically identify the sentiment of a text. It is usually trained using a labeled dataset. The model can be based on traditional machine learning, such as support vector machines (SVMs), random forests, etc., or it can be based on deep learning, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc. In addition to sentiment, sentiment intensity is also an important aspect of sentiment analysis. Use sentiment intensity analysis tools such as VADER (Valence Aware Dictionary and sEntimentReasoner), which not only distinguishes between positive and negative sentiment, but also provides a score for sentiment intensity.

[0058] Use syntactic analysis to understand sentence structure and identify the context and intensity of sentiment words. For example, negative words (such as "not") or degree adverbs (such as "very", "extremely") can change the meaning and intensity of sentiment words. Analyze the dependencies between words in the text to determine the modifiers of sentiment words and the factors that affect the intensity of sentiment. Consider the overall context of the text, including language features such as sarcasm and metaphor, which may affect the accurate judgment of sentiment. If the text is combined with images or other modal data, sentiment feature extraction may also need to consider these multimodal information.

[0059] The steps of sentiment feature extraction include text preprocessing (such as word segmentation and stop word removal), feature extraction, model training and application. Sentiment feature extraction can help realize sentiment analysis, product review analysis, social media monitoring and other functions.

[0060] The method also includes a calculation formula for calculating the initial sentiment feature, the calculation formula is:

[0061] ;

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] in, They are the input gate, forget gate, and output gate, which respectively determine the impact of the current input on the cell state, which information in the cell state needs to be forgotten, and which information will be output as the hidden state. is the cell state, i.e. the initial emotional characteristics, is the hidden state, and are weight and bias parameters, is the sigmoid function, is the hyperbolic tangent function, is the feature weight matrix of the input gate, which is related to the input [ , ] multiply, is the hidden state at the previous time step, is the input of the current time step, is the bias term of the input gate, is the forget gate, which determines how much old information is retained in the cell state at the current time step t. It is also calculated by a neural network layer with a sigmoid activation function; is the feature weight matrix of the forget gate, is the bias term of the forget gate, is the feature weight matrix of the output gate, is the bias term of the output gate, are candidate memory cells, is the weight matrix of the candidate memory cell state, is the bias term of the candidate memory cell state, The cell state at the previous time step.

[0067] Extract the emotion change features in the time series from the user's input, and process them based on them to generate the initial emotion features. For example, when the user enters a paragraph of text, process it word by word and retain the previous emotion information. The emotional state of the current time step is determined by calculating the values ​​of the input gate, forget gate, and output gate. The cell state The long-term dependency information of emotions is saved, while the hidden state Then output the emotional features of the current moment.

[0068] S104, processing the original feature vector based on preset processing rules to generate a training set and a verification set.

[0069] In one implementation, a preset judgment matrix, feature weight information matching the original feature vector, and some engine information are obtained, and the feature weight information is calculated based on the following calculation formula, specifically: ,in, is an element in the judgment matrix, is the number of criteria, is the feature weight information.

[0070] The original feature vector and the feature weight information matching the original feature vector are processed to generate a target feature vector, wherein the target feature vector includes word frequency, inverse document frequency value and sentiment feature. Assume that this application has three features: word frequency , inverse document frequency value and emotional characteristics , and their weights are , and , the target feature vector is calculated based on the following formula: ; Among them, word frequency , inverse document frequency value and emotional characteristics , and their weights are , and .

[0071] The target feature vector and some engine information are processed to generate the engine selection result; the feature weight information is calculated based on the following calculation formula, specifically: ,in, It is in the characteristics Select Engine The probability of It is a feature In the engine The conditional probability under can be estimated through historical data. It's the engine The prior probability of The probability of It is a feature The prior probability of .

[0072] Generate training and validation sets based on the engine selection results, and select the appropriate algorithm or model based on the type of problem and data characteristics. For example, for classification problems, you may choose naive Bayes, support vector machine, random forest, etc., and use data segmentation techniques to divide the data set into training and validation sets. Usually, you can use the `train_test_split` function implemented in the `sklearn` library, which allows you to specify the proportion of the test set (for example, `test_size=0.2` means 20% of the data is used as a validation set).

[0073] The original feature vector is divided into a training set and a validation set, usually randomly allocated in a certain ratio (such as 80% training set, 20% validation set). The training set is used to train the initial document recognition model, and the model parameters are iteratively optimized. The model performance is evaluated on the validation set, and the model structure or parameters are adjusted to improve accuracy and generalization ability. Based on the performance on the validation set, the model is further optimized using techniques such as cross-validation and hyperparameter tuning. After the training and validation process is completed, an optimized digital human-driven model is obtained, which can process data and generate digital human-driven information. The digital human-driven model is comprehensively evaluated, including indicators such as accuracy, recall rate, and F1 score.

[0074] In another embodiment, adjacent features are generated based on the distance between any number of data features and other number of data features of the same category, wherein the adjacent features include a preset number of any number of data features; in the field of data analysis and machine learning, generating adjacent features based on the distance between data features is a feature engineering method that aims to discover the relationship and pattern between features. Specifically, a certain number of data features are selected from the original feature data, and the distance between any data feature and other data features of the same category is calculated using a distance metric (such as Euclidean distance, Manhattan distance or cosine similarity). Based on the calculated distance, other data features closest to any data feature are determined, and these features are considered to be adjacent. A preset number is set, that is, the number of data features that are expected to be included in the adjacent feature set. For each data feature, a preset number of data features closest to it is selected to form an adjacent feature set, and the adjacent feature set is analyzed to identify potential relationships and patterns between features, which may be crucial to understanding the use of electronic certificates, and to evaluate the importance of each data feature in the adjacent feature set, and to determine which features are most critical to predicting the target variable. That is, which places may users go to in which time periods, and whether there is a sequence of appearance between different places, etc.

[0075] If there are too many features in the adjacent feature set, it may be necessary to apply dimensionality reduction techniques (such as principal component analysis PCA) to reduce the number of features while retaining important information. The generated adjacent feature set is used to train the machine learning model to predict or classify the use of electronic certificates, evaluate the performance of the model, such as accuracy, recall rate, and F1 score, and optimize the model as needed. It may be necessary to adjust the preset number and distance measurement method multiple times to find the optimal feature set. This method can better understand the complexity of the use of electronic certificates and improve the predictive ability of the early warning model. This is very important for the security management and risk assessment of electronic certificates. At the same time, this also helps to improve the model's ability to identify abnormal behaviors and ensure the security and reliability of the electronic certificate system.

[0076] Based on the number of data features in the training set, a sampling ratio is determined; based on the sampling ratio, a sampling ratio is determined; based on the sampling ratio, adjacent features are sampled to generate a preset number of sampling features; based on any data feature and each sampling feature, multiple data groups are generated, wherein each data group contains a preset number of data samples, wherein each data group contains a preset number of data samples, wherein at least one data sample includes identification information.

[0077] In data analysis and machine learning, the sampling ratio and sampling rate are determined based on the number of data features in the training set. By counting the number of each data feature in the training set, the distribution of different features can be understood. The sampling ratio is determined based on the number of data features and the size of the data set. The sampling ratio can help us decide how many samples to extract from each feature. According to the sampling ratio, the specific sampling ratio can be calculated. The sampling ratio is the ratio of the actual number of samples to the total number. The adjacent features are sampled using the sampling ratio to generate a preset number of sampled features. This process ensures the representativeness and diversity of the samples. Through the sampling process, a set of sampled features is generated, which will be used for subsequent data analysis and model training. Multiple data sets are constructed based on any data feature and each sampled feature. Each data set contains a preset number of data samples. Ensure that the samples in each data set are diverse and contain at least one sample with identification information, which helps the model learn the feature performance in different situations. Use these data sets for model training to evaluate the contribution of different features to the prediction target. Use the sampled feature set to train the machine learning model and evaluate the performance of the model through the validation set. According to the results of model training and validation, the sampling ratio and rate may need to be adjusted to optimize the prediction ability of the model. With this approach, large-scale data sets can be processed efficiently while maintaining data quality and model generalization capabilities, helping to identify potential security risks and abnormal behaviors.

[0078] S105: Acquire an initial digital human driving model that matches the identification information.

[0079] In one implementation, obtaining an initial digital human driving model that matches the identification information involves detecting and processing abnormal user states, which may include no sound, excessive body movements, or distorted facial features. This requires a system that can accurately analyze user needs and preferences in order to provide personalized merchandise and health recommendations, and optimize based on the user's immediate feedback to improve the relevance and practicality of the recommendations. In the field of personalized recommendations, recommendation algorithms based on user behavior analysis play a vital role. For example, user behavior information is converted into a user rating matrix, and an improved regularized non-negative matrix decomposition algorithm is used to add deviation information to make full use of user behavior information such as clicks, purchases, browsing, and collections to actively recommend items of interest to users.

[0080] Based on the above content, the corresponding digital human is generated, which displays corresponding information according to the user's needs. Through AI algorithms, the facial expressions and body movements of the digital human can be generated from audio. This requires a unified framework to process facial expressions and body movements at the same time to ensure the overall coordination of the animation.

[0081] In addition, digital twins provide new opportunities for real-time monitoring of data processing, which can take into account changes in the processing and operating environment, and help improve the accuracy of anomaly detection and fault diagnosis results. This framework consists of three parts: physical products, virtual products, and data stream connections, and can be applied to real-time tool condition monitoring (TCM). The Wav2Lip algorithm is used to generate videos by voice-driven lip movement, and the accuracy of lip synchronization with audio is improved through an expert lip synchronization discriminator.

[0082] S106, training the initial digital human driving model based on the training set and the verification set to generate a target digital human driving model.

[0083] In one implementation, the initial digital human-driven model is trained based on the training set to generate a prediction result label, the initial digital human-driven model is predicted based on the validation set to generate a target result label, the prediction result label and the target result label are processed to generate a model loss function. Before training the model, the data of the training set and the validation set need to be preprocessed, which may include normalization, denoising, feature extraction and other steps to improve the training efficiency and accuracy of the model. The digital human-driven model is trained using the training set data. At this stage, the parameters of the model are optimized by learning the patterns and rules in the training data. After the training is completed, the model can generate prediction result labels for the data in the training set. These labels are the classification or regression predictions of the data based on the learned knowledge of the model. The trained model is predicted using the validation set data to generate the target result label, which is used to evaluate the performance of the model on unseen data, that is, the generalization ability of the model. The loss function is a function that measures the difference between the model prediction value and the actual value on the validation set. By calculating the loss function, the prediction error of the model can be quantified. Common loss functions include mean square error (MSE), cross entropy loss, etc., which are not limited in this application.

[0084] The value of the loss function is used to evaluate the performance of the model. If the value of the loss function is high, it means that the difference between the model's prediction and the actual label is large, and the model performance needs to be improved. According to the result of the loss function, adjust the parameters or structure of the model, such as changing the number of network layers, the number of neurons, the learning rate and other hyperparameters to optimize the model performance. According to the performance feedback on the validation set, it may be necessary to adjust and retrain the model multiple times to gradually reduce the value of the loss function and improve the prediction accuracy of the model. After the model is adjusted, the validation set is used again to evaluate the performance of the model. This process may need to be repeated multiple times until the model reaches a satisfactory performance level. Finally, an independent test set can be used to test the performance of the final model to ensure the generalization ability of the model on new data.

[0085] Throughout the process, the goal is to train a digital human-driven model that can accurately predict or classify, while ensuring that the model does not overfit (perform well on the training data, but poorly on new data) or underfit (underperform on the training data). The performance and application effect of the digital human-driven model can be improved by properly selecting the loss function and optimization algorithm, as well as adjusting the model structure and hyperparameters.

[0086] The model loss function is processed to generate a model adjustment parameter. The method also includes a calculation formula for calculating the model loss function, which is: ;in, is the model loss function, is the target result label, is the prediction result label, N is the total number of samples, and i is the index of the sample. Based on the model adjustment parameters, the initial digital human driving model is processed to generate the target digital human driving model. Through this dynamic adjustment mechanism, the system can continuously improve the selection strategy and response quality in long-term use. The loss function is usually used to measure the difference between the predicted output of the model and the actual target result. After the loss function is calculated, the gradient of each parameter needs to be calculated through the backpropagation algorithm. The gradient indicates the sensitivity of the loss function to each parameter and is the key information for adjusting the parameters during the optimization process.

[0087] Use Gradient Descent or its variants (such as Adam, RMSprop, etc.) to update the parameters of the model. During the model training process, by continuously iterating the above steps, the parameters of the model will gradually adjust to minimize the loss function. This dynamic adjustment mechanism allows the model to continuously learn and adapt to new data in the long run, thereby improving the performance and response quality of the model. In addition to model parameters, the choice of hyperparameters such as learning rate, batch size, optimizer type, etc. will also have an important impact on model performance. It is usually necessary to determine the best hyperparameter settings through experiments or automated hyperparameter optimization techniques (such as grid search, random search, Bayesian optimization, etc.).

[0088] During the training process, it is necessary to regularly evaluate the model performance on the validation set and select the best model version based on performance indicators (such as accuracy, F1 score, etc.). After the model is deployed, feedback and new data can continue to be collected, and the model can be continuously updated through online learning or regular retraining to adapt to environmental changes and data distribution shifts. In this way, the digital human-driven model can be continuously optimized to improve its accuracy and naturalness in speech synthesis, facial expression generation, body movement simulation, etc., thereby providing a more realistic and attractive interactive experience.

[0089] S107: Process the target user information to generate attribute information of the target user.

[0090] In one implementation, real-time data processing information of a target server is obtained, wherein the real-time data processing information of the target server is used to characterize data processing progress information of the target server, and data cleaning processing is performed on target user information to generate real-time physiological parameter information and to-do information of the target user.

[0091] Acquiring real-time data processing information of the target server and performing data cleaning on the target user information are two key steps that ensure the progress and quality of data processing. First of all, real-time data processing information of the target server usually involves real-time data collection, analysis and processing. An effective real-time data processing system needs to be able to handle high-concurrency data streams and ensure low latency. The core goal of real-time data processing is to be able to process, store and analyze data immediately from the source so as to provide valuable information immediately when needed.

[0092] Secondly, data cleaning of target user information is another important step to ensure data quality. Data cleaning includes identifying and correcting erroneous, incomplete, inaccurate, irrelevant or duplicate data to improve data quality and make it more suitable for data analysis or data mining. The difficulties of data cleaning include data quality issues, large data volume, diverse data sources and time costs. In order to solve these problems, professional data processing tools can be used, such as FineDataLink, which provides data filtering, adding calculated columns, data association and other functions to help users quickly complete data cleaning and processing without writing complex SQL statements, thereby improving development efficiency. By real-time monitoring of the data processing progress of the target server and thorough data cleaning of user information, real-time physiological parameter information and to-do information of the target user can be generated to provide support for decision-making.

[0093] Based on the data processing progress information of the target server, the target user's real-time physiological parameter information and the target user's to-do information are processed to generate the priority information of the target user's to-do information, and the target user's attribute information is generated based on the priority information of the target user's to-do information, wherein the target user's attribute information is used to characterize the target user's real-time to-do information. That is, the server can process various data of the target user to complete matters such as product recommendations and medical treatment, but it is also necessary to consider the target user's current voice, body movement data, facial expressions and to-do items to confirm the most urgent to-do items that the target server can handle at the moment, and use the most urgent to-do items as the attributes of the target user.

[0094] Based on the data processing progress information of the target server and the real-time physiological parameter information and to-do information of the target user, the priority of the to-do information of the target user can be generated. This process involves the real-time collection, analysis and processing of data, as well as the monitoring and evaluation of the user's physiological parameters.

[0095] First, it is necessary to collect data from multiple sources in real time, such as physiological parameter information such as the user's voice, body movements, facial expressions, and the progress and status of to-do items. The real-time collection of this data is crucial to understanding the user's current needs and urgency. Then, the collected data is analyzed through intelligent algorithms to determine which physiological parameters are beyond the normal range or which to-do items are about to reach the deadline, so as to evaluate the urgency of each to-do item. For example, a digital human system that integrates multiple AI capabilities can dynamically adjust and optimize the user's task priority based on the user's real-time physiological parameters and to-do information. In addition, using user portrait generation tools such as InstantPersonas, you can have a deeper understanding of the target user's behavior patterns, needs, and pain points, so as to more accurately generate the target user's attribute information and characterize the user's real-time to-do information. Finally, through the Todo follow-up template, you can prioritize the user's to-do items, manage the task calendar, and count the number, to ensure that the most urgent to-do items are given priority. In this way, the server can complete matters such as product recommendations and medical consultations based on the real-time data and priority information of the target user, and confirm the most urgent to-do items and process them as attributes of the target user.

[0096] By comprehensively analyzing the data processing progress information of the target server and the real-time physiological parameter information and to-do information of the target user, the priority information of the target user's to-do items can be effectively generated, and the attribute information of the target user can be generated accordingly to characterize the user's real-time to-do items. This process requires not only the technical ability of real-time data processing, but also a deep understanding of user behavior and needs.

[0097] S108: Process the attribute information of the target user based on the target digital human driving model to generate target digital human information.

[0098] In one implementation, the target user's attribute information is processed based on the target digital human driving model to generate a data processing result that matches the target user's implementation to-do information. The target user's attribute information is processed based on the target digital human driving model to generate a data processing result that matches the target user's implementation to-do information, involving the application of multiple steps and technologies.

[0099] First, the target user's real-time physiological parameter information and to-do information need to be cleaned and preprocessed to ensure the accuracy and availability of the data. This step is the basis of data processing and can be completed through professional data processing tools, such as FineDataLink, which provides functions such as data filtering and adding calculated columns to help users quickly complete data cleaning and processing without writing complex SQL statements. Then, using the digital human-driven model, combined with the user's real-time physiological parameters and to-do information, the priority information of the target user's to-do can be generated. This process can be achieved through intelligent algorithms, such as the Whenable application, which integrates AI technology to automatically convert user tasks and goal priorities into a personalized life roadmap.

[0100] Furthermore, based on the priority information of the target user's to-do information, the target user's attribute information can be generated. These attribute information can characterize the target user's real-time to-do information, such as the InstantPersonas platform, which uses AI technology to transform complex user data into clear and easy-to-understand user portraits, helping companies better position the target market, optimize product design, and develop more effective marketing strategies. Ultimately, through this dynamic adjustment mechanism, the digital human system can continuously improve selection strategies and improve response quality in long-term use. For example, the digital human can process various data of the target user, complete matters such as product recommendations and medical treatment, and optimize according to the user's instant feedback to improve the relevance and practicality of the recommendation.

[0101] In addition, when constructing a digital human system, it is necessary to consider multiple aspects such as voice input and recognition, AI interaction processing, and voice synthesis. The application of these technologies enables digital humans to interact with users more naturally. The processing of target user attribute information based on the target digital human driving model involves multiple aspects such as data cleaning, intelligent algorithm application, user portrait generation, and system optimization. Through these steps, data processing results that match the target user's to-do information can be generated.

[0102] The data processing results are processed based on the target digital human driving model to generate the digital human's text information, digital human's motion information and digital human's expression information. First, the target digital human driving model integrates the large language model LLM, the EMAGE framework and the VideoReTalking model. The digital human's text information generation can be achieved by integrating the large language model (LLM). These models have the ability to understand and generate natural language text, and are called through FastAPI, which improves the interactivity and realism of the digital human. Secondly, the generation of digital human motion information usually relies on motion capture data and advanced animation technology. For example, the EMAGE framework can generate human body movements from audio and motion masks, including face, local limbs, hands and global movements, which provides strong technical support for the generation of digital human motion information.

[0103] Finally, the generation of digital human expression information requires facial expression recognition and synthesis technology. The VideoReTalking model is an open source voice-driven facial expression model that can generate lip shapes and facial expressions synchronized with voice based on sound, achieving highly realistic dynamic portrait videos. In addition, through the deep learning model combined with audio and facial landmarks, highly realistic dynamic portrait videos are created, which support multiple languages ​​and are suitable for a variety of scenarios. Combining these technologies, the digital human-driven model can process the input text, audio and video data to generate digital humans with rich expressions and movements, providing a more natural and realistic interactive experience.

[0104] The text information, action information and expression information of the digital person are processed to generate the target digital person information, which is used to represent the information reply to the target user based on the target digital person. The digital person needs to be able to "listen" to the user's instructions, which is usually achieved through automatic speech recognition (ASR) technology to convert voice input into text. The "brain" function of the digital person is provided by the large language model (LLM), which can understand natural language input and generate appropriate response content. The digital person needs to be able to "speak", which is achieved through the speech synthesis (TTS) AI model, which converts text into an audio stream, and can choose a synthetic timbre or use real human voice to train the synthesis model. Finally, the synthesized audio stream is used to drive the digital person so that it can "speak" and "move" at the same time, including sound, action, and expression. This may require the use of different engines or AI models, such as SadTalker or RAD-NeRF, depending on the type of digital person.

[0105] Through WebSocket and other communication technologies, the audio stream and emotional data generated by the back-end controller are sent to the front-end digital human to drive it to produce corresponding actions and expressions. According to the application scenario, the digital human image, behavior logic, technology selection and engineering tuning are personalized and optimized to improve the stability and response speed of the system. Through these steps, a highly interactive digital human can be created that can understand and respond to the target user's information and provide a personalized interactive experience.

[0106] In another embodiment, text feedback: Based on the emotion classification results, the system uses a rule-based approach to generate appropriate text responses. For example, for an angry user, the system may generate soothing text. Voice feedback: The system uses speech synthesis technology to generate emotional voice responses.

[0107] Speech speed = base speed + · Emotional intensity;

[0108] Voice intonation = base pitch + · Emotional intensity;

[0109] in, and are the adjustment coefficients of the reference speed and reference pitch respectively, and the emotion intensity is determined by the emotion probability value output by the emotion recognition module.

[0110] The present application obtains target user information and a training sample set by a server, wherein the target user information includes the target user's voice data, body movement data and facial expression data within a preset time period, and the training sample set includes other user information that matches the target user information; preprocesses the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize that other user data is in an abnormal state; obtains a preset judgment matrix, feature weight information matching the original feature vector and a number of engine information; processes the original feature vector and the feature weight information matching the original feature vector to generate a target feature vector, wherein the target feature vector includes word frequency, inverse document frequency value and emotional features; processes the target feature vector and a number of engine information to generate an engine selection result; and generates a training set and a verification set based on the engine selection result.

[0111] Acquire an initial digital human driving model that matches the identification information; perform training processing on the initial digital human driving model based on the training set to generate a prediction result label; perform prediction processing on the initial digital human driving model based on the validation set to generate a target result label; process the prediction result label and the target result label to generate a model loss function; process the model loss function to generate a model adjustment parameter; process the initial digital human driving model based on the model adjustment parameter to generate a target digital human driving model.

[0112] Acquire the real-time data processing information of the target server, wherein the real-time data processing information of the target server is used to characterize the data processing progress information of the target server; perform data cleaning processing on the target user information to generate the real-time physiological parameter information and to-do information of the target user; process the real-time physiological parameter information and to-do information of the target user based on the data processing progress information of the target server to generate the priority information of the to-do information of the target user; generate the attribute information of the target user based on the priority information of the to-do information of the target user, wherein the attribute information of the target user is used to characterize the real-time to-do information of the target user; process the attribute information of the target user based on the target digital human driving model to generate a data processing result matching the implementation to-do information of the target user; process the data processing result based on the target digital human driving model to generate the text information of the digital human, the action information of the digital human and the expression information of the digital human; process the text information, the action information and the expression information of the digital human to generate the target digital human information, wherein the target digital human information is used to characterize the information reply to the target user based on the target digital human. Collect the user's voice, body movements and facial expression data, build training samples, clean the data and extract features, such as text word frequency, TF-IDF value and emotional features. Use these features to create training sets and validation sets to train the digital human driving model. Through model training, optimize parameters and generate a target digital human driving model that can accurately reflect user needs. Finally, process user information and combine the model to generate digital human information that can effectively respond to users, ensuring that digital humans can provide personalized and accurate services.

[0113] In one embodiment, if Figure 2 As shown, the present application also provides a virtual digital human management device, including:

[0114] The acquisition module 201 is used to acquire target user information and a training sample set, wherein the target user information includes voice data, body movement data and facial expression data of the target user within a preset time period, and the training sample set includes other user information matching the target user information; and acquire an initial digital human driving model matching the identification information;

[0115] The processing module 202 is used to pre-process the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize that other user data is in an abnormal state; perform feature extraction processing on the training sample set with identification information to generate an original feature vector, wherein the original feature vector includes an initial text word frequency, an initial inverse document frequency value (TF-IDF value) and an initial emotional feature, and the initial emotional feature is generated based on the voice data, body movement data and facial expression data of other users; process the original feature vector based on preset processing rules to generate a training set and a verification set; train the initial digital human driving model based on the training set and the verification set to generate a target digital human driving model; process the target user information to generate attribute information of the target user; process the attribute information of the target user based on the target digital human driving model to generate target digital human information, wherein the target digital human information is used to characterize the information reply to the target user based on the target digital human.

[0116] The present application embodiment provides an electronic device, such as Figure 3 As shown, the electronic device 3 includes a first processor 300, a memory 301, a bus 302 and a communication interface 303. The first processor 300, the communication interface 303 and the memory 301 are connected via the bus 302; the memory 301 stores a computer program that can be run on the first processor 300, and when the first processor 300 runs the computer program, the virtual digital human management method provided in any of the aforementioned embodiments of the present application is executed.

[0117] The memory 301 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 303 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0118] The bus 302 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store programs, and the first processor 300 executes the programs after receiving the execution instruction. The virtual digital human management method disclosed in any implementation of the embodiment of the present application may be applied to the first processor 300, or implemented by the first processor 300.

[0119] The first processor 300 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 300. The above-mentioned first processor 300 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be embodied as a hardware decoding processor to execute, or a combination of hardware and software modules in the decoding processor to execute. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 301, and the first processor 300 reads the information in the memory 301 and completes the steps of the above method in combination with its hardware.

[0120] The electronic device provided by the above-mentioned embodiments of the present application and the virtual digital human management method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0121] The present application embodiment provides a computer-readable storage medium, such as Figure 4 As shown, the computer-readable storage medium 401 stores a computer program, and when the computer program is read and executed by the second processor 402, the aforementioned virtual digital human management method is implemented.

[0122] The technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device (which may be an air conditioner, a refrigeration device, a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0123] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the virtual digital human management method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0124] An embodiment of the present application provides a computer program product, including a computer program, wherein the computer program is executed by a third processor to implement the method described above.

[0125] The computer program product provided by the above-mentioned embodiments of the present application and the virtual digital human management method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0126] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or still includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0127] Each embodiment in the present application is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the management method, electronic device, electronic device, and readable storage medium for evaluating virtual digital humans, since they are basically similar to the above-mentioned embodiment of the management method for virtual digital humans, the description is relatively simple, and the relevant parts can be referred to the partial description of the embodiment of the management method for virtual digital humans.

[0128] Although the present application is disclosed as above, the present application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application shall be subject to the scope defined by the claims.

Claims

1. A method for managing virtual digital humans, characterized in that: include: Acquire target user information and a training sample set, wherein the target user information includes voice data, body movement data, and facial expression data of the target user within a preset time period, and the training sample set includes other user information matching the target user information; Preprocessing the training sample set to generate a training sample set with identification information, wherein the identification information is used to indicate that other user data is in an abnormal state; Performing feature extraction processing on the training sample set with identification information to generate an original feature vector, wherein the original feature vector includes an initial text word frequency, an initial inverse document frequency value, and an initial emotional feature, and the initial emotional feature is generated based on voice data, body movement data, and facial expression data of other users; Process the original feature vector based on the preset processing rules to generate a training set and a validation set; Acquire an initial digital human driving model matching the identification information; Training the initial digital human driving model based on the training set and the verification set to generate a target digital human driving model; Processing the target user information to generate attribute information of the target user; The attribute information of the target user is processed based on the target digital human driving model to generate target digital human information, wherein the target digital human information is used to represent information reply to the target user based on the target digital human.

2. The method according to claim 1, characterized in that Performing feature extraction processing on the training sample set with identification information to generate an original feature vector includes: The method also includes a calculation formula for calculating the initial emotion feature, and the calculation formula is: ； ； ； ； ； in, They are input gate, forget gate and output gate. is the cell state, i.e. the initial emotional characteristics, is the hidden state, and are weight and bias parameters, is the sigmoid function, is the hyperbolic tangent function, is the feature weight matrix of the input gate, is the hidden state at the previous time step, is the input of the current time step, is the bias term of the input gate, It's the Gate of Forgetfulness. is the feature weight matrix of the forget gate, is the bias term of the forget gate, is the feature weight matrix of the output gate, is the bias term of the output gate, are candidate memory cells, is the weight matrix of the candidate memory cell state, is the bias term of the candidate memory cell state, is the cell state at the previous time step.

3. The method according to claim 1, characterized in that The original feature vector is processed based on the preset processing rules to generate training sets and validation sets, including: Obtaining a preset judgment matrix, feature weight information matching the original feature vector, and some engine information; Processing the original feature vector and feature weight information matching the original feature vector to generate a target feature vector, wherein the target feature vector includes a word frequency, an inverse document frequency value, and a sentiment feature; Processing the target feature vector and a number of engine information to generate an engine selection result; Generate a training set and a validation set based on the engine selection result; The method also includes a calculation formula for calculating feature weight information, and the calculation formula is: ； in, is an element in the preset judgment matrix, is the number of criteria, is the feature weight information; The method also includes a calculation formula for calculating the target feature vector, and the calculation formula is: ； Among them, word frequency , inverse document frequency value and emotional characteristics , and their weights are , and .

4. The method according to claim 3, characterized in that The target feature vector and some engine information are processed to generate an engine selection result, including: The method further includes a calculation formula for calculating the probability of selecting a target engine based on the target feature vector, and the calculation formula is: ； in, It is in the characteristics Select Engine The probability of It is a feature In the engine The conditional probability under It's the engine The prior probability of It is a feature The prior probability of .

5. The method according to claim 1, characterized in that The initial digital human driving model is trained based on the training set and the verification set to generate a target digital human driving model, including: Performing training processing on the initial digital human driving model based on the training set to generate a prediction result label; Performing prediction processing on the initial digital human driving model based on the validation set to generate a target result label; Processing the predicted result label and the target result label to generate a model loss function; Processing the model loss function to generate model adjustment parameters; Processing the initial digital human driving model based on the model adjustment parameters to generate a target digital human driving model; The method also includes a calculation formula for calculating the model loss function, which is: in, is the model loss function, is the target result label, is the predicted result label, N is the total number of samples, and i is the index of the sample.

6. The method according to claim 1, characterized in that The target user information is processed to generate attribute information of the target user, including: Acquire real-time data processing information of the target server, wherein the real-time data processing information of the target server is used to represent data processing progress information of the target server; Performing data cleaning processing on the target user information to generate real-time physiological parameter information and to-do information of the target user; Processing the real-time physiological parameter information of the target user and the to-do list of the target user based on the data processing progress information of the target server to generate priority information of the to-do list of the target user; The attribute information of the target user is generated based on the priority information of the target user's to-do information, wherein the attribute information of the target user is used to characterize the real-time to-do information of the target user.

7. The method according to claim 6, characterized in that The target digital human driving model is used to process the target user's attribute information to generate target digital human information, including: Processing the attribute information of the target user based on the target digital human driving model to generate a data processing result that matches the real-time to-do information of the target user; Processing the data processing result based on the target digital human driving model to generate text information, action information and expression information of the digital human; The text information of the digital human, the action information of the digital human and the expression information of the digital human are processed to generate target digital human information.

8. A management device for virtual digital human, characterized in that: The device comprises: An acquisition module is used to acquire target user information and a training sample set, wherein the target user information includes voice data, body movement data and facial expression data of the target user within a preset time period, and the training sample set includes other user information matching the target user information; and to acquire an initial digital human driving model matching the identification information; A processing module is used to pre-process the training sample set to generate a training sample set with identification information, wherein the identification information is used to characterize that other user data is in an abnormal state; perform feature extraction processing on the training sample set with identification information to generate an original feature vector, wherein the original feature vector includes an initial text word frequency, an initial inverse document frequency value and an initial emotional feature, and the initial emotional feature is generated based on the voice data, body movement data and facial expression data of other users; process the original feature vector based on preset processing rules to generate a training set and a verification set; train the initial digital human driving model based on the training set and the verification set to generate a target digital human driving model; process the target user information to generate attribute information of the target user; process the attribute information of the target user based on the target digital human driving model to generate target digital human information, wherein the target digital human information is used to characterize information reply to the target user based on the target digital human.

9. An electronic device, characterized in that: include: a first processor; and a memory for storing executable instructions of the first processor; Wherein, the first processor is configured to execute the virtual digital human management method described in any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the second processor, the virtual digital human management method described in any one of claims 1 to 7 is implemented.

Images