Animation virtual video generation method and system

By combining image recognition and convolutional neural network technology, the mapping relationship between head movement and facial movement is established, and the problem of unnatural matching of face expressions and head movements in anime virtual videos is solved, and the naturalness and vividness of the video is improved. .

CN118941680AInactive Publication Date: 2024-11-12HANGZHOU QIGUO CULTURE MEDIA CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410974029.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the existing animation virtual video technology, the matching of facial expressions and head movements is unnatural, resulting in insufficient vividness and naturalness of the video.

Method used

By combining image recognition detection technology and convolutional neural network, facial and head movement data of real faces is obtained, the mapping relationship between head movement and facial movement is established, and a machine learning algorithm is used to achieve natural expression changes and action matching, and through The feedback mechanism optimizes the quality of video animation.

Benefits of technology

It realizes the natural matching of facial expressions and head movements in anime virtual videos, improves the vividness and nature of the video, and enhances the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941680A_ABST
    Figure CN118941680A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of virtual videos, in particular to a cartoon virtual video generation method and system, and the method comprises the steps: obtaining a virtual three-dimensional face model; face motion data of a real face is obtained by using a face motion capture technology, and real-time motion of face key points is tracked; converting the captured facial action data into facial action units through a machine learning algorithm or a machine vision algorithm, and coding each facial action unit; the action unit codes are applied to the virtual face model, so that the virtual face model generates expression changes similar to a real face; acquiring real head motion data and head motion data features by using a head motion tracking technology; establishing a mapping relation between the head motion and the facial motion through a convolutional neural network model, and completing matching of the head motion and the expression motion; and a feedback mechanism is designed to collect feedback data, the convolutional neural network model is optimized, and the natural feeling of the video animation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of virtual video technology, and in particular to a method and system for generating cartoon virtual videos. Background Art

[0002] Animation virtual video is a video animation video produced by computer technology. It is characterized by anime style, usually with virtual characters as the protagonists, and presents lifelike character images and storylines through facial capture, 3D modeling and video animation production. Facial capture technology is widely used in the production of animation virtual video. The facial expressions and movements of real people are captured by cameras and mapped to virtual characters in real time, thereby achieving rich expression changes and vivid movement performances. The main application areas are: in the field of games, not only non-player characters (NPCs) in the game can be produced by this technology, but also virtual game anchors and spokespersons can be created to enhance the interactivity and entertainment of the game; in the field of film and television video animation, the production of animation films and series video animations is also increasingly using virtual video technology, especially in the pre-processing stage of character design, video animation testing and other aspects, which greatly improves production efficiency and quality.

[0003] In virtual videos, facial expressions are very important and directly affect the vividness and naturalness of the video. Currently, many facial movements are driven by voice or text, and very few are driven by head movements, resulting in unnatural matching between facial expressions and head movements. Summary of the invention

[0004] Based on image recognition detection technology, the present invention combines an image detection system with a medical record system and fuses multiple data to further improve the accuracy of detection and predict the possibility of future illness to solve the problems in the background technology.

[0005] The technical solution provided by the present invention is: a method for generating an animated virtual video, the method comprising:

[0006] Obtaining a virtual three-dimensional face model;

[0007] Use facial motion capture technology to obtain facial motion data of real faces and track the real-time movement of key facial points;

[0008] Through machine learning algorithms or machine vision algorithms, the captured facial action data is converted into facial action units, and each facial action unit is encoded, with each facial unit corresponding to a specific facial muscle group or expression feature;

[0009] Apply action unit coding to the virtual face model to make it produce expression changes similar to those of a real face;

[0010] Use head motion tracking technology to obtain real head motion data and head motion data features;

[0011] The mapping relationship between head movement and facial movement is established through the convolutional neural network model to complete the matching of head movement and facial expression movement;

[0012] Design feedback mechanisms, collect feedback data, optimize convolutional neural network models, and improve the quality of video animations.

[0013] Preferably, the input layer of the convolutional neural network is used to receive head motion vectors, the output layer is used to generate facial action unit coding vectors, and the intermediate layer is used to learn the mapping relationship between the head motion vector set and the facial action unit coding vector set.

[0014] Preferably, establishing a mapping relationship between head movement and facial movement through a convolutional neural network model comprises the following steps:

[0015] Build a convolutional neural network model;

[0016] Acquire head motion data and facial action unit coding data from a historical database, and perform data cleaning on the head motion data and facial action unit coding data;

[0017] Extract multiple key features from the head motion data after data cleaning;

[0018] Convert the key features and facial action unit codes into vector form to form a head motion vector set and a facial action unit code vector set;

[0019] The head motion vector set and the facial action unit vector set are divided into a training set and a test set;

[0020] Use the training set to train the convolutional neural network model and optimize the network weights through the back-propagation algorithm;

[0021] Use the test set to validate the convolutional neural network model and optimize the model performance by adjusting the hyperparameters;

[0022] Use the trained convolutional neural network model to establish the mapping relationship between head movement and facial movement.

[0023] Preferably, the method of using the trained convolutional neural network model to establish a mapping relationship between head movement and facial movement comprises the following steps:

[0024] Construct head motion feature vector;

[0025] Take the head motion feature vectors at N time points to form an N×3 matrix as the input sample of the convolutional neural network;

[0026] Constructing a facial action unit encoding vector, wherein the facial action unit encoding corresponding to the plurality of facial feature points represents a dimension;

[0027] The facial action coding vector is normalized;

[0028] Take the facial action encoding vectors of N time points to construct an N×F matrix as the output sample of the convolutional neural network, where F represents the number of facial action unit encodings.

[0029] Preferably, the step of constructing the head motion feature vector comprises the following steps:

[0030] The head motion data is collected using a sliding window to obtain the time series of the head motion data;

[0031] Obtain key features: Extract features from the head motion data within the corresponding time window and obtain the acceleration values ​​of the head motion in the three axes of X, Y, and Z;

[0032] Integrate the acceleration value to obtain the velocity of the head in the three axes of X, Y, and Z;

[0033] Get the rotation angle value of the head;

[0034] The acceleration, velocity and rotation angle values ​​are normalized to form a head motion feature vector.

[0035] According to the method for generating an animated virtual video according to claim 1, the step of constructing a facial action unit coding vector comprises the following steps:

[0036] Use facial feature point detection algorithm to automatically locate key facial feature points, including eyes, eyebrows, nose, and mouth areas;

[0037] The facial action coding system analyzes the movement of facial feature points, determines the activation of each facial action unit, and encodes it;

[0038] A temporal sequence of facial action unit codes is constructed, and each facial action unit code and head motion feature are aligned on the time axis.

[0039] Preferably, the design feedback mechanism to collect feedback data includes the following steps:

[0040] Embed a feedback module in the video, and display a questionnaire to obtain feedback data by actively sending the feedback module or by user triggering, specifically obtaining text data and rating data of the user's evaluation on the naturalness and realism of the video animation through the questionnaire;

[0041] A log recording module is set up to automatically collect the interaction data between the user and the video animation to obtain the interaction log data; the interaction data between the user and the video animation includes the expression triggering frequency.

[0042] Preferably, the optimization of the convolutional neural network model to improve the quality of the video animation comprises the following steps:

[0043] Extracting multiple head motion features from the head motion data, wherein the head motion features are one or more of speed, acceleration, direction, and rotation angle of the head motion;

[0044] After normalizing the head motion features, the head motion feature value is obtained;

[0045] After normalizing the expression triggering frequency and the scoring data, the expression reaction feature value is obtained;

[0046] Establish linear regression model 1 and linear regression model 2;

[0047] The user rating is used as the output variable of the linear regression model 1, and multiple head motion feature values ​​are used as the input variables of the linear regression model 2 to obtain the relationship between the user rating and the head motion feature, so as to improve the user rating by adjusting the head motion feature;

[0048] The expression triggering frequency is used as the output variable of the linear regression model 2, and multiple head movement feature values ​​are used as the input variables of the linear regression model 2, so as to establish the relationship between the expression triggering frequency and the head movement feature values, so as to improve the expression triggering frequency by adjusting the head movement features.

[0049] The present invention also provides a cartoon virtual video generation system, wherein the system is used to execute the cartoon virtual video generation method.

[0050] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for generating an animated virtual video.

[0051] Beneficial effects of the present invention:

[0052] 1. The present invention constructs a convolutional neural network model, including multiple convolutional layers, pooling layers and fully connected layers. Through these layers, spatial hierarchical features can be automatically learned from input data. ReLU is selected as the activation function. Head movement data (such as accelerometer or gyroscope data) and facial action unit (AUs) encoding data are collected from the historical database to train the CNN model; the mapping relationship between head movement and facial movement is established using the trained CNN model, the complex relationship between head movement and facial expression is captured, and automatic prediction from head movement data to facial action units is realized.

[0053] 2. The present invention embeds a feedback module in the video and creates a questionnaire containing questions about the naturalness and realism evaluation of the video animation. In actual operation, the questionnaire can be a text question or a scoring system. At an appropriate time during the playback of the video animation, such as when the video animation ends or through a button triggered by the user, the questionnaire is displayed to collect user feedback. Users can enter text data in the questionnaire to express their views on the video animation, and can also give a rating, such as a 1 to 5-star rating; the user's text and rating data are stored in a database for subsequent analysis and processing.

[0054] 3. The present invention sets a log recording module to automatically record the interaction events between users and video animations, such as the frequency of expression triggering, the speed of head movement, etc. Collect all relevant data generated by users during viewing or interaction in order to analyze user behavior and video animation performance. Analyze the frequency of expression triggering to understand which expressions are more popular and which expressions may be too frequent or unnatural. By analyzing the user's operating habits and viewing time, identify user preferences and possible improvement points.

[0055] 4. In the present invention, linear regression model 1 and linear regression model 2 are set up, wherein linear regression model 1 is used to analyze and predict users' evaluation of the naturalness of video animation. The regression coefficient of model 1 can be used to analyze the size and direction of the influence of each head movement feature on the user's score. Linear regression model 1 can help us understand how head movement features affect user scores and optimize video animation effects accordingly; linear regression model 2 is used to analyze and predict the influence of head movement on expression triggering. This model can help us understand how head movement features affect the triggering frequency of expressions. Through its regression coefficient, we can analyze the size and direction of the influence of each head movement feature on the triggering frequency of expressions, thereby improving the naturalness of video animation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0057] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0058] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0059] Please combine Figure 1 The present invention provides a method for generating an animation virtual video, the method comprising the following steps:

[0060] Obtaining a virtual three-dimensional face model;

[0061] Facial motion capture technology is used to obtain facial motion data of real faces and track the real-time movement of key facial points. Specifically, high-precision cameras, infrared sensors or 3D scanners are used to let participants perform predetermined activities in a controlled environment, and their facial motion data are recorded using facial capture equipment.

[0062] Through machine learning algorithms or machine vision algorithms, the captured facial action data is converted into facial action units, and each facial action unit is encoded, with each facial unit corresponding to a specific facial muscle group or expression feature;

[0063] Specifically, machine vision algorithms are used to automatically detect key facial feature points, such as the positions of eyes, nose, and mouth. Based on the positions of feature points, facial geometric features are calculated, such as the distance between eye corners and the degree of upturned corners of the mouth. Machine learning models can choose support vector machines (SVMs), random forests, or deep learning networks. The model is trained using annotated facial action data and corresponding facial action unit labels.

[0064] Apply action unit coding to the virtual face model to make it produce expression changes similar to those of a real face;

[0065] Using head motion tracking technology to obtain real head motion data and head motion data features; the head motion tracking technology is to collect head motion data through head tracking equipment, such as head-mounted sensors (including accelerometers and inertial sensors, etc.) and optical tracking systems;

[0066] The mapping relationship between head movement and facial movement is established through the convolutional neural network model to complete the matching of head movement and facial expression movement;

[0067] Design feedback mechanisms, collect feedback data, optimize convolutional neural network models, and improve the quality of video animations.

[0068] First, we need to build a convolutional neural network model, including multiple convolutional layers, pooling layers, and fully connected layers. These layers can automatically learn spatial hierarchical features from the input data. Select ReLU as the activation function. Collect head motion data (such as accelerometer or gyroscope data) and facial action unit (AUs) encoding data from the historical database. Clean the collected data, remove invalid or damaged records, and ensure data quality and consistency.

[0069] Then, key features that can characterize the motion characteristics, such as speed, acceleration, rotation angle, etc., are extracted from the head motion data. The selected features are standardized or normalized to make them suitable as inputs to the neural network. The extracted key features and facial action unit codes are converted into vector form to form a head motion vector set and a facial action unit code vector set. If the data is in the form of a time series, a time window or a recurrent neural network (RNN) is used for processing.

[0070] The vector set is randomly split into training and test sets, with a common ratio of 80% training and 20% test.

[0071] Use the training set to train the CNN model and optimize the network weights through forward propagation and back propagation algorithms. Adjust the network's hyperparameters, such as learning rate, batch size, regularization parameters, etc., through performance feedback on the validation set. Use the test set to evaluate the trained CNN model, focusing on indicators such as accuracy, recall, F1 score, etc. Continue to adjust the model structure or hyperparameters based on the test results until satisfactory performance is achieved.

[0072] Use the trained CNN model to establish the mapping relationship between head movement and facial movement, and realize the automatic prediction from head movement data to facial action units. In real-time applications, the model can be deployed on the device to capture head movement in real time and predict the corresponding facial expression changes. This is achieved through the following steps: Use the trained convolutional neural network (CNN) model to establish the mapping relationship between head movement and facial movement to capture the complex relationship between head movement and facial expression.

[0073] First, construct a head motion feature vector: extract key features from the head motion data, such as accelerometer data, velocity data, or head rotation angle.

[0074] Combine these features into a vector, where each feature corresponds to a dimension in the vector, for example where a i 、v i 、c iThey represent the acceleration, velocity and rotation angle of the head at the i-th time point respectively;

[0075] The head motion data of N time points are collected through a sliding window, and the corresponding timestamps are added. The head motion feature vectors of N consecutive time points are selected to form an N×3 matrix, where 3 is the number of dimensions of the head motion data.

[0076] The N×3 matrix is ​​specifically expressed as: After normalization: matrix As an input sample of the CNN model for subsequent training or prediction.

[0077] Then, a facial action unit coding vector is constructed: the facial expression is coded according to the Facial Action Coding System (FACS) to obtain the AU code corresponding to each facial feature point. The facial expression is obtained through video or by capturing the face of the person being photographed through a camera device.

[0078] Convert the AU code of each facial feature point into a vector form, where each AU corresponds to a dimension in the vector. For example, the AU code at time point j is in vector form: where y 1 ,y 2 ,y 3 …、y m Represents AU coding, j≤N.

[0079] After normalizing the facial action encoding vector, we obtain the normalized

[0080] Finally, construct the CNN output sample: select the facial action encoding vector at the same time point as the head motion data, and construct an N×F matrix, where F represents the number of facial action unit encodings, F≤m.

[0081] The N×F matrix can be specifically expressed as:

[0082] This N×F matrix Q is used as an output sample of the CNN model to train the model to learn the mapping relationship from head movement to facial movement.

[0083] In order to optimize the video animation display of human faces in the video, the present invention sets a feedback mechanism, specifically:

[0084] Embed a feedback module in the video video to create a questionnaire containing questions about the naturalness and realism of the video animation. In actual operation, the questionnaire can be a text question or a rating system. At the appropriate time of the video animation, such as when the video animation ends or through a button triggered by the user, display the questionnaire to collect user feedback. Users can enter text data in the questionnaire to express their opinions on the video animation, and can also give ratings, such as 1 to 5 star ratings. Store the user's text and rating data in the database for subsequent analysis and processing.

[0085] Set up a logging module to automatically record user interaction events with video animations, such as the frequency of expression triggering, the speed of head movement, etc. Collect all relevant data generated by users during viewing or interaction in order to analyze user behavior and video animation performance. Analyze the frequency of expression triggering to understand which expressions are more popular and which expressions may be too frequent or unnatural. By analyzing user operating habits and viewing time, identify user preferences and possible improvement points.

[0086] Using the collected data, we fine-tune the video animation system, such as adjusting the sensitivity of expression triggers or modifying the transition effects of video animations. Based on user feedback and interaction data, we continuously iterate and update the video animation system to improve naturalness and user satisfaction.

[0087] Use statistical software or programming environment (such as Python's scikit-learn library) to build linear regression model 1. Build linear regression model 1 to analyze and predict users' evaluation of the naturalness of video animation. Model 1 can help us understand how head movement characteristics affect users' ratings and optimize video animation effects accordingly.

[0088] Linear regression model 1 can be expressed as: 1 =β 0 +β 1 X 1 +β 2 X 2 +…+β l X l +δ, where β 0 represents the intercept, β 0 , β 1 , β 2 , …, β l , δ represent the regression coefficient and error parameter respectively;

[0089] Head movement data and scoring data are obtained from the database, and after normalization, the data are divided into training set and test set. Training is performed using the training set, and model evaluation is performed using the test set. The determination coefficient is used to evaluate the model's fit. The regression coefficient of model one can be used to analyze the size and direction of the impact of each head movement feature on the user's score. Based on the results of the analysis, suggestions are put forward to improve the naturalness of the video animation, such as adjusting the speed or smoothness of the head movement.

[0090] Similarly, a linear regression model 2 is established using statistical software or programming environment (such as Python's scikit-learn library) to analyze and predict the impact of head movement on expression triggering. This model can help us understand how head movement characteristics affect the triggering frequency of expressions and optimize video animation effects accordingly.

[0091] Linear regression model 2 can be expressed as: y 2 =α 0 +α 1 X 1 +α 2 X 2 +…+α l X l +θ, where α 0 represents the intercept of linear regression model 2, α 1 , α 2 , …, α l , θ represent the regression coefficient and error parameter of linear regression model 2 respectively.

[0092] The linear regression model 2 is trained using the method mentioned above. Through its regression coefficient, the magnitude and direction of the impact of each head movement feature on the expression triggering frequency can be analyzed. Based on the analysis results, suggestions are put forward to improve the naturalness of video animation.

[0093] The present invention also provides a cartoon virtual video generation system, wherein the system is used to execute a cartoon virtual video generation method.

[0094] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for generating an animated virtual video.

[0095] The embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the above-mentioned computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0096] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0097] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be deformed or modified in any way.

Claims

1. A method for generating an animated virtual video, It is characterized in that The method comprises: Obtaining a virtual three-dimensional face model; Use facial motion capture technology to obtain facial motion data of real faces and track the real-time movement of key facial points; Through machine learning algorithms or machine vision algorithms, the captured facial action data is converted into facial action units, and each facial action unit is encoded, with each facial unit corresponding to a specific facial muscle group or expression feature; Apply action unit coding to the virtual face model to make it produce expression changes similar to those of a real face; Use head motion tracking technology to obtain real head motion data and head motion data features; The mapping relationship between head movement and facial movement is established through the convolutional neural network model to complete the matching of head movement and facial expression movement; Design feedback mechanisms, collect feedback data, optimize convolutional neural network models, and improve the quality of video animations.

2. A method for generating an animated virtual video according to claim 1, It is characterized in that The input layer of the convolutional neural network is used to receive the head motion vector, the output layer is used to generate the facial action unit coding vector, and the intermediate layer is used to learn the mapping relationship between the head motion vector set and the facial action unit coding vector set.

3. A method for generating an animated virtual video according to claim 1, It is characterized in that The mapping relationship between head movement and facial movement is established by using a convolutional neural network model, comprising the following steps: Build a convolutional neural network model; Acquire head motion data and facial action unit coding data from a historical database, and perform data cleaning on the head motion data and facial action unit coding data; Extract multiple key features from the head motion data after data cleaning; Convert the key features and facial action unit codes into vector form to form a head motion vector set and a facial action unit code vector set; The head motion vector set and the facial action unit vector set are divided into a training set and a test set; Use the training set to train the convolutional neural network model and optimize the network weights through the back-propagation algorithm; Use the test set to validate the convolutional neural network model and optimize the model performance by adjusting the hyperparameters; Use the trained convolutional neural network model to establish the mapping relationship between head movement and facial movement.

4. A method for generating an animated virtual video according to claim 3, It is characterized in that The method of using the trained convolutional neural network model to establish a mapping relationship between head movement and facial movement includes the following steps: Construct head motion feature vector; Take the head motion feature vectors at N time points to form an N×3 matrix as the input sample of the convolutional neural network; Constructing a facial action unit encoding vector, wherein the facial action unit encoding corresponding to the plurality of facial feature points represents a dimension; The facial action coding vector is normalized; Take the facial action encoding vectors of N time points to construct an N×F matrix as the output sample of the convolutional neural network, where F represents the number of facial action unit encodings.

5. A method for generating an animated virtual video according to claim 4, It is characterized in that The step of constructing the head motion feature vector comprises the following steps: The head motion data is collected using a sliding window to obtain the time series of the head motion data; Obtain key features: Extract features from the head motion data within the corresponding time window and obtain the acceleration values ​​of the head motion in the three axes of X, Y, and Z; Integrate the acceleration value to obtain the velocity of the head in the three axes of X, Y, and Z; Get the rotation angle value of the head; The acceleration, velocity and rotation angle values ​​are normalized to form a head motion feature vector.

6. A method for generating an animated virtual video according to claim 4, It is characterized in that The step of constructing a facial action unit encoding vector comprises the following steps: Use facial feature point detection algorithm to automatically locate key facial feature points, including eyes, eyebrows, nose, and mouth areas; The facial action coding system analyzes the movement of facial feature points, determines the activation of each facial action unit, and encodes it; A temporal sequence of facial action unit codes is constructed, and each facial action unit code and head motion feature are aligned on the time axis.

7. The method for generating an animated virtual video according to claim 1, It is characterized in that The design feedback mechanism collects feedback data, including the following steps: Embed a feedback module in the video, display a questionnaire through the feedback module actively or triggered by the user, and obtain feedback data, specifically obtaining text data and rating data of the user's evaluation on the naturalness and realism of the video animation through the questionnaire; A log recording module is set up to automatically collect the interaction data between the user and the video animation to obtain the interaction log data; the interaction data between the user and the video animation includes the expression triggering frequency.

8. The method for generating an animated virtual video according to claim 1, It is characterized in that The optimization of the convolutional neural network model to improve the quality of the video animation includes the following steps: Extracting multiple head motion features from the head motion data, wherein the head motion features are one or more of speed, acceleration, direction, and rotation angle of the head motion; After normalizing the head motion features, the head motion feature value is obtained; After normalizing the expression triggering frequency and the scoring data, the expression reaction feature value is obtained; Establish linear regression model 1 and linear regression model 2; The user rating is used as the output variable of the linear regression model 1, and multiple head motion feature values ​​are used as the input variables of the linear regression model 2 to obtain the relationship between the user rating and the head motion feature, so as to improve the user rating by adjusting the head motion feature; The expression triggering frequency is used as the output variable of the linear regression model 2, and multiple head movement feature values ​​are used as the input variables of the linear regression model 2, so as to establish the relationship between the expression triggering frequency and the head movement feature values, so as to improve the expression triggering frequency by adjusting the head movement features.

9. An animation virtual video generation system, It is characterized in that The system is used to execute the method for generating an animated virtual video as described in any one of claims 1 to 8.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for generating an animated virtual video as described in any one of claims 1 to 8.