An AI algorithm driven and interactive entertainment system of a virtual digital human

Abstract

The application discloses an AI algorithm driving and interactive entertainment system of a virtual digital person, and relates to the technical field of cross of virtual digital person and artificial intelligence of digital cultural creativity, and the system comprises a semantic emotion dynamics causal reasoning module, which is used for synchronously extracting hidden layer semantic features during word-by-word decoding of a large language model and obtaining skeleton rotation parameters and facial mixed deformation weights through three-layer directed causal graph reasoning, a personality constraint module, which is used for embedding a low-rank adaptive matrix into an attention layer to constrain role personality consistency, and supporting multi-user sharing of personality parameters and independent cache isolation coexistence, and a cloud-edge collaborative rendering module, which is used for maintaining a unified skeleton causal topology on a cloud side, issuing individualized emotional response incremental parameters to an edge terminal, and binding and rendering a covariance matrix of a three-dimensional Gaussian element and spherical harmonic coefficients, so that the application shortens interaction delay, matches action expression with semantic intention, and reduces rendering computing power consumption in a high-concurrency scene.

Description

Technical Field

[0001] This invention relates to the field of digital cultural and creative virtual digital humans and artificial intelligence, specifically to an AI algorithm-driven and interactive entertainment system for virtual digital humans. Background Technology

[0002] A virtual digital human system typically consists of four interconnected parts: a large language model dialogue generation module, a text-to-speech module, an audio-driven facial and body animation module, and a 3D rendering engine. The large language model generates response text based on user input, the text-to-speech module synthesizes the response text into a speech waveform, and the audio-driven module extracts driving parameters based on the phoneme features, pitch, and energy envelope of the sound, controls the playback of preset facial hybrid deformation bases and skeletal animation libraries, and finally outputs a visualized digital human image through the rendering engine.

[0003] In the prior art closest to this invention, virtual digital human interaction systems generally employ the following serial processing pipeline:

[0004] First, the large language model receives user input and generates a complete response text;

[0005] The text-to-speech module then synthesizes speech audio using the reply text as input.

[0006] Next, the audio-driven algorithm extracts acoustic features such as phoneme timestamps, fundamental frequency, and energy based on the speech waveform, and drives a pre-trained or rule-defined lip-sync model and a limited facial expression library accordingly.

[0007] Finally, the rendering engine renders the driven skeleton and hybrid deformation parameters into a visual image, and presents it to the user in the form of a video stream or local rendering. In terms of personality consistency maintenance, existing solutions usually concatenate information such as character setting text and historical dialogue records into the prompt words and input them into the model before each call to the large language model, so as to constrain the consistency of the character's language style and behavior. In terms of rendering and distribution, existing solutions mostly adopt centralized cloud rendering and then distribute it to each terminal user in the form of a video stream, or use low-poly models for local rendering on the terminal to reduce computing power requirements.

[0008] The aforementioned prior art has the following drawbacks:

[0009] Firstly, because the large language model needs to generate the complete response text before speech synthesis can be started, the audio-driven animation can be started only after the complete audio is synthesized, and the rendering can only be performed after the animation is completed, the four steps are strictly executed in sequence, resulting in a high end-to-end interaction delay. Users can clearly perceive the sluggish response, which destroys the immersive interactive experience.

[0010] Secondly, since the audio driving module can only extract driving parameters from the physical acoustic features of the sound, and the acoustic features cannot fully carry the deep semantic intent behind the text, such as irony, puns and other complex emotional expressions, even if the large language model generates semantically rich text, as long as the tone of the speech synthesis is flat, the subsequent driving algorithm will inevitably only generate flat expressions that match the tone, and will not be able to generate fine facial micro-expressions and body movements that match the semantic intent, thus forming a problem of separation between semantic and action expression.

[0011] Third, since the existing personality maintenance scheme relies on splicing the character settings and historical dialogue text into the prompt words, when the number of concurrent users or the number of dialogue rounds increases, the length of the prompt words expands accordingly, causing the reasoning cost to increase significantly with the increase of the context length. Furthermore, the large language model is prone to attention dilution in long contexts, which can lead to the forgetting of personality settings or the collapse of personality consistency due to contradictions.

[0012] Fourth, since the existing rendering solution centralizes high-fidelity rendering tasks in the cloud and allocates independent rendering instances one by one according to the number of users, the computing resources required for rendering increase linearly with the number of concurrent users. When the concurrency scale expands, the cloud computing cost and hardware deployment scale are difficult to bear, and the only compromise is to reduce the rendering accuracy or increase the rendering queuing delay.

[0013] In summary, existing virtual digital human interaction systems suffer from technical bottlenecks that are difficult to balance in terms of interaction latency, semantic and action matching precision, personality consistency maintenance, and rendering computational power consumption in high-concurrency scenarios. There is an urgent need for a virtual digital human driving solution that can solve the above problems from the underlying architecture. Summary of the Invention

[0014] The technical problems to be solved by the present invention include: the existing virtual digital human interaction system adopts a serial processing architecture of text generation, speech synthesis and audio driving, which results in high end-to-end interaction latency and a disconnect between semantics and action expression, making it difficult to generate refined facial micro-expressions and body movements that match deep semantics.

[0015] In high-concurrency interactive entertainment scenarios, existing personality maintenance schemes based on prompt word concatenation are prone to causing context window expansion, increased reasoning costs, and personality consistency collapse.

[0016] The computing power consumption of existing cloud-based centralized high-fidelity rendering solutions increases linearly with the number of concurrent users, making it difficult to balance rendering quality with high concurrency, low latency, and real-time performance.

[0017] To address the aforementioned technical problems, this invention provides an AI algorithm-driven and interactive entertainment system for virtual digital humans, including a semantic emotion dynamics causal reasoning module, a personality constraint module, and a cloud-edge collaborative rendering module.

[0018] The semantic-emotional dynamics causal reasoning module is used to extract the hidden layer semantic feature vector corresponding to the current word in real time during the process of decoding and generating reply text word by word in the large language model. Based on the pre-trained and solidified three-layer directed causal graph, the semantic feature vector is mapped to the emotional state vector in sequence. Then, the emotional state vector and the semantic feature vector are jointly mapped to the skeleton rotation parameters and facial hybrid deformation weights to obtain the dynamic parameters used to drive the virtual digital human's movements. Thus, the causal reasoning from semantics to dynamic parameters can be completed synchronously at the word level without waiting for the text-to-speech process to be completed.

[0019] The personality constraint module is used to embed the low-rank adaptation matrix pre-trained for the digital human character into the attention layer of the large language model, correct the attention score matrix in the decoding process of the large language model, so that the corrected hidden layer features are subject to the underlying constraint of the unified personality features, and the constrained hidden layer features are passed to the semantic emotion dynamics causal reasoning module as the correction input for the micro-expression baseline state, thereby replacing the surface cue word splicing method with a deep embedding of the attention layer to achieve personality consistency maintenance.

[0020] The cloud-edge collaborative rendering module is used to maintain unified skeletal causal topology data of virtual digital humans in the cloud. For each user session, it calculates the skeletal rotation offset and facial hybrid deformation weight offset relative to the unified skeletal causal topology data. The obtained offset is sent to the edge terminal device as a personalized emotional response increment parameter. The edge terminal device binds the personalized emotional response increment parameter with the covariance matrix and spherical harmonic function coefficients in the locally preset 3D Gaussian meta-database. Then, it performs rasterization rendering on the bound Gaussian meta-database, so that the rendering computing power consumption no longer increases linearly with the number of concurrent users.

[0021] Compared with the prior art, the beneficial effects of the present invention by adopting the above technical solution are as follows:

[0022] 1. This invention, by simultaneously performing causal reasoning from semantics to dynamic parameters while decoding lexical units in a large language model, bypasses the waiting stage of text-to-speech conversion, which can significantly shorten end-to-end interaction latency and effectively improve the real-time response in interactive entertainment scenarios.

[0023] 2. Because the dynamic parameters of this invention are directly derived from semantic features through causal graph reasoning, rather than indirectly derived from acoustic features, the generated micro-expressions and body movements can be truly based on semantic understanding rather than acoustic features, thus solving the problem of the separation between semantic and action expression;

[0024] 3. This invention deeply embeds the personality constraint vector into the attention layer in the form of a low-rank adaptation matrix, so that the personality consistency constraint does not decay with the increase of the number of concurrent users or the number of dialogue rounds. At the same time, each user shares the same personality weight matrix while independently maintaining the historical dialogue cache, thus avoiding the increase in reasoning cost caused by the expansion of the context window.

[0025] 4. This invention maintains a unified skeletal topology in the cloud and sends personalized emotional response incremental parameters to edge terminals with a data volume much smaller than that of a complete rendering frame. Combined with adaptive rasterization rendering on the terminal side, it decouples rendering computing power consumption from the number of concurrent users, and balances rendering quality with high concurrency and low latency real-time performance under the same system architecture. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall system architecture in an embodiment of the present invention;

[0027] Figure 2 This is a flowchart of a three-layer directed causal graph in an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the online reasoning process for semantic emotion dynamics causal reasoning in an embodiment of the present invention. Detailed Implementation

[0029] The following is in conjunction with the appendix Figure 1-3 The specific embodiments of the present invention will be further described below. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0030] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0031] The virtual digital human AI algorithm-driven and interactive entertainment system provided in this embodiment consists of a user interaction terminal, a cloud inference service cluster, and an edge terminal device. The three are connected through a network. The cloud inference service cluster includes a large language model decoding module, a semantic emotion dynamics causal inference module, and a personality constraint module. The three work together on the same data pipeline, replacing the traditional serial processing architecture of text generation, speech synthesis, and audio driving. The edge terminal device includes a three-dimensional Gaussian meta-library, a covariance matrix and spherical harmonic coefficient binding unit, and a rasterization rendering output unit. The user interaction terminal sends text or voice interaction requests to the cloud inference service cluster. The large language model decoding module decodes word by word to generate reply text. A bidirectional data path is established between the semantic emotion dynamics causal inference module and the large language model decoding module. Simultaneously, the hidden layer features of the large language model are received and dynamic parameters are output while word generation is being performed.

[0032] The personality constraint module is bypassed and connected to the attention layer of the large language model decoding module. The cloud inference service cluster sends the unified skeletal causal topology data and personalized emotional response incremental parameters to the edge terminal device via the network. The edge terminal device completes the parameter binding and rendering, and outputs the rendered screen to the display part of the user interaction terminal.

[0033] I. Construction and Online Reasoning of the Semantic Emotional Dynamics Causal Reasoning Module

[0034] The construction process of the three-layer directed causal graph used in the semantic-emotion-dynamic causal reasoning module is as follows: First, a multimodal training dataset is collected. Each training sample contains a quadruple consisting of a text fragment, a real emotion label, a high-precision 3D facial hybrid deformation weight sequence, and a skeletal rotation quaternion sequence. By aligning timestamps, the text, emotion label, and dynamic parameters are mapped one-to-one on the time axis, forming causal supervision data for training. Based on this, a cross-modal encoder is constructed using a graph neural network framework. The aligned text features, emotion labels, and dynamic parameters are mapped to the same latent space, and the encoder is trained in a supervised manner to learn the causal mapping between semantic nodes, emotion nodes, and dynamic nodes. After training, the intermediate layer parameters of the cross-modal encoder are solidified into a three-layer directed causal graph edge weight matrix, forming a causal graph structure containing semantic node layer, sentiment node layer and dynamic node layer. Each node in the semantic node layer corresponds to the hidden layer semantic features of the large language model at a certain time when generating a word. Each node in the sentiment node layer represents the sentiment state mapped from the semantic node. Each node in the dynamic node layer corresponds to the skeletal rotation parameters or facial hybrid deformation weights. The semantic node layer and the sentiment node layer, as well as the sentiment node layer and the dynamic node layer, are connected by directed edges. Horizontal connection edges are also set between the nodes inside the sentiment node layer to represent the temporal continuity constraints of the sentiment state.

[0035] In actual interaction, for each word generated by the large language model, the hidden layer feature vector corresponding to that word is immediately extracted and used as the input to the semantic node layer in the three-layer directed causal graph. Without waiting for the complete text or speech to be generated, the causal graph receives the semantic feature and performs graph neural network message passing calculations in the order of propagation from the semantic node layer to the emotion node layer and from the emotion node layer to the dynamics node layer. It outputs the skeletal rotation parameters and facial hybrid deformation weights corresponding to the current moment and writes the obtained dynamic parameters into the dynamics parameter buffer queue in the order of word generation. This allows the cloud-edge collaborative rendering module to read and drive the digital human model in real time, thereby realizing synchronous pipeline processing of semantic decoding and action generation and eliminating the waiting time in the speech synthesis stage in the traditional architecture.

[0036] The mapping relationship from semantic nodes to sentiment nodes can be represented as:

[0037] ;

[0038] in, The large language model is represented in the first... The hidden layer feature vector output at each word decoding time. This represents the mapping function from semantic nodes to sentiment nodes. This represents the trainable parameters of the mapping function. Indicates the first The emotional state vector obtained through time-matter reasoning.

[0039] The mapping relationship from emotion nodes to dynamic nodes can be represented as:

[0040] ;

[0041] in, Indicates the first The dynamic parameter vector output at any given time includes skeletal rotation quaternion components and facial hybrid deformation weight components. This represents the mapping function from emotion nodes to dynamic nodes. This represents the trainable parameters of the mapping function.

[0042] The computation of message passing on a causal graph by a graph neural network can be represented as follows:

[0043] ;

[0044] in, Represents a node In the The message aggregation results of the layer, Represents a node The set of adjacent nodes in a causal graph Indicates the first The trainable weight matrix of the layer, and Representing nodes respectively and nodes In the Layer feature representation, This represents a vector concatenation operation. Representing a nonlinear activation function, the graph neural network described above enables end-to-end causal reasoning from semantic features to emotional state and then to dynamic parameters through hierarchical message passing on a three-layer directed causal graph. This allows the generated micro-expressions and body movements to reflect the emotional intent behind the semantics. For example, when a semantic feature corresponds to a sarcastic intent, the causal graph can infer and output a matching combination of facial deformation weights such as raised eyebrows and pursed lips, rather than just an expression that matches a flat tone of voice.

[0045] II. Construction of the Personality Constraint Module and Concurrent Inference

[0046] The personality constraint module is used to deeply embed the personality characteristics of the digital human character into the attention layer of the generative model in the form of a low-rank matrix, taking the representative layer prompt word concatenation method. Specifically, the textual descriptions of the character's worldview, language style, micro-expression baseline state, and action habits are first input into the pre-trained text encoder to obtain a fixed-dimensional personality embedding vector.

[0047] Then, a pair of low-rank matrices are constructed as the dimension reduction matrix and the dimension increase matrix, respectively. With the personality embedding vector as a condition, the low-rank matrix is ​​fine-tuned and trained using a small amount of example dialogue and action data of the character, so that it can learn the attention bias pattern of the character without changing the parameters of the backbone network of the large language model.

[0048] The trained low-rank adaptation matrix is ​​then solidified into a personality weight matrix specific to the digital human character and deployed to the cloud-based inference service as a fixed parameter shared by the character in all sessions.

[0049] When multiple users send high-concurrency requests to the inference service, the personality weight matrix of all requests is set to the same shared parameter. At the same time, each user is independently allocated and maintains an attention key-value cache space for their historical dialogues. The cache spaces of each user do not overwrite or interfere with each other during the calculation process. When a response is generated each time, the attention calculation result is affected by both the shared personality weight matrix and the independent cache of each user. The output hidden layer features contain both the user's personalized dialogue context information and are subject to the underlying constraints of the unified personality features. The constrained hidden layer features are then passed to the semantic emotion dynamics causal inference module as the correction input for the micro-expression baseline state, thereby ensuring that the core worldview, micro-expression baseline and action habits presented by the digital human character remain consistent regardless of the number of concurrent users.

[0050] The calculation of personality constraints at the attention level can be expressed as:

[0051] ;

[0052] in, This represents the original weight matrix of the attention layer in the large language model. This represents the dimension reduction matrix in the low-rank fitness matrix. This represents the increasing-dimensional matrix in the low-rank fitting matrix. This represents the low-rank increment matrix obtained from training based on personality traits. This represents the weight matrix after personality constraint correction.

[0053] Attention calculation after incorporating personality constraints can be expressed as:

[0054] ;

[0055] in, , , These represent the query matrix, key matrix, and value matrix in the attention mechanism, respectively. The dimension of the key vector is used to scale the attention score. This represents the attention bias matrix calculated from the personality weight matrix. The normalized exponential function is represented by the fact that since the personality weight matrix is ​​shared among multiple users and does not need to be concatenated into the input sequence with each dialogue turn, the inference cost does not increase linearly with the length of the context, and the problem of forgetting personality settings caused by attention dilution in long contexts is avoided.

[0056] III. Cloud-Edge Collaborative Lightweight Rendering

[0057] The cloud-edge collaborative rendering module addresses the issue of high-fidelity rendering computational power increasing linearly with the number of concurrent users. It decouples the rendering load by dividing tasks between the cloud and edge terminal devices. Specifically, the cloud maintains a unique skeletal tree structure and joint rotation constraint parameters for the digital human character, serving as unified skeletal causal topology data shared by all user sessions. This topology data does not incur additional storage or computational overhead due to the increase in the number of concurrent users. For each concurrent user's independent session, based on the user's dialogue context, the semantic emotion dynamics causal inference module calculates the skeletal rotation offset and facial hybrid deformation weight offset relative to the unified skeletal causal topology data. These serve as personalized emotional response increment parameters unique to that user. The cloud only distributes these personalized emotional response increment parameters to the corresponding user's edge terminal device via the network, without performing a complete per-user rendering calculation in the cloud. As a result, the amount of data distributed is far less than the amount of data in a complete rendering frame or complete skeletal parameters.

[0058] After receiving the personalized emotional response incremental parameters, the edge terminal device explicitly binds them with the covariance matrix and spherical harmonic function coefficients in the locally pre-set 3D Gaussian meta-database to obtain the set of Gaussian meta-data parameters used for rendering at that moment. The terminal adaptively adjusts the rendering detail level of the Gaussian meta-data according to its own computing power level, performs rasterization rendering on the bound Gaussian meta-data set, and directly generates the image locally without relying on cloud video stream encoding and network transmission, thereby eliminating rendering queuing bottlenecks and bandwidth pressure.

[0059] The explicit binding of 3D Gaussian sputtering rendering parameters to dynamics output can be expressed as:

[0060] ;

[0061] in, Indicates the first in the scene One original Gaussian element, Let Gausky's element be the covariance matrix. This represents the spherical harmonic color coefficient of the Gaussian element. Indicates by the first Time-based dynamic parameters The calculated rotation transformation matrix, Indicates by the first Emotional state at all times The calculated color correction offset, This refers to the Gaussian meta-parameters used for rendering after being modified by binding dynamic parameters with emotional state.

[0062] IV. Integrated Interactive Implementation Example

[0063] To illustrate the overall working process of this invention, take a specific interactive scenario as an example. When the user's interactive terminal sends a dialogue request with irony to the cloud inference service cluster, the large language model decoding module starts to generate the reply text word by word. At the same time, the personality constraint module applies the low-rank adaptation matrix exclusive to the digital human character to the current attention layer, so that the decoding process is constrained by the unified personality characteristics of the character.

[0064] The semantic emotion dynamics causal reasoning module extracts the corresponding hidden layer features at the same time as each word is generated. Through three layers of directed causal graph reasoning, it identifies the ironic emotional tendency carried by the word and outputs the corresponding facial mixed deformation weights such as raised eyebrows and unilateral upward corner of the mouth, as well as the corresponding skeletal rotation parameters. These are written into the dynamic parameter buffer queue. The cloud inference service cluster calculates the user's exclusive personalized emotional response incremental parameters based on these dynamic parameters and unified skeletal causal topology data, and sends them to the user's corresponding edge terminal device. After receiving the incremental parameters, the edge terminal device binds them with the covariance matrix and spherical harmonic function coefficients in the local 3D Gaussian meta-library. After adaptively adjusting the rendering detail level according to the local computing power, it performs rasterization rendering, presenting the digital human image with the detailed expression corresponding to the ironic semantics to the user in real time. The entire process does not require waiting for complete speech synthesis, and the end-to-end latency is controlled within 300 milliseconds.

[0065] In another concurrent scenario, when multiple users are having conversations with the same digital human character at the same time, the cloud-based inference service cluster calls the same personality weight matrix for each user's request, but maintains an independent historical conversation cache space for each user, so that the conversations of each user do not interfere with each other.

[0066] At the same time, the cloud only calculates and distributes personalized emotional response incremental parameters with a small amount of data for each user, instead of performing complete high-fidelity rendering calculations for each user separately. As a result, the rendering computing power consumption does not increase linearly with the number of concurrent users.

[0067] Those skilled in the art should understand that the above specific embodiments are only preferred embodiments of the present invention. At the implementation level of the semantic emotion dynamics causal reasoning module, a conditional generation method based on the diffusion model can also be adopted, using the hidden layer features of the large language model and the emotion label as conditional inputs, and generating the skeleton rotation parameters and facial hybrid deformation weight sequence through the denoising diffusion process, which can also achieve direct mapping from semantics to dynamic parameters.

[0068] At the cloud-edge collaborative rendering level, a lightweight variant of neural radiation field or a differentiable rendering scheme based on symbolic distance field can also be adopted. Combined with model distillation technology, the high-fidelity rendering model can be compressed into a lightweight version suitable for real-time operation on edge devices. This can also achieve the technical effect of decoupling computing power consumption and the number of concurrent users under the architecture of maintaining only unified topology data in the cloud and completing personalized rendering on the terminal side. The above alternative implementation methods have not departed from the technical concept of the present invention and should be regarded as equivalent replacement schemes within the scope of protection of the present invention.

[0069] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope defined by the claims of the present invention.

Claims

1. An AI algorithm-driven and interactive entertainment system for virtual digital humans, characterized in that, include: The semantic emotion dynamics causal reasoning module is used to extract the hidden layer semantic feature vector corresponding to the current word in real time during the process of decoding the word-by-word character of the large language model to generate the reply text. Based on the pre-trained and solidified three-layer directed causal graph, the semantic feature vector is mapped to the emotion state vector in sequence. Then, the emotion state vector and the semantic feature vector are jointly mapped to the skeleton rotation parameters and facial hybrid deformation weights to obtain the dynamic parameters used to drive the virtual digital human's movements. The personality constraint module is used to embed the low-rank adaptation matrix pre-trained for the digital human character into the attention layer of the large language model, correct the attention score matrix in the decoding process of the large language model, so that the corrected hidden layer features are constrained by the unified personality features, and pass the constrained hidden layer features to the semantic emotion dynamics causal reasoning module as the correction input for the micro-expression baseline state. The cloud-edge collaborative rendering module is used to maintain the unified skeletal causal topology data of the virtual digital human in the cloud. For each user session, it calculates the skeletal rotation offset and facial hybrid deformation weight offset relative to the unified skeletal causal topology data based on the dynamic parameters output by the semantic emotion dynamics causal inference module. The obtained offset is sent to the edge terminal device as a personalized emotional response increment parameter. The edge terminal device binds the personalized emotional response increment parameter with the covariance matrix and spherical harmonic function coefficients in the locally preset 3D Gaussian meta-database and performs rasterization rendering on the bound Gaussian meta-database.

2. The system according to claim 1, characterized in that, The three-layer directed causal graph includes a semantic node layer, an emotion node layer, and a dynamic node layer. Each node in the semantic node layer corresponds to the hidden layer semantic features of the large language model at a certain time when generating a word. Each node in the emotion node layer corresponds to the emotion state mapped by the semantic node layer. Each node in the dynamic node layer corresponds to the skeletal rotation parameters and the facial hybrid deformation weights. The semantic node layer and the emotion node layer, as well as the emotion node layer and the dynamic node layer, are connected by directed edges. Horizontal connecting edges are set between the nodes in the emotion node layer to represent the temporal continuity constraints of the emotion state.

3. The system according to claim 1, characterized in that, The semantic emotion dynamics causal reasoning module is configured to extract the hidden layer semantic feature vector corresponding to each word generated by the large language model and input it into the three-layer directed causal graph for reasoning, thereby obtaining dynamic parameters synchronized with the word, without waiting for the complete response text to be generated or for the text-to-speech module to synthesize speech.

4. The system according to claim 1, characterized in that, The personality constraint module includes a dimensionality reduction matrix and an dimensionality increase matrix pre-trained for the digital human character. The product of the dimensionality reduction matrix and the dimensionality increase matrix forms a low-rank increment matrix. The low-rank increment matrix is ​​added to the original weight matrix of the attention layer of the large language model to obtain a modified weight matrix. The modified weight matrix is ​​used to calculate the attention score matrix.

5. The system according to claim 4, characterized in that, The personality constraint module is configured to set the low-rank incremental matrix as a fixed parameter shared by multiple users in a multi-user concurrent request scenario, while allocating and maintaining an attention key-value cache space for each user's historical dialogue independently, and the attention key-value cache spaces of each user do not overlap during the calculation process.

6. The system according to claim 1, characterized in that, The unified skeletal causal topology data includes the skeletal tree structure and joint rotation constraint range of the virtual digital human. The unified skeletal causal topology data is shared by multiple user sessions and does not incur additional storage or computational overhead due to an increase in the number of concurrent users.

7. The system according to claim 1, characterized in that, The edge terminal device is configured to convert the bone rotation offset into a rotation transformation matrix, use the rotation transformation matrix to transform the covariance matrix of the Gaussian elements in the 3D Gaussian element library, and simultaneously use the color correction offset related to the emotional state in the personalized emotional response increment parameters to correct the spherical harmonic function coefficients of the Gaussian elements, thereby obtaining a set of Gaussian element parameters for rendering.

8. The system according to claim 1, characterized in that, The edge terminal device is pre-set with multiple levels of rendering detail and monitors the local rendering frame rate in real time. When the rendering frame rate is lower than a preset threshold, it switches to a lower level of rendering detail and when the rendering frame rate is higher than a preset threshold, it switches to a higher level of rendering detail. Rasterization rendering is performed on the adjusted Gaussian meta-parameter set to obtain a visual image of the virtual digital human.

9. An AI algorithm-driven and interactive entertainment method for virtual digital humans, characterized in that, include: During the process of decoding word by word in the large language model to generate the response text, the hidden layer semantic feature vector corresponding to the current word is extracted in real time. The semantic feature vector is input into a pre-trained and solidified three-layer directed causal graph to obtain the emotional state vector and the skeletal rotation parameters and facial hybrid deformation weights, which are used as dynamic parameters to drive the virtual digital human's movements. The low-rank adaptation matrix, which is pre-trained for the digital human character, is embedded into the attention layer of the large language model to correct the attention score matrix in the decoding process of the large language model. The corrected hidden layer features are then passed to the three-layer directed causal graph as the correction input for the micro-expression baseline state. The unified skeletal causal topology data of the virtual digital human is maintained in the cloud. For each user session, the skeletal rotation offset and facial hybrid deformation weight offset relative to the unified skeletal causal topology data are calculated. The obtained offsets are sent to the edge terminal device as personalized emotional response incremental parameters. The edge terminal device binds the personalized emotional response increment parameters with the covariance matrix and spherical harmonic function coefficients in the locally preset three-dimensional Gaussian meta-database, and performs rasterization rendering on the resulting Gaussian meta-database.

10. The method according to claim 9, characterized in that, In multi-user concurrent request scenarios, the low-rank adaptation matrix is ​​set as a fixed parameter shared by multiple users. At the same time, each user independently allocates and maintains its own attention key-value cache space for historical dialogues. The attention key-value cache spaces of each user do not overwrite each other during the calculation process.

Images