AI digital human real-time interaction method, system and equipment based on WEB browser

Abstract

The invention discloses an AI digital human real-time interaction method, system and device based on a WEB browser, and the method specifically comprises the steps: processing a large-scale AI digital human generation model of a cloud end through a model distillation and quantitative compression technology, generating and deploying a lightweight model to a user Web browser, and forming a local generation engine; based on a local generation engine, dynamically constructing and loading a personalized digital human model according to user historical interaction data safely synchronized from a server; in the interaction process, multi-modal input data of the user is collected, real-time analysis is conducted on the multi-modal input data, and the emotional state and interaction intention of the user are recognized; and based on the emotional state and the interaction intention, according to the personalized digital human model, generating a digital human expression, an action frame sequence and corresponding speech synthesis parameters matched with the personalized digital human model in real time. The AI digital human real-time interaction method based on the WEB browser realizes AI digital human real-time interaction based on the WEB browser, improves interaction convenience and real-time performance, and reduces dependence on cloud resources.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, device and medium for real-time interaction of AI digital humans based on a web browser. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI digital humans have demonstrated enormous potential and commercial value in numerous application scenarios such as virtual customer service, online education, and entertainment, attracting widespread attention and welcome. However, existing methods for implementing AI digital humans have many limitations, severely restricting their further promotion and application.

[0003] First, most current AI digital human technologies rely on specific platforms or software, such as specific operating systems, graphics rendering engines, or dedicated client software. This dependency makes it difficult for AI digital humans to run stably on different types and versions of devices and browsers. For example, some AI digital human applications developed based on the Windows platform cannot be used directly on Mac OS or Linux systems; some solutions that rely on specific graphics rendering engines cannot be displayed and interacted with properly in environments where that engine is not installed. This lack of cross-platform compatibility greatly limits the application scope of AI digital humans and fails to meet users' needs for using AI digital humans on different devices and in different environments.

[0004] Secondly, implementing existing AI digital human technology typically requires specialized graphics rendering engines and experienced developers. Specialized graphics rendering engines are often expensive and require developers to possess deep knowledge in graphics, computer vision, and other related fields, as well as proficiency in using the relevant engines. From model design and animation production to interactive logic development, the entire process involves multiple complex stages, requiring significant investment of human, material, and time resources. This not only makes it difficult for small businesses and individual developers to afford the costs of developing AI digital human applications but also limits the speed of innovation and development in AI digital human technology.

[0005] Furthermore, many existing AI digital human solutions require users to download additional plugins or client software to use them. This requirement increases the difficulty and steps involved, posing a significant hurdle for users unfamiliar with computer operations or concerned about the security of downloaded software. Users need to spend time and effort finding, downloading, installing, and configuring the relevant plugins or clients, and this process may need to be repeated on different devices. This greatly reduces users' willingness and enthusiasm to use AI digital humans, hindering the widespread adoption of AI digital human technology.

[0006] Finally, some existing AI digital human interaction methods typically employ a model of fully processing user input data and generating complete video or audio content before playback. This non-real-time interaction method leads to significant delays when handling complex tasks or large amounts of data, requiring users to wait a considerable amount of time to see the digital human's response, severely impacting the user experience. For example, in virtual customer service scenarios, if users have to wait several seconds or even longer for a response after asking a question, it will cause user impatience, reducing service quality and user satisfaction. Summary of the Invention

[0007] The purpose of this invention is to provide a method, system, device, and medium for real-time interaction of AI digital humans based on a web browser, which realizes real-time interaction of AI digital humans based on a web browser, improves the convenience and real-time performance of interaction, and reduces the dependence on cloud resources, thereby solving at least one of the aforementioned problems of the prior art.

[0008] In a first aspect, the present invention provides a real-time interactive method for AI digital humans based on a web browser, the method specifically comprising: The large-scale AI digital human generation model in the cloud is processed using model distillation and quantization compression technology to generate and deploy a lightweight model containing core facial expressions, lip movements and simple body movements to the user's web browser, forming a local generation engine. Based on a local generation engine, a personalized digital human model containing user preference appearance features, a voice style library, and behavioral response strategies is dynamically built and loaded according to user historical interaction data securely synchronized from the server. During the interaction, multimodal input data of users is collected, and an integrated lightweight multimodal emotion computing model is used to analyze the multimodal input data in real time to identify the user's emotional state and interaction intention. Based on emotional state and interaction intent, the system generates matching digital human expressions, action frame sequences, and corresponding speech synthesis parameters in real time according to the personalized digital human model. By automatically mining implicit behavioral patterns in user feedback using contrastive learning techniques, the response strategy and generation quality of personalized digital human models are optimized through generative adversarial networks, and the optimized personalized digital human models are incrementally updated to the browser.

[0009] Secondly, this invention provides a real-time interactive system for AI digital humans based on a web browser, the system specifically comprising: The model distillation module is used to process large-scale AI digital human generation models in the cloud using model distillation and quantization compression technology, generate and deploy lightweight models containing core facial expressions, lip movements and simple body movements to the user's web browser, forming a local generation engine. The local model module is used to dynamically build and load personalized digital human models containing user preference appearance features, voice style library and behavioral response strategies based on the local generation engine and user historical interaction data securely synchronized from the server. The interaction analysis module is used to collect multimodal input data from users during the interaction process, and to perform real-time analysis of the multimodal input data using an integrated lightweight multimodal emotion computing model to identify the user's emotional state and interaction intent. The parameter generation module is used to generate matching digital human facial expressions, action frame sequences, and corresponding speech synthesis parameters in real time based on the emotional state and interaction intent, according to the personalized digital human model. The optimization and update module is used to automatically mine implicit behavioral patterns in user feedback based on contrastive learning technology, optimize the response strategy and generation quality of personalized digital human models through generative adversarial networks, and incrementally update the optimized personalized digital human models to the browser.

[0010] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, and a computer program stored in the memory, wherein when the computer program is executed on the processor, it implements the real-time interactive method for AI digital humans based on a web browser as described in any of the above methods.

[0011] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the real-time interactive method for AI digital humans based on a web browser as described in any of the above methods.

[0012] Compared with the prior art, the present invention has at least one of the following technical effects: 1. This invention enables real-time interaction of AI digital humans based on a web browser, improving the convenience and real-time nature of the interaction and reducing the dependence on cloud resources.

[0013] 2. This invention deploys a lightweight model to the user's web browser to form a local generation engine. It does not rely on a specific platform or software and can run stably on different types and versions of devices and browsers, greatly expanding the application scope of AI digital humans and meeting the user's needs in different scenarios.

[0014] 3. This invention employs model distillation and quantization compression techniques to process large models, generating lightweight models. This reduces reliance on professional graphics rendering engines and lowers hardware and software costs during development. Simultaneously, it simplifies the development process, reduces the professional skills required of developers, and enables small businesses and individual developers to easily participate in the development of AI digital human applications, promoting technological innovation and development.

[0015] 4. In this invention, users do not need to download plugins or clients; they can interact with AI digital humans in real time directly in a web browser. The operation is simple and convenient, greatly reducing the difficulty of use for users and increasing their willingness and enthusiasm to use AI digital humans, which is conducive to the popularization and promotion of AI digital human technology.

[0016] 5. During the interaction process, this invention collects and analyzes the user's multimodal input data in real time, and generates matching digital human expressions, action frame sequences and speech synthesis parameters in real time based on the analysis results, so as to realize true real-time interaction, reduce user waiting time, and improve user experience. It has important application value, especially in scenarios with high real-time requirements such as virtual customer service and online education.

[0017] 6. This invention dynamically constructs personalized digital human models based on historical interaction data, which enhances the adaptability of digital humans to user preferences and improves the personalized experience.

[0018] 7. This invention utilizes a lightweight multimodal emotion computing model to analyze user input in real time, accurately identify emotions and intentions, and improve the relevance of interaction.

[0019] 8. This invention continuously improves the model's response strategy and generation quality by comparing and optimizing the personalized digital human model through comparative learning and generative adversarial networks and incrementally updating it.

[0020] 9. The present invention trains a pattern encoder that distinguishes between good and bad response strategies, providing an accurate basis for optimizing personalized digital human models and improving the optimization effect.

[0021] 10. This invention drives the optimization of personalized digital human models through adversarial training using generative adversarial networks, making the output approach a positive response mode and improving the quality of interaction. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a real-time interactive method for AI digital humans based on a web browser, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an AI digital human real-time interactive system based on a web browser, according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0025] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0028] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] In this application embodiment, the entity executing the process includes a terminal device. This terminal device includes, but is not limited to, devices capable of executing the methods disclosed in this application, such as servers, computers, smartphones, and tablets. Figure 1 A flowchart illustrating a real-time interactive method for AI digital humans based on a web browser, according to an embodiment of the present invention, is shown below in detail: S101 uses model distillation and quantization compression technology to process large-scale AI digital human generation models in the cloud, generating and deploying lightweight models containing core facial expressions, lip movements, and simple body movements to the user's web browser, forming a local generation engine.

[0031] In this embodiment, a large-scale AI digital human generation model is prepared in a cloud environment. This model, trained on a large amount of data, is capable of generating high-quality and complex digital human expressions, lip movements, and body movements. The core idea of ​​model distillation is to use the knowledge of this large model (the teacher model) to guide the training of a small model (the student model). In specific implementation, key feature representations and decision logic are extracted from the teacher model. For example, for digital human expression generation, the teacher model can accurately identify subtle changes in facial muscles under different emotions. By analyzing these patterns, feature vectors that summarize key information about expression generation are extracted. For lip movement generation, the teacher model can accurately generate corresponding lip shapes based on the input speech content, from which key mapping relationships for lip movement generation are extracted. In terms of body movement generation, the teacher model can generate natural and smooth body movements based on different scenarios and interaction requirements, from which core rules for body movement generation are extracted. These extracted key information are used as knowledge to train the student model. During training, the student model uses the output of the teacher model as a reference standard and continuously adjusts its own parameters so that when processing the same input data, the output of the student model is as close as possible to the output of the teacher model. In this way, the student model can learn the core capabilities of the teacher model, while having a simpler structure and fewer parameters.

[0032] After the student model is trained using distillation, its parameters are further quantized. Quantization refers to converting model parameters from high-precision floating-point representation to low-precision integer or fixed-point representation. For example, model parameters originally represented as 32-bit floating-point numbers can be converted to 8-bit integer representations after quantization. This significantly reduces the storage space and computational resources required by the model. During quantization, appropriate quantization methods and bit depths need to be selected. Different quantization strategies can be adopted for different parts of the model based on their varying accuracy requirements. For example, for parts of facial expression generation with relatively low detail requirements, lower bit depth quantization can be used to further compress the model size; while for key feature parameters in lip-sync generation, relatively higher bit depth quantization can be used to ensure accuracy. Through reasonable quantization, the student model is compressed into a lightweight model while minimizing the loss of model accuracy.

[0033] The lightweight model obtained after model distillation and quantization compression is packaged and format-converted to adapt to the user's web browser runtime environment. Considering the compatibility and performance limitations of web browsers, the model is converted to a universal, lightweight format, such as WebAssembly. WebAssembly is a low-level bytecode format that can run in modern web browsers, featuring high efficiency and security, ensuring fast loading and execution of the lightweight model in the browser. During the conversion process, the model is optimized to remove unnecessary redundant information, further improving the model's loading speed and running efficiency.

[0034] Finally, the converted lightweight model is deployed to the user's web browser. The model file is transferred to the user's browser via a web server. In the browser, JavaScript and other related technologies are used to load and initialize the lightweight model, making it a local generation engine. When a user visits a webpage containing this AI digital human application, the browser automatically downloads and loads this lightweight model, eliminating the need for the user to install any complex software or plugins. Once running in the browser, the local generation engine can generate the digital human's core facial expressions, lip movements, and simple body gestures in real time based on subsequent interaction commands, providing the user with a smooth interactive experience.

[0035] S102, based on a local generation engine, dynamically builds and loads a personalized digital human model containing user preference appearance features, a voice style library, and behavioral response strategies based on user historical interaction data securely synchronized from the server.

[0036] In this embodiment, the local generation engine initiates a data synchronization request to the server upon the user's first use of the AI ​​digital human service or at preset time intervals. Upon receiving the request, the server authenticates the request to ensure it originates from a legitimate user terminal. After successful authentication, the server retrieves the user's historical interaction data from the database. This historical interaction data covers various aspects of the user's past interactions with the AI ​​digital human, such as the user's preference for the digital human's appearance in different scenarios (e.g., the user repeatedly selecting digital human images with specific hairstyles, skin tones, and clothing styles); the user's feedback on voice style during communication with the digital human (e.g., the user prefers a gentle, lively, or serious tone); and the user's behavioral patterns in different interaction scenarios (e.g., whether the user prefers to ask questions directly or provide some background information beforehand). The server encrypts this retrieved data using a symmetric encryption algorithm and a pre-negotiated key with the user terminal, then transmits the encrypted data back to the user terminal's local generation engine. Upon receiving the encrypted data, the local generation engine decrypts it using the corresponding key, thereby securely obtaining the user's historical interaction data.

[0037] The local generation engine incorporates a data analysis module that meticulously analyzes decrypted user history interaction data. Regarding user preference appearance features, it extracts the most frequently chosen appearance elements such as hairstyle, facial features, body proportions, and clothing styles by statistically analyzing the frequency and preference of users selecting digital human appearances in different interaction scenarios. For example, if a user consistently chooses a digital human image with fashionable casual wear and a short hairstyle in multiple virtual shopping scenarios, these features will be extracted as the user's preferred appearance features. In constructing the voice style library, it analyzes user feedback on voice during interactions with the digital human, such as preferences for speech rate, tone, and timbre. If a user consistently shows positive feedback towards voices with moderate speed, gentle tone, and sweet timbre in multiple interactions, these voice features will be included in the user's voice style library. For behavioral response strategies, it summarizes user behavioral habits and corresponding expected response strategies by analyzing user behavior patterns and expected digital human responses in different interaction scenarios. For example, when a user asks a relatively complex question, the user may expect the digital human to first give a brief summary of the question and then provide a detailed answer. This behavioral response pattern will be recorded.

[0038] The local generation engine constructs a personalized digital human model based on extracted user-preferred appearance features, a voice style library, and behavioral response strategies. For appearance construction, it utilizes a pre-stored basic digital human model framework within the local generation engine, applying the extracted user-preferred appearance features to this framework. For example, user-preferred hairstyles and facial features are accurately rendered onto the basic model using graphics rendering technology, generating a digital human image with the user's unique appearance style. Regarding the application of the voice style library, the local generation engine calls its built-in speech synthesis module, adjusting the speech rate, tone, and timbre based on the feature parameters in the library to ensure the generated speech matches the user's preferences. For behavioral response strategies, the summarized user behavior patterns and expected response strategies are compiled into rule logic and embedded into the interaction logic of the personalized digital human model. For example, when a complex question is detected from a user, the expected response strategy is followed by a summary of the question before a detailed answer.

[0039] After the local generation engine completes the construction of the personalized digital human model, it loads it into the runtime environment. During the loading process, the model's performance is optimized to ensure smooth operation in the user's web browser. For example, the model's graphics rendering is optimized to reduce unnecessary rendering calculations and improve rendering efficiency; the speech synthesis part is cached, pre-synthesizing some commonly used speech segments to reduce the computational load of real-time synthesis. Once loaded, users can interact with the personalized AI digital human in real time in their web browser, enjoying an interactive experience tailored to their preferences and habits.

[0040] S103 collects multimodal input data from users during the interaction process, and uses an integrated lightweight multimodal emotion computing model to analyze the multimodal input data in real time to identify the user's emotional state and interaction intention.

[0041] In this embodiment, when a user interacts with a web-based AI digital human, the system simultaneously activates multiple data acquisition channels to obtain the user's multimodal input data. For the visual modality, the system uses the camera integrated into the user's device to capture visual information such as facial expressions, eye movements, and body gestures in real time. For example, the camera continuously captures images of the user's face, recording subtle changes in different expressions such as smiling, frowning, and surprise, while also capturing the user's eye focus direction and movement trajectory, as well as body movements such as head rotation and hand gestures. For the auditory modality, the system uses the microphone on the device to collect the user's voice information, including the content, tone, speed, and volume of the user's speech. For example, the microphone accurately records the user's voice when asking a question, and analyzes whether the user is speaking in an urgent, calm, or low tone, and whether the speaking speed is fast or slow. In addition, for the text modality, if the user inputs text information in the interactive interface, the system will acquire this text content in real time, such as questions or feedback entered by the user in the chat box.

[0042] The collected multimodal input data often contains noise and interference, requiring preprocessing to improve data quality. The preprocessed multimodal data is then input into an integrated, lightweight multimodal emotion computing model for analysis. This model employs a multimodal fusion architecture, capable of simultaneously processing visual, auditory, and textual data. In visual modality analysis, the model's visual feature extraction module extracts features from the preprocessed images, using algorithms such as convolutional neural networks to identify facial expression features, such as eyebrow shape and mouth opening / closing, and determines the user's emotional state based on these features, for example, whether the user is happy, angry, or sad. Simultaneously, it analyzes the user's body language features, such as gesture shape and body posture, further assisting in determining the user's emotions and interaction intentions. In auditory modality analysis, the auditory feature extraction module extracts features from speech segments, analyzing characteristics such as intonation, speech rate, and volume, and combining these with the speech content to identify the user's emotional state and interaction intentions. For example, a high speech rate and enthusiastic tone may indicate that the user is excited or eager. In terms of text modality analysis, the text semantic analysis module performs semantic understanding on the segmented text, analyzes the meaning of the text through natural language processing algorithms, judges the user's questions, feedback, etc., and thus identifies the user's interaction intent.

[0043] The lightweight multimodal emotion computing model analyzes visual, auditory, and textual data separately, then fuses the results from all three modalities. A weighted fusion approach is used, assigning weights to each modal's analysis result based on its importance in emotion recognition and intent assessment. The weighted results are then combined to arrive at the final user emotional state and interaction intent recognition result. For example, when determining user satisfaction with the current service, a smiling expression in the visual modality, a calm tone of voice in the auditory modality, and positive feedback in the text modality may all be given higher weights, leading to a conclusion of user satisfaction. Finally, the system outputs the identified user emotional state and interaction intent results for use by the subsequent digital human response generation module, enabling the digital human to respond with corresponding facial expressions, actions, and voice based on the user's emotions and intentions.

[0044] S104, based on emotional state and interaction intent, generates matching digital human expressions, action frame sequences, and corresponding speech synthesis parameters in real time according to a personalized digital human model.

[0045] In this embodiment, during the interaction, the lightweight multimodal emotion computing model integrated into the system identifies the user's emotional state and interaction intent, and then transmits this key information to the digital human response generation module in a structured data format. Upon receiving the emotional state and interaction intent data, the response generation module immediately invokes a pre-built personalized digital human model. This personalized digital human model is dynamically constructed and loaded based on a local generation engine, combined with user historical interaction data securely synchronized from the server. It includes user-preferred appearance features such as facial contours, hairstyle, and skin tone; a voice style library covering features such as tone, speech rate, and timbre; and behavioral response strategies that define the digital human's behavioral patterns in different interaction scenarios. By invoking this personalized model, it is ensured that the generated digital human image and behavior better meet the user's personalized needs.

[0046] Based on the received emotional state information, the response generation module generates matching digital human expressions in real time, using the pre-set expression generation rules and algorithms in the model, based on the appearance features of the personalized digital human model.

[0047] Combining emotional state and interaction intent, the response generation module further generates a sequence of motion frames for the digital human. The digital human's actions will differ depending on the emotion and interaction scenario. For example, when a user inquires about product information and is relatively calm, the digital human might nod, smile, or gently point to relevant materials to indicate agreement and provide guidance; while when a user expresses dissatisfaction, the digital human might first bow slightly to show apology, then spread its hands to explain and reassure. The motion frame sequence is generated in real-time using animation generation algorithms after analyzing a large amount of real human motion data in similar emotional and interaction scenarios and summarizing the patterns. Each motion frame contains information such as the position, posture, and direction of movement of various parts of the digital human's body, ensuring the continuity and naturalness of the movements.

[0048] Based on the emotional state and interaction intent, the response generation module also generates corresponding speech synthesis parameters. These parameters include tone, speech rate, volume, and timbre. For example, when a user is happy, the digital human's tone will be relatively light and rising, with a moderate speech rate and volume, giving a friendly and pleasant feeling; when a user is angry, the tone will become deep and stern, the speech rate may be slightly faster, and the volume will increase accordingly to reflect a serious and forceful attitude. The timbre selection is determined based on the personalized digital human model's speech style library to ensure that the voice matches the overall image of the digital human. These speech synthesis parameters are then transmitted to the speech synthesis engine to generate the final speech content.

[0049] The generated digital human facial expressions, action frame sequences, and speech synthesis parameters are integrated to form a complete digital human response. The system precisely matches facial expressions, actions, and speech according to a specific time sequence and synchronization mechanism, ensuring that the digital human can simultaneously emit corresponding speech while displaying facial expressions and actions. Finally, this complete response is displayed in real-time through the user's web browser, allowing the user to see and hear the digital human's real-time responses based on its emotional state and interaction intentions.

[0050] S105 automatically mines the implicit behavioral patterns in user feedback using contrastive learning technology, optimizes the response strategy and generation quality of the personalized digital human model through generative adversarial networks, and incrementally updates the optimized personalized digital human model to the browser.

[0051] In this embodiment, during real-time interaction between the user and the web-based AI digital human, the system collects user feedback data. This feedback data takes various forms, including direct evaluations of the digital human's responses, such as "satisfied" or "unsatisfied" feedback provided by the user via interface buttons; supplementary information entered by the user during the interaction, such as further details of the problem provided by the user after being dissatisfied with the initial response in a virtual customer service scenario; and user behavioral data, such as the user's actions after hearing the digital human's reply, whether to continue asking questions or end the interaction. The system categorizes and stores this feedback data according to time sequence and interaction scenario for subsequent analysis and processing.

[0052] From the collected user feedback data, data pairs with comparative value are selected. For example, different responses from the same user to similar types of digital humans in different interaction scenarios, or different evaluations from different users of the same response to a digital human. These data pairs are then organized, and high-quality feedback data recognized by users and low-quality feedback data not recognized are identified. Simultaneously, to enhance the effectiveness of comparative learning, some pre-set high-quality interaction case data are introduced as reference. These case data have been evaluated and screened by professionals and meet good interaction logic and user experience standards. The high-quality feedback data, low-quality feedback data, and pre-set high-quality interaction case data together constitute the comparative learning dataset.

[0053] The prepared dataset is analyzed using contrastive learning techniques. The contrastive learning model learns common features and patterns from high-quality feedback data and pre-set high-quality interaction case data. These features and patterns represent the expected digital human response behavior of users, such as accurate answers, appropriate tone of voice, and natural gestures and expressions. Simultaneously, the model also identifies features in low-quality feedback data that differ significantly from high-quality data. These differences reflect unsatisfactory digital human response behavior. Through this contrastive learning, the model can automatically uncover implicit behavioral patterns in user feedback, clarifying the response strategies and characteristics of the generated content that the digital human should adopt in different interaction scenarios.

[0054] The mined user behavior patterns are used as guiding information and input into Generative Adversarial Networks (GANs). A GAN consists of a generator and a discriminator. The generator is responsible for generating new personalized digital human model response strategies and content based on user behavior patterns, such as generating new facial expressions, action frame sequences, and speech synthesis parameters. The discriminator evaluates the content generated by the generator, judging whether it conforms to the user's expected behavior patterns and high-quality interaction standards. During training, the generator and discriminator compete against each other and continuously optimize. The generator strives to generate content that better meets user needs to fool the discriminator; the discriminator continuously improves its judgment ability to accurately distinguish between high-quality and low-quality content. After multiple rounds of training, GANs can generate high-quality personalized digital human model response strategies and content, thereby optimizing the performance of personalized digital human models.

[0055] After the generative adversarial network (GAN) optimizes the personalized digital human model, it needs to be incrementally updated and deployed to the user's web browser. Incremental updates involve transmitting only the changed parts of the model, rather than the entire model, thus reducing data transmission volume and improving update efficiency. The system first analyzes the optimized model to determine which parts have changed, then encodes and compresses these changed parts. Next, these incremental data are transmitted to the user's web browser via a secure network channel. Upon receiving the incremental data, the browser integrates it into the existing personalized digital human model according to preset decoding and update rules, achieving real-time model updates. The updated personalized digital human model takes effect immediately in subsequent interactions, providing users with a better interactive experience.

[0056] In some embodiments, in step S101 above, the process of using model distillation and quantization compression technology to process the large-scale AI digital human generation model in the cloud, generating and deploying a lightweight model containing core facial expressions, lip movements, and simple body movements to the user's web browser to form a local generation engine specifically includes: In the cloud, based on model distillation technology, a large-scale AI digital human generation model is used as a teacher model to train student models. The student models are used to inherit the core facial expressions, lip movements and body movements of the large-scale AI digital human generation model. The student model is quantized and compressed to generate a lightweight inference model file; On the browser side, the inference model file is compiled into a WebAssembly module and loaded, and a local generation engine is built in the web browser environment; When an interaction is triggered, the JavaScript logic of the web browser calls the local generation engine, and the digital human's sequence image data and associated voice parameters are generated in real time by passing in the driving parameters.

[0057] In this embodiment, a large-scale AI digital human generation model is first selected as the teacher model in the cloud. This teacher model, trained on a large amount of data, is capable of generating rich and realistic digital human expressions, lip movements, and body movements. Next, student models are trained using model distillation technology. The core idea of ​​model distillation is to allow smaller, simpler student models to learn the knowledge and abilities of the teacher model. During training, the data processed by the teacher model and its outputs are used as supervisory signals to guide the student models' learning. Specifically, a dataset covering various interaction scenarios, different emotional states, and action types is prepared. This data is input into the teacher model to obtain the relevant features and output results of the core expressions, lip movements, and body movements generated by the teacher model. Then, these results are used as targets, and the student models are trained on the same data. The parameters of the student models are continuously adjusted so that they can imitate the behavior of the teacher model as closely as possible, inheriting the performance of the large-scale AI digital human generation model in these key capabilities. After multiple rounds of training and optimization, the student models gradually acquire core capabilities similar to the teacher model, enabling them to generate compliant expressions, lip movements, and simple body movements.

[0058] After training the student model, it needs to be quantized and compressed to run efficiently in a resource-constrained web browser environment. Quantization primarily converts the model's parameters from high-precision floating-point numbers to low-precision integers or fixed-point numbers. During quantization, careful selection of appropriate quantization strategies and parameters is crucial to ensure the quantized model maintains good performance with minimal loss of precision. Compression employs common model compression algorithms such as pruning and knowledge distillation. Pruning removes less important neurons or connections, reducing model complexity; knowledge distillation further utilizes information from the teacher model to perform more refined compression optimization of the student model. These quantization and compression operations generate a lightweight inference model file, significantly reducing its size and making it more suitable for transmission and deployment in a web environment.

[0059] On the browser side, the lightweight inference model file generated in the cloud needs to be compiled into a WebAssembly module. WebAssembly is a binary instruction format that can run in modern web browsers, offering advantages such as efficiency, security, and cross-platform compatibility. The compilation process uses specialized tools to convert the inference model file into a WebAssembly module file. This conversion process generates and optimizes code based on the model's structure and parameters, ensuring the generated WebAssembly module runs correctly in the browser. After compilation, the WebAssembly module is loaded into the webpage using the browser's JavaScript code. During loading, the browser parses the WebAssembly module file, loads it into memory, and initializes and configures it, preparing for subsequent model execution. In this way, a local generation engine is successfully built in the web browser environment, capable of running lightweight inference models.

[0060] When a user triggers an interaction with the AI ​​digital human on a web page, such as by inputting text or voice information, the web browser's JavaScript logic responds immediately. The JavaScript code generates corresponding driving parameters based on the user's input. These parameters contain key information needed to generate the digital human's facial expressions, lip movements, and body language, such as emotional state, interaction intent, and action type. The JavaScript logic then calls the local generation engine, passing the driving parameters to the engine. Upon receiving the parameters, the local generation engine performs real-time inference using a loaded lightweight inference model. During inference, the model generates sequential image data and associated speech parameters for the digital human based on the input driving parameters and its learned knowledge and abilities. The sequential image data describes the changes in the digital human's facial expressions and body language at different points in time, while the associated speech parameters contain information such as pitch, speech rate, and timbre needed to generate speech. This generated image data and speech parameters can then be further rendered and played in the browser, enabling real-time interaction between the digital human and the user, providing a smooth and natural interactive experience.

[0061] In some embodiments, step S102 above, which involves dynamically constructing and loading a personalized digital human model containing user preference appearance features, a voice style library, and behavioral response strategies based on a local generation engine and according to user historical interaction data securely synchronized from the server, specifically includes: In the cloud, analyze the historical interaction data of a specified user in the server, extract and generate a set of personalized configuration parameters that includes appearance feature vectors, voice style parameters and behavioral response strategy weights; Based on a secure communication link, the personalized configuration parameter set is synchronized from the server to the web browser on the browser side; Within the web browser, the deployed local generation engine receives and parses the synchronized set of personalized configuration parameters to obtain the parsing results. Based on the analysis results, a style generation module is constructed according to the appearance feature vector, a speech synthesis pipeline is constructed according to the speech style parameters, and a behavior decision logic module is constructed according to the behavior response strategy weights, thus forming a personalized digital human model.

[0062] In this embodiment, the first step is to collect historical interaction data of a specified user from the server. This data includes the user's past dialogue records with the digital human, records of the user's adjustments to the digital human's appearance, user feedback on different voice styles, and user behavior choices in different interaction scenarios. After the data is collected, it is preprocessed to remove noisy data and invalid information, ensuring the accuracy and completeness of the data.

[0063] Data analysis algorithms are used to conduct in-depth analysis of the preprocessed data. Regarding appearance features, image recognition and feature extraction techniques are employed to extract key features such as facial contours, hairstyle, skin tone, and clothing style from the user-adjusted digital human appearance images, and these features are then transformed into appearance feature vectors. For example, if a user repeatedly selects an appearance with rounded facial contours, long brown hair, and warm-toned clothing, then the dimensions corresponding to these features will have corresponding numerical representations in the appearance feature vector.

[0064] In extracting speech style parameters, we analyze user feedback on different speech patterns, such as preferences for speech rate, intonation, and timbre. Through speech processing algorithms, we quantify these preferred speech features into speech style parameters. For example, if a user prefers a moderate speech rate, a soft intonation, and a sweet timbre, then the corresponding speech rate, intonation, and timbre parameters will be set to their appropriate values.

[0065] The weights for behavioral response strategies are determined based on user behavior choices in various interaction scenarios. For example, in a question-and-answer scenario, if a user prefers a detailed and organized answer from the avatar, the weight of a detailed answer will be increased in the behavioral response strategy; conversely, if a user prefers a concise and direct answer, the weight of a concise answer will increase. Through comprehensive analysis, a personalized configuration parameter set is generated, including appearance feature vectors, voice style parameters, and behavioral response strategy weights.

[0066] To ensure the security of personalized configuration parameter sets during transmission, synchronization is performed via a secure communication link. Encryption techniques are employed to encrypt the personalized configuration parameter sets, such as using symmetric or asymmetric encryption algorithms, converting the parameter sets into ciphertext form. Simultaneously, a secure communication protocol, such as HTTPS, is established. This protocol encrypts and authenticates data at the transport layer, preventing data theft or tampering during transmission.

[0067] The cloud sends the encrypted set of personalized configuration parameters to the web browser on the browser side via a secure communication link. Upon receiving the data, the web browser first verifies the integrity and legitimacy of the data's origin. It ensures that the received data originates from the server and has not been tampered with by checking the data's digital signature or using a pre-agreed authentication mechanism. Then, it uses the corresponding decryption key to decrypt the encrypted data, restoring the original set of personalized configuration parameters.

[0068] Within the web browser, a deployed local generation engine receives and parses the synchronized set of personalized configuration parameters. This local generation engine has a dedicated parameter parsing module that can identify different types of parameters in the set and parse them according to predetermined rules. For example, for appearance feature vectors, the parsing module breaks them down into feature values ​​for each dimension; for speech style parameters, it clarifies the speech features represented by each parameter; and for behavior response strategy weights, it determines the weight values ​​corresponding to different behavior strategies.

[0069] Based on the analysis results, we began to build the various modules of the personalized digital human model.

[0070] A style generation module is constructed based on appearance feature vectors. This module uses predefined style generation rules and models to generate digital human appearance styles that match user preferences, based on the values ​​of each dimension in the feature vectors. For example, it selects a suitable facial model based on facial contour feature values ​​and adds corresponding hairstyle styles based on hairstyle feature values.

[0071] A speech synthesis pipeline is constructed based on speech style parameters. This pipeline includes multiple processing steps, such as text analysis, speech feature mapping, and speech synthesis. Speech style parameters are incorporated into these steps to ensure the synthesized speech possesses user-preferred characteristics such as speech rate, intonation, and timbre. For example, in the speech feature mapping step, an appropriate timbre model is selected for speech synthesis based on timbre parameters.

[0072] A behavior decision-making logic module is constructed based on the weights of behavior response strategies. This module determines the digital human's behavior response method according to different interaction scenarios and user input, combined with the weights of behavior response strategies. For example, in a question-and-answer scenario, the module selects the answer method that better matches the user's preference based on the weights of detailed and concise answers.

[0073] By constructing these three modules, a complete personalized digital human model is formed.

[0074] In some embodiments, step S103 above, which involves collecting the user's multimodal input data during the interaction process and using an integrated lightweight multimodal emotion computing model to perform real-time analysis of the multimodal input data to identify the user's emotional state and interaction intent, specifically includes: In a web browser environment, text streams, audio streams, and video streams generated by user interactions are collected in parallel as multimodal input data; Multimodal input data is processed by the corresponding localized lightweight feature extraction module to generate text semantic feature vectors, speech acoustic feature vectors, and visual expression feature vectors. Text semantic feature vectors, speech acoustic feature vectors, and visual expression feature vectors are simultaneously input into a locally deployed lightweight multimodal fusion decision model. The lightweight multimodal fusion decision model performs fusion calculations on the input multimodal feature vectors to identify the user's real-time emotional state classification and instant interaction intent.

[0075] In this embodiment, a parallel acquisition method is adopted in the web browser environment to achieve comprehensive and accurate collection of multimodal input data generated by user interactions. For text streams, the browser's built-in text input monitoring mechanism captures text information entered by the user in areas such as text input boxes and chat windows in real time. Whether the text is manually entered by the user or added through copy and paste, it can be acquired in a timely manner and formed into a continuous text stream.

[0076] Audio stream acquisition utilizes browser-supported audio capture APIs, such as the Web Audio API. Once the user has microphone access enabled, this API can acquire the audio signal generated by the user's speech in real time and convert it into a digital audio stream. During the acquisition process, the audio signal undergoes preliminary preprocessing, such as noise reduction, to minimize the interference of ambient noise on audio quality and ensure that the acquired audio stream is clearly identifiable.

[0077] Video stream acquisition also relies on browser functionality, utilizing the getUserMedia API to access the user's camera device. After user authorization, the camera begins working, capturing the user's facial image and video footage in real time, forming a video stream. The acquired video stream undergoes appropriate resolution and frame rate adjustments to suit subsequent processing needs while ensuring smooth acquisition. Through this parallel acquisition method, text, audio, and video streams generated by user interactions are simultaneously acquired as multimodal input data.

[0078] The collected multimodal input data are processed by the corresponding localized lightweight feature extraction modules.

[0079] For text streams, the localized lightweight text feature extraction module employs word embedding methods from natural language processing, mapping each word in the text to a fixed-dimensional vector. These vectors are combined to form the text semantic feature vector. This module is optimized to minimize computation and model size while ensuring feature extraction accuracy, thus adapting to the computing capabilities of local devices.

[0080] For audio streams, the localized lightweight speech feature extraction module employs speech signal processing techniques to extract acoustic features of the speech. For example, it transforms the audio signal from the time domain to the frequency domain using a short-time Fourier transform to obtain spectral features; simultaneously, it extracts representative speech feature parameters such as Mel-frequency cepstral coefficients (MFCCs) and combines these parameters into a speech acoustic feature vector. This module uses simplified algorithms and model structures during feature extraction to reduce computational complexity.

[0081] For video streams, the localized lightweight visual feature extraction module utilizes computer vision technology to analyze user facial expressions in video frames. It locates key facial features such as eyes, mouth, and eyebrows using facial keypoint detection algorithms, and then extracts visual expression feature vectors based on the position and movement changes of these keypoints. This module also emphasizes lightweight design, employing an efficient convolutional neural network structure to reduce the number of model parameters and improve feature extraction speed. After processing, it generates text semantic feature vectors, speech acoustic feature vectors, and visual expression feature vectors.

[0082] The generated text semantic feature vectors, speech acoustic feature vectors, and visual expression feature vectors are simultaneously input into a locally deployed lightweight multimodal fusion decision model. This model employs multimodal fusion technology to organically combine feature vectors from different modalities. During the fusion process, each feature vector is first normalized to ensure they have the same scale and range, facilitating subsequent calculation and fusion. Then, the lightweight multimodal fusion decision model uses a weighted fusion approach, assigning corresponding weights to each feature vector based on the importance of different modalities in emotion recognition and intent judgment. For example, when recognizing a user's emotional state, visual expression feature vectors may have higher weights because facial expressions often more intuitively reflect emotions; while when judging interaction intent, text semantic feature vectors may be more important because text content can more accurately express intent.

[0083] Through fusion computation, the model integrates feature information from different modalities to form a comprehensive feature representation. Next, a pre-trained classifier within the model is used to classify this comprehensive feature representation, identifying the user's real-time emotional state, such as happiness, sadness, anger, or calmness. Simultaneously, an intent recognition algorithm determines the user's immediate interaction intent, such as asking for information, expressing needs, or making suggestions. Ultimately, this process achieves accurate identification of the user's emotional state and interaction intent.

[0084] In some embodiments, step S104 above, which involves generating a matching digital human expression, action frame sequence, and corresponding speech synthesis parameters in real time based on the emotional state and interaction intention according to the personalized digital human model, specifically includes: Based on emotional state and interaction intent, combined with user preference parameters in the loaded personalized digital human model and the text content to be replied to, a comprehensive conditional encoding vector is obtained. The conditional encoding vector is input into a locally deployed lightweight conditional generation model for single inference, simultaneously generating a temporally aligned sequence of digital human visual action parameters and a sequence of speech synthesis parameters. The visual action parameter sequence is used to define the digital human's facial expressions and body movements, and the speech synthesis parameter sequence is used to define the acoustic features of the synthesized speech. The visual motion parameter sequence is fed to the web browser's graphics rendering interface to drive the real-time animation of the digital human figure, and the speech synthesis parameter sequence is fed to the web browser's audio synthesis interface to generate the corresponding speech waveform.

[0085] In this embodiment, after recognizing the user's emotional state and interaction intent, the system begins to obtain a comprehensive conditional encoding vector based on this information and relevant parameters from the loaded personalized digital human model. First, user preference parameters are extracted from the personalized digital human model. These parameters are collected and recorded during long-term interactions between the user and the digital human, covering the user's preferences for the digital human's appearance, interaction style, voice characteristics, and other aspects. For example, a user might prefer a digital human with a gentle appearance, a tendency to smile, soft movements, and a calm, moderate speaking pace.

[0086] Simultaneously, the system acquires the text content to be replied to, which is the response information generated based on the user's interaction intent. It integrates and processes four types of information: emotional state, interaction intent, user preference parameters, and the text content to be replied to. Using information fusion, different types of information are mapped to a unified feature space. For emotional states, they are converted into numerical representations, such as a higher value for happiness and a lower value for sadness; interaction intents undergo similar encoding processing, with different encoded values ​​assigned to inquiries and expressions of needs. Key feature vectors are extracted from user preference parameters and the text content to be replied to. Finally, these processed information are concatenated and weighted to generate a comprehensive conditional encoding vector, which contains the key information needed to generate the digital human's expressions, movements, and speech.

[0087] The resulting comprehensive conditional encoding vector is input into a locally deployed lightweight conditional generation model for a single inference pass. This lightweight conditional generation model is specially designed and trained to reduce computational resource and time consumption while maintaining generation quality. During model inference, the model's internal multi-layer neural network structure performs complex nonlinear transformations and feature extraction on the conditional encoding vector.

[0088] The lightweight conditional generation model consists of two parallel branches: one responsible for generating the visual-motor parameter sequence for the digital human, and the other for generating the speech synthesis parameter sequence. In the visual-motor parameter sequence generation branch, the model generates a series of temporally aligned parameters based on the emotional state, interaction intent, and user preferences for the digital human's expressions and movements from the conditional encoding vector. These parameters define in detail the digital human's facial expressions, such as the degree of eye opening and closing, the curvature of the eyebrows, and the shape of the mouth, as well as body movements, such as the amplitude of arm swings and the angle of body tilt. In the speech synthesis parameter sequence generation branch, the model generates a parameter sequence defining the acoustic features of the synthesized speech, including pitch, duration, volume, and timbre, based on the emotional state, interaction intent, and user voice preferences from the conditional encoding vector. This ensures that the generated speech matches the digital human's expressions and movements and meets the user's expectations.

[0089] The generated sequence of visual motion parameters is sent to the web browser's graphics rendering interface. The web browser's graphics rendering interface has powerful graphics processing capabilities, enabling it to adjust the state of various parts of the digital human avatar in real time based on the received sequence of visual motion parameters. For example, it can change the display effect of the digital human's eyes based on the eye opening / closing parameter; and control the animation movement of the digital human's arms based on the arm swing amplitude parameter. In this way, the digital human avatar is driven to perform real-time animation, allowing the digital human to interact with the user in a vivid and natural manner.

[0090] Simultaneously, the speech synthesis parameter sequence is sent to the audio synthesis interface of the web browser. The audio synthesis interface uses these parameters and a built-in speech synthesis algorithm to generate the corresponding speech waveform. This algorithm precisely controls the frequency, duration, and intensity of the speech based on parameters such as pitch, duration, and volume, thereby synthesizing speech that meets the requirements. The generated speech waveform can be played directly in the browser, allowing users to hear the digital human's clear and accurate responses, achieving comprehensive visual and auditory interaction between the digital human and the user.

[0091] In some embodiments, step S105 above, which involves automatically mining implicit behavioral patterns in user feedback using contrastive learning techniques, optimizing the response strategy and generation quality of the personalized digital human model through a generative adversarial network, and incrementally updating the optimized personalized digital human model to the browser, specifically includes: Based on the desensitized implicit feedback sequences collected during local interaction with a web browser, an unlabeled interaction dataset containing labels of subsequent user behaviors is constructed in the cloud. Based on an unlabeled interactive dataset, implicit positive and negative response patterns are automatically mined using unsupervised contrastive learning techniques, and a pattern encoder is trained to distinguish the superiority or inferiority of response strategies. Based on the optimization signal provided by the pattern encoder, the discriminator and generator in the generative adversarial network deployed on the server are subjected to an adversarial training process, which drives the personalized digital human model as the generator to optimize the response strategy and generation quality, so that the output of the personalized digital human model approaches the positive response mode. The parameters of the optimized personalized digital human model are compared differentially with the original model parameters on the browser side to generate an incremental update patch, which is then synchronized to the web browser environment on the browser side.

[0092] In this embodiment, during local interaction based on a web browser, the system continuously collects anonymized implicit feedback sequences from the user. These feedback sequences cover various subtle behavioral information of the user during interaction with the digital human, such as the duration of the user's pause after hearing the digital human's response, subsequent actions (such as asking further questions, ending the conversation, etc.), mouse movement trajectory, and click locations. Because this feedback information has been anonymized, it does not involve the user's sensitive personal privacy data, ensuring the security and compliance of user data.

[0093] After collecting the anonymized implicit feedback sequences, they were organized and labeled in the cloud. The labels are user follow-up behavior tags, which indicate the type of subsequent action taken by the user after receiving a response from the digital human. For example, if the user continues to ask new questions after hearing a response, it can be labeled "continued interaction"; if the user ends the conversation directly, it can be labeled "end of conversation"; if the user does not take any further action within a period of time, it can be labeled "short stay" or "long stay" based on the duration of the stay, etc. In this way, an unlabeled interaction dataset containing user follow-up behavior tags is constructed, which provides a rich data foundation for subsequent pattern mining.

[0094] Using a pre-constructed unlabeled interactive dataset, unsupervised contrastive learning is employed to automatically uncover implicit positive and negative response patterns. The core idea of ​​unsupervised contrastive learning is to allow the model to learn the similarities and differences between data points, thereby discovering potential patterns. Specifically, data samples from the unlabeled interactive dataset are randomly sampled and combined to form positive and negative sample pairs. Positive sample pairs typically refer to data sample pairs with similar positive or negative response patterns, while negative sample pairs refer to data sample pairs with significantly different response patterns. These sample pairs are then input into a pattern encoder for training. The pattern encoder is a deep neural network model that automatically extracts key features from data samples and encodes the samples based on these features. During training, the pattern encoder continuously adjusts its parameters to ensure that positive sample pairs are as close as possible to each other in the encoding space, while negative sample pairs are as far apart as possible. In this way, the pattern encoder learns the feature representations that distinguish between positive and negative response patterns, thus training a pattern encoder capable of accurately differentiating the merits of response strategies. This pattern encoder will provide important optimization signals for subsequent optimization of personalized digital human models.

[0095] Based on the optimized signals provided by the trained pattern encoder, a generative adversarial network (GAN) is deployed on the server. Through the adversarial training process between the discriminator and the generator, the personalized digital human model, which acts as the generator, is driven to optimize its response strategy and generation quality.

[0096] Generative Adversarial Networks (GANs) consist of two parts: a generator and a discriminator. The generator, or personalized digital human model, is tasked with generating responses from the digital human, including text replies, emoticons, and actions. The discriminator is a binary classification model that determines whether the input response comes from a genuine positive response pattern or is generated by the generator.

[0097] During training, the generator continuously generates responses from the digital human and inputs them into the discriminator. The discriminator judges the generated responses based on the feature information of positive and negative response patterns provided by the pattern encoder, and provides a judgment result and corresponding feedback signal. If the discriminator considers the generated responses to be close to a positive response pattern, it provides positive feedback to the generator, encouraging it to continue generating in that direction; if the discriminator considers the generated responses to be close to a negative response pattern, it provides negative feedback to the generator, prompting it to adjust its parameters and improve its generation strategy.

[0098] The generator continuously optimizes its response strategy and generation quality based on feedback signals from the discriminator, gradually bringing the generated digital human's responses closer to a positive response pattern. Through this adversarial training process, the personalized digital human model can continuously improve its ability to interact with users and provide responses that better meet user expectations.

[0099] Once the personalized digital human model is optimized on the server side, the optimized model parameters need to be synchronized to the browser to achieve real-time model updates and improve user experience. To reduce data transfer volume and update time, an incremental update approach is adopted. First, the parameters of the optimized personalized digital human model are compared differentially with the original model parameters on the browser side. By comparing the differences between the two model parameters, the parameters that need updating are identified, and an incremental update patch is generated. This incremental update patch only includes the changed parts of the model parameters, not all parameters of the entire model, thus significantly reducing data transfer volume. Then, the generated incremental update patch is synchronized to the web browser environment on the browser side. On the browser side, there is a dedicated model loading and updating module that can receive the incremental update patch and update the original model parameters according to the information in the patch. The update process is fast and efficient, and will not significantly affect the user's normal use. In this way, the optimized personalized digital human model is updated in real-time on the browser side, allowing users to enjoy the better interactive experience brought by model optimization in a timely manner.

[0100] Furthermore, the step of automatically mining implicit positive and negative response patterns using unsupervised contrastive learning techniques based on an unlabeled interaction dataset, and training a pattern encoder to distinguish the superiority or inferiority of response strategies, specifically includes: Based on an unlabeled interactive dataset, pseudo-labels are used to label each round of digital human response strategies as either positive or negative response modes, according to pre-defined objective indicators of conversation continuity. Based on pseudo-labels, positive and negative sample pairs are constructed from unlabeled interactive datasets for contrastive learning. The positive sample pairs contain response strategies belonging to the same positive response mode, and the negative sample pairs contain response strategies belonging to positive and negative response modes respectively. Based on positive and negative sample pairs, a pattern encoder is trained using a contrastive loss function. The pattern encoder is used to map the input digital human response policy context features into policy pattern embedding vectors, and in the embedding space, the policy pattern embedding vectors corresponding to positive response patterns are close to each other, while the policy pattern embedding vectors corresponding to negative response patterns are far from each other.

[0101] In this embodiment, based on an unlabeled interaction dataset (which records detailed information about the digital human and the user's interactions over multiple rounds, including user input, the digital human's response strategy, and the context of the interaction), pseudo-labels are assigned to each round of the digital human's response strategy according to preset objective metrics for conversation continuity. These objective metrics cover multiple aspects, such as the user's willingness to continue interacting after receiving a digital human response and the smoothness of the interaction. Specifically, if the user quickly asks a new question or explores the topic further after hearing the digital human's reply, it indicates that the digital human's response has sparked the user's interest, allowing the interaction to continue smoothly. In this case, the digital human's response strategy for that round is labeled as a positive response mode. Conversely, if the user does not respond for a long time after receiving a reply or ends the conversation directly, it indicates that the digital human's response has failed to meet the user's needs, leading to an interruption in the interaction. In this way, the response strategy for that round is labeled as a negative response mode. In this way, each round of the digital human's response strategy in the unlabeled interaction dataset is assigned a corresponding pseudo-label, laying the foundation for subsequent comparative learning.

[0102] After completing the pseudo-label annotation, positive and negative sample pairs for contrastive learning are constructed from the unlabeled interactive dataset based on these pseudo-labels.

[0103] To construct positive sample pairs, two response strategies labeled as positive response patterns are randomly selected as a pair. These response strategies, belonging to the same positive response pattern, exhibit similar characteristics in terms of conversation continuity; for example, they can effectively guide users to continue interacting and maintain the fluency of the dialogue. By pairing them as positive sample pairs, the model can learn the common features of positive response patterns in subsequent contrastive learning.

[0104] The construction of negative sample pairs differs, selecting one response strategy from both the positive and negative response modes to form a pair. These negative sample pairs highlight the differences between positive and negative response modes; for example, a positive response strategy might promote interaction, while a negative response strategy might disrupt it. In contrastive learning, the model further clarifies the boundaries between positive and negative response modes by distinguishing these negative sample pairs.

[0105] Based on the constructed positive and negative sample pairs, the pattern encoder is trained. The pattern encoder is a deep neural network model whose function is to map the input digital human's response policy context features into policy pattern embedding vectors.

[0106] During training, positive and negative sample pairs are sequentially input into the pattern encoder. The pattern encoder processes the input response policy context features to generate corresponding policy pattern embedding vectors. Then, a contrastive loss function is used to evaluate the model's performance. The design goal of the contrastive loss function is to ensure that, in the embedding space, policy pattern embedding vectors corresponding to positive response patterns are close to each other, while those corresponding to negative response patterns are far apart.

[0107] Specifically, when inputting positive sample pairs, the contrastive loss function encourages the pattern encoder to minimize the distance between the two policy pattern embedding vectors, meaning the model needs to learn the similarity between positive response patterns. When inputting negative sample pairs, the contrastive loss function encourages the pattern encoder to maximize the distance between the two policy pattern embedding vectors, thereby strengthening the model's ability to distinguish between positive and negative response patterns.

[0108] Through continuous iterative training, the pattern encoder constantly adjusts its parameters, gradually optimizing the mapping of response policy context features, enabling it to accurately distinguish between positive and negative response patterns in the embedding space. Ultimately, a pattern encoder capable of efficiently and accurately distinguishing the merits of digital human response policies is trained, providing strong support for subsequent optimization of digital human models.

[0109] Furthermore, the optimization signal provided by the pattern encoder, through the adversarial training process between the discriminator and the generator in the generative adversarial network deployed on the server, drives the personalized digital human model, which acts as the generator, to optimize its response strategy and generation quality, so that the output of the personalized digital human model approaches a positive response mode. Specifically, this includes: Based on the policy pattern embedding vector output by the pattern encoder, a discriminator of the generative adversarial network is constructed. The discriminator is used to inherit the pattern encoder's ability to recognize positive response patterns. Set the strategy generation module of the personalized digital human model to be optimized as the generator of the generative adversarial network; According to the adversarial training framework, the discriminator training phase and the generator training phase are executed alternately. During the discriminator training phase, positive response pattern samples provided by the pattern encoder are compared with samples generated by the generator for training. During the generator training phase, the generator parameters are optimized based on the discriminator's evaluation of the generated samples and the task completion requirements. By iteratively executing the adversarial training process, the response strategy generated by the generator approaches the positive response pattern defined by the pattern encoder in terms of distribution.

[0110] In this embodiment, the discriminator of the generative adversarial network is constructed based on the policy pattern embedding vectors output by the pattern encoder. The pattern encoder, after prior training, already possesses the ability to distinguish between positive and negative response patterns. It can map the input digital human response policy context features into policy pattern embedding vectors, and in the embedding space, it ensures that vectors corresponding to positive response patterns are close to each other, while vectors corresponding to negative response patterns are far apart.

[0111] When constructing the discriminator, this characteristic of the pattern encoder is utilized, using the policy pattern embedding vector output by the pattern encoder as the input feature of the discriminator. Specifically, a large number of positive response pattern samples already processed by the pattern encoder are collected. These samples contain policy pattern embedding vectors corresponding to positive response policies under different scenarios and themes. Simultaneously, policy pattern embedding vectors of some negative response pattern samples are also prepared for comparison. Based on these policy pattern embedding vectors of positive and negative response patterns, the network structure of the discriminator is designed, for example, using common neural network structures such as multilayer perceptrons (MLP). The training objective of the discriminator is to inherit the pattern encoder's ability to recognize positive response patterns, accurately distinguishing whether the input policy pattern embedding vector comes from a positive or negative response pattern. During training, the discriminator undergoes supervised learning using labeled positive and negative response pattern samples, continuously adjusting the discriminator's parameters to gradually improve its accuracy in judging positive response pattern samples.

[0112] In another possible implementation, when constructing the discriminator of the generative adversarial network, the policy pattern embedding vector output by the pattern encoder is further processed to more accurately inherit the pattern encoder's ability to recognize positive response patterns. First, clustering analysis is performed on the policy pattern embedding vectors of the positive response pattern samples, grouping vectors with similar features into one class. For example, clustering can be performed based on factors such as the theme of the response policy and the interaction scenario, making positive response pattern samples in the same class more similar semantically and functionally. Then, the input layer structure of the discriminator is designed based on the clustering results. The center vector of each cluster is used as one feature dimension of the discriminator's input layer, while the original policy pattern embedding vector is retained as other feature dimensions. In this way, the discriminator can capture the local similarity and overall features of the positive response pattern samples at the input stage. In the hidden layer of the discriminator, a structure combining convolutional neural networks (CNN) and recurrent neural networks (RNN) is adopted. CNN can extract local features, and RNN can process sequence information because the policy pattern embedding vector may contain certain sequence features. Through this complex network structure, the discriminator can learn the characteristics of positive response patterns more deeply, improving its ability to distinguish between positive and negative response pattern samples.

[0113] The policy generation module of the personalized digital human model to be optimized is set as the generator of a generative adversarial network (GAN). This module is responsible for generating corresponding response policies based on user input and the current interaction context. This module typically contains multiple sub-modules, such as a semantic understanding sub-module, a knowledge retrieval sub-module, and a policy generation sub-module. When set as the generator, its input is the user's input information and the current interaction context information, and its output is the generated digital human response policy. The generator's initial parameters are initialized based on certain prior knowledge or a pre-trained model, but the generated response policy may not fully meet the requirements of an aggressive response pattern and needs to be optimized through subsequent adversarial training.

[0114] In another possible implementation, when the strategy generation module of the personalized digital human model is set as the generator of a generative adversarial network, the generator's functionality is extended. In addition to the basic generation of response strategies based on user input and interaction context, the generator adds a feedback learning module. This module receives the discriminator's evaluation results of the generated samples and transforms these results into quantifiable metrics, such as the probability that the generated sample is judged by the discriminator as a positive response pattern sample, and the similarity between the generated sample and positive response pattern samples. Besides user input and interaction context information, the generator's input also incorporates external knowledge sources, such as knowledge graphs and FAQ databases. These external knowledge sources can provide the generator with more information support, making its generated response strategies richer and more accurate. The generator's output remains the digital human response strategy, but during the generation process, the generated strategy is adjusted and optimized based on the metrics provided by the feedback learning module.

[0115] According to the adversarial training framework, the discriminator training phase and the generator training phase are executed alternately. The core idea of ​​the adversarial training framework is to drive the generator to continuously optimize its generation strategy through the game between the discriminator and the generator, making it increasingly difficult for the discriminator to distinguish between real samples (positive response pattern samples) and generated samples.

[0116] During the discriminator training phase, positive response pattern samples provided by the pattern encoder are compared with samples generated by the generator for training. Specifically, a subset of positive response pattern samples processed by the pattern encoder is selected as real samples, while the generator produces a batch of response policy samples based on the current parameters. These real and generated samples are then input into the discriminator, and the classification loss for the real and generated samples is calculated. Based on the classification loss, the discriminator parameters are updated using the backpropagation algorithm, enabling the discriminator to better distinguish between real and generated samples, further improving its ability to recognize positive response patterns.

[0117] During the generator training phase, the generator parameters are optimized based on the discriminator's evaluation of the generated samples and the task completion requirements. The discriminator's evaluation of the generated samples can be seen as feedback on the quality of the generated samples. If the discriminator classifies a generated sample as having a high probability of being a negative response pattern, it indicates that the generated sample still has a significant gap from the positive response pattern. Simultaneously, considering the task completion requirements, such as the requirement that the generated response strategy effectively guides the user to continue interacting and meets the user's information needs, the generator parameters are adjusted using optimization algorithms such as gradient descent based on this feedback information. This allows the generator to optimize in a direction that better aligns with the positive response pattern when generating response strategies in subsequent iterations.

[0118] By iteratively executing the adversarial training process described above, the discriminator training phase and the generator training phase are repeatedly repeated. In each iteration, both the discriminator and the generator continuously learn and improve. As the number of iterations increases, the response strategy generated by the generator gradually approaches the positive response pattern defined by the pattern encoder in terms of distribution. Specifically, in the initial stage, the samples generated by the generator may be easily distinguishable by the discriminator, but as training progresses, the generator continuously learns the characteristics of the positive response pattern, and the quality of the generated samples gradually improves. The discriminator also continuously adapts to the new samples generated by the generator, improving its recognition ability. After multiple iterations, the response strategy generated by the generator increasingly conforms to the requirements of the positive response pattern in terms of semantic understanding and interaction guidance, effectively improving the response strategy and generation quality of the personalized digital human model, making the interaction between the digital human and the user smoother and more natural, and improving user satisfaction.

[0119] Reference Figure 2 An embodiment of the present invention provides an AI digital human real-time interaction system 2 based on a web browser, the system 2 specifically comprising: Model distillation module 201 is used to process large-scale AI digital human generation models in the cloud using model distillation and quantization compression technology, generate and deploy lightweight models containing core facial expressions, lip movements and simple body movements to the user's web browser to form a local generation engine. The local model module 202 is used to dynamically build and load a personalized digital human model containing user preference appearance features, voice style library and behavioral response strategy based on the local generation engine and the user's historical interaction data securely synchronized from the server. The interaction analysis module 203 is used to collect multimodal input data from users during the interaction process, and to perform real-time analysis of the multimodal input data using an integrated lightweight multimodal emotion computing model to identify the user's emotional state and interaction intent. The parameter generation module 204 is used to generate, in real time, matching digital human facial expressions, action frame sequences and corresponding speech synthesis parameters based on the emotional state and interaction intention, according to the personalized digital human model. The optimization and update module 205 is used to automatically mine the implicit behavioral patterns in user feedback based on contrastive learning technology, optimize the response strategy and generation quality of the personalized digital human model through generative adversarial networks, and incrementally update the optimized personalized digital human model to the browser.

[0120] It is understandable that, such as Figure 1The content of the web browser-based AI digital human real-time interaction method embodiment shown is applicable to this web browser-based AI digital human real-time interaction system embodiment. The specific functions implemented in this web browser-based AI digital human real-time interaction system embodiment are the same as those shown below. Figure 1 The illustrated embodiment of the real-time interactive method for AI digital humans based on a web browser is the same, and the beneficial effects achieved are the same as those shown. Figure 1 The beneficial effects achieved by the illustrated embodiment of the real-time interactive method for AI digital humans based on a web browser are also the same.

[0121] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0122] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0123] Reference Figure 3 The present invention also provides a computer device 3, including: a memory 302 and a processor 301, and a computer program 303 stored on the memory 302. When the computer program 303 is executed on the processor 301, it implements the real-time interactive method of AI digital human based on a web browser as described in any of the above methods.

[0124] The computer device 3 may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will understand that... Figure 3 The computer device 3 is merely an example and does not constitute a limitation on the computer device 3. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0125] The processor 301 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may be an external storage device of the computer device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 3. Furthermore, the memory 302 may include both internal and external storage units of the computer device 3. The memory 302 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0127] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the real-time interactive method for AI digital humans based on a web browser, as described in any of the above methods.

[0128] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0129] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0130] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0131] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A real-time interactive method for AI digital humans based on a web browser, characterized in that, The method specifically includes: The large-scale AI digital human generation model in the cloud is processed using model distillation and quantization compression technology to generate and deploy a lightweight model containing core facial expressions, lip movements and simple body movements to the user's web browser, forming a local generation engine. Based on a local generation engine, a personalized digital human model containing user preference appearance features, a voice style library, and behavioral response strategies is dynamically built and loaded according to user historical interaction data securely synchronized from the server. During the interaction, multimodal input data of users is collected, and an integrated lightweight multimodal emotion computing model is used to analyze the multimodal input data in real time to identify the user's emotional state and interaction intention. Based on emotional state and interaction intent, the system generates matching digital human expressions, action frame sequences, and corresponding speech synthesis parameters in real time according to the personalized digital human model. By automatically mining implicit behavioral patterns in user feedback using contrastive learning techniques, the response strategy and generation quality of personalized digital human models are optimized through generative adversarial networks, and the optimized personalized digital human models are incrementally updated to the browser.

2. The method according to claim 1, characterized in that, The process employs model distillation and quantization compression techniques to process large-scale AI digital human generation models in the cloud, generating and deploying lightweight models containing core facial expressions, lip movements, and simple body movements to the user's web browser, forming a local generation engine. Specifically, this includes: In the cloud, based on model distillation technology, a large-scale AI digital human generation model is used as a teacher model to train student models. The student models are used to inherit the core facial expressions, lip movements and body movements of the large-scale AI digital human generation model. The student model is quantized and compressed to generate a lightweight inference model file; On the browser side, the inference model file is compiled into a WebAssembly module and loaded, and a local generation engine is built in the web browser environment; When an interaction is triggered, the JavaScript logic of the web browser calls the local generation engine, and the digital human's sequence image data and associated voice parameters are generated in real time by passing in the driving parameters.

3. The method according to claim 1, characterized in that, The process, based on a local generation engine, dynamically constructs and loads a personalized digital human model containing user-preferred appearance features, a voice style library, and behavioral response strategies, using user historical interaction data securely synchronized from the server. Specifically, this includes: In the cloud, analyze the historical interaction data of a specified user in the server, extract and generate a set of personalized configuration parameters that includes appearance feature vectors, voice style parameters and behavioral response strategy weights; Based on a secure communication link, the personalized configuration parameter set is synchronized from the server to the web browser on the browser side; Within the web browser, the deployed local generation engine receives and parses the synchronized set of personalized configuration parameters to obtain the parsing results. Based on the analysis results, a style generation module is constructed according to the appearance feature vector, a speech synthesis pipeline is constructed according to the speech style parameters, and a behavior decision logic module is constructed according to the behavior response strategy weights, thus forming a personalized digital human model.

4. The method according to claim 1, characterized in that, During the interaction, multimodal input data from the user is collected, and an integrated lightweight multimodal emotion computing model is used to analyze the multimodal input data in real time to identify the user's emotional state and interaction intent. Specifically, this includes: In a web browser environment, text streams, audio streams, and video streams generated by user interactions are collected in parallel as multimodal input data; Multimodal input data is processed by the corresponding localized lightweight feature extraction module to generate text semantic feature vectors, speech acoustic feature vectors, and visual expression feature vectors. Text semantic feature vectors, speech acoustic feature vectors, and visual expression feature vectors are simultaneously input into a locally deployed lightweight multimodal fusion decision model. The lightweight multimodal fusion decision model performs fusion calculations on the input multimodal feature vectors to identify the user's real-time emotional state classification and instant interaction intent.

5. The method according to claim 1, characterized in that, The process of generating matching digital human facial expressions, action frame sequences, and corresponding speech synthesis parameters in real time based on emotional state and interaction intent, according to a personalized digital human model, specifically includes: Based on emotional state and interaction intent, combined with user preference parameters in the loaded personalized digital human model and the text content to be replied to, a comprehensive conditional encoding vector is obtained. The conditional encoding vector is input into a locally deployed lightweight conditional generation model for single inference, simultaneously generating a temporally aligned sequence of digital human visual action parameters and a sequence of speech synthesis parameters. The visual action parameter sequence is used to define the digital human's facial expressions and body movements, and the speech synthesis parameter sequence is used to define the acoustic features of the synthesized speech. The visual motion parameter sequence is fed to the web browser's graphics rendering interface to drive the real-time animation of the digital human figure, and the speech synthesis parameter sequence is fed to the web browser's audio synthesis interface to generate the corresponding speech waveform.

6. The method according to claim 1, characterized in that, The process of automatically mining implicit behavioral patterns in user feedback using contrastive learning techniques, optimizing the response strategy and generation quality of the personalized digital human model through generative adversarial networks, and incrementally updating the optimized personalized digital human model to the browser includes: Based on the desensitized implicit feedback sequences collected during local interaction with a web browser, an unlabeled interaction dataset containing labels of subsequent user behaviors is constructed in the cloud. Based on an unlabeled interactive dataset, implicit positive and negative response patterns are automatically mined using unsupervised contrastive learning techniques, and a pattern encoder is trained to distinguish the superiority or inferiority of response strategies. Based on the optimization signal provided by the pattern encoder, the discriminator and generator in the generative adversarial network deployed on the server are subjected to an adversarial training process, which drives the personalized digital human model as the generator to optimize the response strategy and generation quality, so that the output of the personalized digital human model approaches the positive response mode. The parameters of the optimized personalized digital human model are compared differentially with the original model parameters on the browser side to generate an incremental update patch, which is then synchronized to the web browser environment on the browser side.

7. The method according to claim 6, characterized in that, The process of automatically mining implicit positive and negative response patterns using unsupervised contrastive learning techniques based on an unlabeled interactive dataset, and training a pattern encoder to distinguish the superiority or inferiority of response strategies, specifically includes: Based on an unlabeled interactive dataset, pseudo-labels are used to label each round of digital human response strategies as either positive or negative response modes, according to pre-defined objective indicators of conversation continuity. Based on pseudo-labels, positive and negative sample pairs are constructed from unlabeled interactive datasets for contrastive learning. The positive sample pairs contain response strategies belonging to the same positive response mode, and the negative sample pairs contain response strategies belonging to positive and negative response modes respectively. Based on positive and negative sample pairs, a pattern encoder is trained using a contrastive loss function. The pattern encoder is used to map the input digital human response policy context features into policy pattern embedding vectors, and in the embedding space, the policy pattern embedding vectors corresponding to positive response patterns are close to each other, while the policy pattern embedding vectors corresponding to negative response patterns are far from each other.

8. The method according to claim 7, characterized in that, The optimization signal provided by the pattern encoder drives the personalized digital human model, acting as the generator, to optimize its response strategy and generation quality through an adversarial training process between the discriminator and the generator in a generative adversarial network deployed on the server. This makes the output of the personalized digital human model approach a positive response mode, specifically including: Based on the policy pattern embedding vector output by the pattern encoder, a discriminator of the generative adversarial network is constructed. The discriminator is used to inherit the pattern encoder's ability to recognize positive response patterns. Set the strategy generation module of the personalized digital human model to be optimized as the generator of the generative adversarial network; According to the adversarial training framework, the discriminator training phase and the generator training phase are executed alternately. During the discriminator training phase, positive response pattern samples provided by the pattern encoder are compared with samples generated by the generator for training. During the generator training phase, the generator parameters are optimized based on the discriminator's evaluation of the generated samples and the task completion requirements. By iteratively executing the adversarial training process, the response strategy generated by the generator approaches the positive response pattern defined by the pattern encoder in terms of distribution.

9. A real-time interactive system for AI digital humans based on a web browser, characterized in that, The system specifically includes: The model distillation module is used to process large-scale AI digital human generation models in the cloud using model distillation and quantization compression technology, generate and deploy lightweight models containing core facial expressions, lip movements and simple body movements to the user's web browser, forming a local generation engine. The local model module is used to dynamically build and load personalized digital human models containing user preference appearance features, voice style library and behavioral response strategies based on the local generation engine and user historical interaction data securely synchronized from the server. The interaction analysis module is used to collect multimodal input data from users during the interaction process, and to perform real-time analysis of the multimodal input data using an integrated lightweight multimodal emotion computing model to identify the user's emotional state and interaction intent. The parameter generation module is used to generate matching digital human facial expressions, action frame sequences, and corresponding speech synthesis parameters in real time based on the emotional state and interaction intent, according to the personalized digital human model. The optimization and update module is used to automatically mine implicit behavioral patterns in user feedback based on contrastive learning technology, optimize the response strategy and generation quality of personalized digital human models through generative adversarial networks, and incrementally update the optimized personalized digital human models to the browser.

10. A computer device, characterized in that, include: The memory and processor, and the computer program stored in the memory, which, when executed on the processor, implement the real-time interactive method for AI digital humans based on a web browser as described in any one of claims 1 to 8.

Images