A method for simultaneous live streaming in multiple live streaming rooms based on AI digital humans

Abstract

This invention belongs to the field of multi-live-room synchronous live streaming technology, specifically involving a method for multi-live-room synchronous live streaming based on AI digital humans. The method includes: deploying a system template; constructing audience profiles and script templates; rendering seeds and initializing a lightweight ledger; performing edge deterministic rendering during live streaming; employing distributed ledger recording and local perception optimization; performing playback verification based on the ledger and frame records; quantifying business effects and conducting offline training. This invention can reduce the cross-node consistency of visual and audio-visual aspects of digital humans in multiple live-streaming rooms, thereby reducing the perceptual latency on the viewing end and improving the real-time viewing experience for the audience.

Description

Technical Field

[0001] This invention belongs to the field of multi-room synchronous live streaming technology, specifically relating to a method for multi-room synchronous live streaming based on AI digital humans. Background Technology

[0002] The Chinese patent application number CN202311617801.8 discloses a method and apparatus for simultaneous live streaming across multiple live streaming rooms based on AI digital humans. This invention involves: acquiring a live streaming request from a client device; determining N digital human images based on the products to be sold in the live stream; rendering the N digital human images according to a live streaming script to obtain N live streaming videos; sending the N live streaming videos to the client device; acquiring a start-up command from the client device; and controlling the playback progress of the live streaming videos in the live streaming rooms of each of the N live streaming accounts, ensuring that the playback progress of the live streaming videos in each live streaming account's live streaming room is consistent at preset live streaming nodes.

[0003] While this invention addresses the issues of poor streaming quality and low conversion rates caused by relying solely on a single digital human in live video applications within image communication, it also reveals a problem in multi-room live streaming: audio and video may become out of sync, leading to conflicts between different rooms and causing viewers to miss out on group-buying deals. Furthermore, the consistency and real-time nature of the digital human's lip movements, facial expressions, and voice during cross-room live streaming are difficult to guarantee. Additionally, the digital human's live streaming room struggles to aggregate backend events during interactions, resulting in duplicate billing and missed orders. Therefore, a multi-room synchronous live streaming method based on AI digital humans is urgently needed to solve these problems. Summary of the Invention

[0004] This invention provides a method for simultaneous live streaming in multiple live streaming rooms based on AI digital humans. The purpose is to solve the problem of audio and video desynchronization in live streaming rooms when digital humans are live streaming in multiple live streaming rooms, which leads to conflicts between different rooms and causes people in the live streaming room to miss group purchases. It is also difficult to guarantee the consistency and real-time performance of the lip movements, facial expressions and voice of digital humans when live streaming across rooms. Furthermore, it is difficult to summarize backend events during the interaction of digital human live streaming rooms, resulting in duplicate billing and missed orders.

[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a method for simultaneous live streaming in multiple live rooms based on AI digital humans, comprising:

[0006] Deploy system templates; build audience profiles by collecting user information; build script templates based on the products sold in the live stream and the live stream content; render the seed and initialize the lightweight ledger;

[0007] During live streaming, edge deterministic rendering is performed by calling a deterministic rendering function to render the current seed and generating text and voice events corresponding to the current frame through a central script; a distributed ledger is used for recording, and sorting is performed according to the lexicographical priority of the keys by defining sorting keys, and key interactions must carry a unique transaction identifier; a policy gradient formula is constructed for local perception optimization.

[0008] Replay verification is performed based on records and frame records from the distributed ledger; business effects are quantified and offline model training formulas are constructed for offline training.

[0009] As a preferred implementation, the specific steps for constructing a script template based on the products sold in the live stream and the live stream content are as follows:

[0010] Based on the products sold and the content of the live stream, the process nodes and compliance requirements of the live stream are determined, resulting in the writing specifications and constraints of the script template. A global script is written according to the script template specifications, resulting in the main script text covering the live stream process. Variable segment templates are obtained by extracting localizable paragraphs into fragments and setting parameter placeholders. A localized dictionary for replacement is obtained by merging the variable segment templates, localized thesaurus, greetings, and promotional statements, comparing and correcting them with the compliance thesaurus. Compliance feedback is obtained by marking the replacements in the main script text and variable segment templates. A deployable and traceable script resource package is obtained by incorporating the global script, variable fragments, localized dictionary, and compliance feedback into the template repository and implementing version control. A verification report on the script rendering and replacement logic is obtained by deploying the script resource package to an edge testing environment and performing replay simulations and item verification.

[0011] As a preferred embodiment, the specific steps for rendering the seed are as follows:

[0012] The central orchestrator reads the session identifier, script version number, and timestamp from the script resource package, concatenates the read data in the order of reading, and performs a secure hash calculation to obtain the rendering seed. It then constructs a seed package by packaging the rendering seed and seed metadata. A signature information is obtained by digitally signing the seed package and attaching the signature information to the seed package. The seed package is written to the lightweight distributed event ledger as a baseline record by calling the write interface of the lightweight distributed event ledger. Edge rendering nodes receive the signed seed package from the central orchestrator and verify its validity using a public key. Under the same software version, they generate identical frame data and combine the generated frame data into a set to obtain a deterministic frame sequence. Each edge rendering node writes the frame hash and hash chain value of each frame to the lightweight distributed event ledger by calling the ledger interface. The central orchestrator compares the frame hashes and hash chain values ​​written by at least two edge rendering nodes under the same rendering seed and determines whether they are consistent.

[0013] As a preferred embodiment, the specific steps for edge deterministic rendering during live streaming are as follows:

[0014] The central orchestrator queries and verifies signed rendering seeds and assigns them a trust tag; edge nodes render the current seed by calling the deterministic rendering function to obtain the rendering result of the current frame; the central script generates the text and speech events corresponding to the current frame to obtain the audio stream and phoneme sequence; dynamic time warping is applied to the audio stream and phoneme sequence to perform alignment mapping to ensure that the digitizer's lip movements are aligned with the audio frame; the output of the deterministic rendering function is fine-tuned according to the alignment result to obtain the aligned rendering result.

[0015] As a preferred implementation, the specific steps for recording using a distributed ledger are as follows:

[0016] By defining sorting keys and sorting them according to lexicographical priority, an event ordering rule for cross-node comparisons is obtained; by stipulating that key interactions must carry unique transaction identifiers, a convention for transaction idempotency and backtracking is obtained; by digitally signing detected interaction events with their own private keys to obtain signature information, and calling the lightweight ledger interface to submit write requests, evidence of the write request submission is obtained; the write request is confirmed by the authorized consensus selected by the ledger node and a confirmation certificate is returned, and the edge node obtains the certificate that the write request is effective by receiving the returned confirmation certificate.

[0017] As a preferred implementation, the specific steps for constructing the strategy gradient formula for local perception optimization are as follows:

[0018] Edge nodes exchange timestamps with other reference nodes, and then subtract the obtained timestamps from their local standard time. The difference in error is within... Within the normal range, exceeding the normal range is not acceptable. If a value is marked as delayed, it is then marked as local delay. The local delay is optimized by constructing a policy gradient update formula, which is:

[0019] ,

[0020] It is the action function value, representing the value at the time of action. Actions used in a state The action function value, D is the action function value. It is the state at time g, and e is the state. The action is executed at time g, where f is the action and g represents the time index. It is an assignment operation. For learning rate, In order to complete the action The timely reward received later, where h represents the timely reward. The discount factor is used to measure the present value of future returns. To find the next state All possible actions peak value To perform the action The next state that is reached later. For all possible actions.

[0021] As a preferred implementation, the specific steps for replay verification based on the records and frame records of the distributed ledger are as follows:

[0022] By calling the lightweight ledger query interface to read the seed resource package, frame record set, and interaction event record set within the session scope, the ledger records required for replay are obtained. Public key verification is performed on the signature fields of the ledger records, and the signature validity of each ledger record is verified to obtain the signature verification result. The resource package list and version number recorded in the ledger are verified, and the corresponding resource packages are called from the resource repository to obtain the data set used for recalculation rendering. The rendering seeds in the ledger are verified and signed to obtain trusted rendering seeds. Under the same software version and resource set conditions, the rendering seeds and frame numbers are recalculated using a deterministic rendering function to obtain the recalculated frame. The hash value of the recalculated frame is calculated and... Using the initial seed, a hash is calculated for each recalculated frame, and a recalculated hash chain is constructed sequentially according to the recursive relationship to obtain a hash chain used for item-by-item comparison and playback verification. By reading the local timestamp of each record in the ledger and using a clock offset correction method to map each local time to a unified reference clock, a unified timeline for playback is obtained. By synthesizing the recalculated frame sequence with the corresponding audio stream according to the unified timeline and generating a playback media file at a predetermined playback rate, playback material that can be automatically reviewed is obtained.

[0023] As a preferred implementation method, the specific steps for quantifying business effectiveness are as follows:

[0024] By reading transaction records, frame records, and scheduling commands from the lightweight distributed event ledger, the original event sequence of the session organization is obtained; by aligning the original event sequence with the time windows of the recalculated frames, sample units aggregated by time windows are obtained; by performing desensitization, missing value imputation, and noise reduction on the original samples, a clean dataset for training is obtained; by encoding and normalizing candidate features, the model input feature vector is obtained; by defining a reward signal, the business indicators obtained from playback are used as positive revenue, and the perceived latency and the lack of consistency in actions or audio of the digital human in multiple rooms during live broadcasts are used as penalty items, thus obtaining the target value required for supervised learning; by defining key business indicators, a quantitative measurement standard is obtained; by simulating the playback environment of historical records and correcting the bias problem of historical strategies using importance sampling, the performance of the corrected historical business is obtained; by calculating various business indicators on the validation set and performing statistical significance tests, the criteria for determining whether the candidate improvement is significant are obtained; by associating the evaluation results, playback logs, and ledger evidence and recording them in the audit log, an auditable evaluation evidence chain is obtained.

[0025] As a preferred embodiment, the specific steps for constructing the offline model training formula and performing offline training are as follows:

[0026] Offline training is performed using de-identified training samples collected from playback and ledger data. An offline model training formula is constructed for this purpose. The formula for offline model training is as follows:

[0027] ,

[0028] Let J be the offline model function, representing the predicted value obtained after w training iterations from the input state feature vector R. Here, J is the predicted value, R is the state vector, and w is the number of training iterations. This is the initial model, representing a constant estimate of the trough of the total loss on the training set. J is the constant estimate of the trough of the total loss, Q is the observed value of the state vector R, and S is the learning rate. Let t be the learning function of the state vector R in the w-th round, and t be the base learner.

[0029] As a preferred implementation, an auditable verification baseline and verification credentials are obtained by writing the version number of the offline training model, the binary representation of the offline training model, and the list of validation samples into a lightweight distributed event ledger. In the replay environment, key metrics are calculated by inferring the offline training model on the validation set and the reserved test set, resulting in an offline performance metric for the offline training model. By running the obtained offline training model as an experimental strategy in a controlled, low-traffic room and collecting key business and consistency metrics in real time, the performance observation value U of the digital human multi-live-room synchronous live streaming under the offline training model can be observed in a near-realistic environment. A consistency performance threshold L is set. If the performance of the representative data improves after using the offline model, then the data will be distributed. If performance is not improved, revert to the previous version.

[0030] The beneficial effects of this invention are as follows:

[0031] 1. By rendering a seed and using that seed to drive a deterministic rendering function at the edge nodes to generate frames and writing the frame hash chain into a lightweight distributed event ledger, consistent visual and lip-sync output that is reproducible at the bit level can be obtained across different live streaming rooms, thereby reducing visual and audio asynchrony and promotional rhythm conflicts caused by differences in cross-room rendering.

[0032] 2. By performing deterministic rendering at the edge and writing interactive events to a hierarchical ledger in a two-stage manner to achieve a fast confirmation and rollback mechanism, the measurability of end-to-end perceived latency and jitter is reduced, as well as the ability to recover quickly in abnormal situations, thereby improving the user experience and reducing latency.

[0033] 3. By writing key interaction events, submission records, and model versions into a distributed ledger with signed, immutable records and comparing the hash chain item by item during playback to verify the order of events and the number of confirmations, we can achieve a controllable reduction in the occurrence rate of duplicate billing and missed orders, as well as the ability to automatically determine disputes, thereby reducing operational settlement and improving transaction consistency and platform trust.

[0034] Legend

[0035] Figure 1 This is a flowchart of a method for simultaneous live streaming across multiple live streaming rooms based on AI digital humans.

[0036] Figure 2 This is a technical rendering of a method for simultaneous live streaming across multiple live streaming rooms based on AI digital humans. Detailed Implementation

[0037] To make the technical means, creative features, and achieved objectives and effects of this invention easier to understand, the invention is further described below with reference to specific embodiments. However, the following embodiments are merely preferred embodiments of this invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort are all within the protection scope of this invention. Unless otherwise specified, the experimental methods in the following embodiments are conventional methods, and the materials and reagents used in the following embodiments are commercially available unless otherwise specified.

[0038] Example 1 Figure 1 A flowchart of a method for simultaneous live streaming across multiple live streaming rooms based on AI digital humans is presented, including: deploying system templates; constructing audience profiles and script templates; rendering seeds and initializing a lightweight ledger; performing edge deterministic rendering during live streaming; employing distributed ledger recording and local perception optimization; performing replay verification based on the ledger and frame recordings; quantifying business effects and performing offline training; the specific implementation steps are as follows:

[0039] Step 1: Deploy system templates; build audience profiles and script templates; render seeds and initialize lightweight ledgers.

[0040] Specifically, the deployment steps of the system template are as follows: by deploying a central orchestrator, a central control service for issuing scripts and scheduling instructions is obtained; and several edge rendering nodes are deployed in the target area to obtain real-time rendering and caching capabilities close to the audience; by installing and configuring lightweight distributed event ledger software, ledger nodes are obtained, and the administrator configures node certificates between ledger nodes to obtain a controlled and verifiable ledger network environment; by configuring a clock synchronization service and issuing anchor timestamps, a time base for cross-node time alignment is obtained; the ledger nodes are key nodes used to record rendering seeds, keyframe hashes, and interactive events.

[0041] The specific steps for constructing audience profiles are as follows: With explicit user consent and ensuring that the information obtained is used only for user profile construction, user registration information, interaction logs, viewing records, and transaction information are collected. The collected data is anonymized and aggregated according to its original data type to obtain the original dataset for profile construction. The original dataset is preprocessed, specifically by deduplication, missing value imputation, and format standardization, resulting in cleaned feature data. The preprocessed feature data is then filtered by field, extracting user interaction behavior, viewing duration and click frequency, device attributes such as device type and network type, demographic data, user age group, and region, thus obtaining user behavioral feature data, device feature data, and demographic feature data as a set of feature vectors for modeling. This set of feature vectors is encoded, and principal component analysis is used to reduce the dimensionality of the encoded high-dimensional feature vector set, resulting in a dimensionality-reduced feature vector set. The dimensionality-reduced feature vector set is then aligned, and the feature vectors in the dimensionality-reduced feature vector set are concatenated as rows of a matrix to obtain a matrix of shape [shape missing]. The encoding feature matrix C is obtained; the audience group labels are obtained by clustering the dimensionality-reduced feature vector set according to data type; the audience group labels are sampled and compared with the manually labeled control set. The confidence scores of the manually labeled control set and the sampled audience group labels are calculated. When the confidence scores of the manually labeled control set and the sampled audience group labels differ, the confidence score is determined. If the confidence levels are not different, it meets the requirements. Within this range, the sampling sample is expanded and re-sampled; by writing audience profiles and audience group tags into the profile database and setting regular updates, profile services are provided to the central orchestrator and edge nodes.

[0042] The specific steps for building a script template are as follows: First, determine the process nodes and compliance requirements of the live stream based on the products sold and the live stream content, thus obtaining the script template writing specifications and constraints. Second, write a global script according to the script template writing specifications, thus obtaining the main script text covering the live stream process. Third, extract localizable paragraphs into segments and set parameter placeholders to obtain variable segment templates. Fourth, merge the variable segment templates, localized thesaurus, greetings, and promotional statements, and compare and correct them with the compliance thesaurus to obtain a localized dictionary for replacement. Fifth, mark the replacements between the main script text and the variable segment templates to obtain compliance feedback. Sixth, incorporate the global script, variable segments, localized dictionary, and compliance feedback into the template repository and implement version control to obtain a template that is both deployable and reproducible. The script resource package is used for tracing; by distributing the script resource package to the edge testing environment and performing playback simulation and item verification, a verification report of the script rendering and replacement logic is obtained; the localizable paragraphs are paragraph units marked in the script text according to predefined placeholders and metadata. The placeholders identify the content items to be replaced according to region and language. By marking the paragraphs and associating them with the localized dictionary, semantically equivalent and culturally adapted text can be generated in different regions; the variable fragment templates are localized local texts generated by the localizable paragraphs according to different regions; the replaceable localized dictionary maps placeholders to text and speech resources in different regional languages ​​according to different regions and languages, and records the dictionary version and signature through a ledger to ensure the integrity and auditability of the replaced content.

[0043] The specific steps for rendering a seed are as follows: The central orchestrator reads the session identifier, script version number, and timestamp from the script resource package, concatenates the read data in the order it was read, and then performs a secure hash calculation to obtain the rendering seed; the rendering seed and seed metadata are packaged together to construct a seed package; the seed metadata includes: session identifier, script version number, and timestamp; the seed package is digitally signed to obtain signature information, which is then attached to the seed package to obtain a seed package containing the signature; the seed package is written to the lightweight distributed event ledger as a baseline record by calling the write interface of the lightweight distributed event ledger, thereby facilitating subsequent verification of the seed by edge nodes. Pre-rendering: Edge rendering nodes receive a signed seed package from the central orchestrator and verify its validity using a public key to obtain a trustworthy rendering seed. Under the same software version, they generate the same frame data and combine the generated frame data into a set to obtain a deterministic frame sequence. Edge rendering nodes write the frame hash and hash chain value of each frame into a lightweight distributed event ledger by calling the ledger interface. The central orchestrator compares the frame hash and hash chain values ​​written by at least two edge rendering nodes under the same rendering seed and determines whether they are consistent. If they are inconsistent, the system triggers a rollback to verify the rendering version and the integrity of the seed resource package, thereby obtaining a preliminary confirmation of rendering consistency between the edges.

[0044] The specific steps for initializing the lightweight ledger are as follows: by generating a key pair for each lightweight ledger node and having a node certificate issued by a trusted issuing authority, credentials for node authentication are obtained; by writing a node list and permission policies into the initial state of the lightweight ledger, a controlled lightweight ledger node access policy is obtained; and by creating a genesis block and writing system parameters, a node list, and seed metadata into the genesis block, a genesis record for starting the ledger network is obtained.

[0045] Step two: Perform edge deterministic rendering during live streaming; use distributed ledger recording and local perception optimization.

[0046] Specifically, the steps for edge deterministic rendering during live streaming are as follows: The central orchestrator queries and verifies signed rendering seeds, thus confirming and tagging them with a trust label; edge nodes load corresponding rendering resource packages, including: emoji libraries, skeletal templates, and material assets, and verify the public key and digital signature of the rendering resource packages, thereby obtaining a rendering runtime environment consistent with the central orchestrator; edge nodes render the current seed by calling a deterministic rendering function to obtain the rendering result of the current frame; the deterministic rendering function takes the rendering seed, frame index, rendering version, and resource set as input, and constructs a... A deterministic random number generator is used as the initial seed, and the quadruple (seed, index, version, resource) is mapped to frame output in a normalized resource loading order, thus ensuring that different edge nodes generate consistent frames under the same output conditions. The normalized resource loading order is based on the ascending order of resources, ensuring that each edge node uses the same resource set and loading sequence under the same input. The central script generates the text and voice events corresponding to the current frame, thus obtaining the audio stream and phoneme sequence. Dynamic time warping is applied to the audio stream and phoneme sequence for alignment mapping, thus ensuring that the lip movements of the digital human are aligned with the audio frames. The output of the deterministic rendering function is fine-tuned according to the alignment result, thus obtaining the aligned rendering result and achieving audio-visual synchronization. Fine-tuning the output of the deterministic rendering function according to the alignment result includes adjusting the frame timestamp. At a predetermined time point, a set of keyframes is pre-rendered to obtain local cached frames for fast switching. The set of keyframes includes: opening, switching, and keyframes at the moment of the lottery, etc. When the frame delay deviation of the digital human live broadcast is detected locally to exceed the threshold, the playback pointer is atomically replaced to switch. Switching to pre-rendered keyframes minimizes visual interruption perceptible to the user. Auditable switching records are obtained by writing and signing the switching event to a lightweight ledger after the switch. A controllable playback buffer is created by dynamically adjusting the buffer size based on clock synchronization and latency estimation. This controllable playback buffer is determined by a latency estimation and compensation decision function. The latency estimation is achieved by real-time measurement of round-trip latency, jitter, and clock offset between edge nodes and the viewer, and the latency estimate is calculated based on the real-time measurement data to obtain data that can be used to determine the triggering of the compensation function. The compensation function... By setting a latency safety threshold, a compensation function is triggered when the latency estimate exceeds the threshold. The compensation action is calculated and executed by taking state vectors such as latency estimate, jitter, and packet loss rate as input, thereby improving the user's perceived latency and viewing experience. By sending the encoded output of real-time rendering to the viewer using a low-latency transmission protocol for streaming media playback, low-latency end-to-end real-time viewing is achieved. Feedback from both the sender and viewer ends is collected, including data such as the receive buffer status and packet loss rate, and reported to the central orchestrator to obtain real-time data for the scheduler.

[0047] The specific steps of using a distributed ledger are as follows: First, sorting is performed according to the lexicographical priority of the keys by defining a sorting key, thus obtaining an event ordering rule for cross-node comparison. The lexicographical priority of the keys is then compared item by item according to a preset rule: for timestamp fields, chronological order is used; for numeric fields, numerical ascending order is used; and for character fields, lexicographical ascending order is used. Second, a convention is established that key interactions must carry a unique transaction identifier, thus achieving transaction idempotency and traceability. Key interactions include actions such as placing orders, participating in lotteries, and making payments. Third, a signature is obtained by digitally signing the detected interaction events using the user's private key, and a write request is submitted via a lightweight ledger interface, thus obtaining evidence of the write request submission. Fourth, the write request is confirmed by the authorized consensus mechanism selected by the ledger node, which returns a confirmation credential. The edge node receives the returned confirmation credential, thus obtaining proof that the write request is effective.

[0048] The specific steps of local perception optimization are as follows: edge nodes exchange timestamps with other reference nodes, and then subtract the obtained timestamps from the local standard time. The difference in error is within... Within the normal range, exceeding the normal range is not acceptable. If a label is marked as delayed, it is labeled as locally delayed. To reduce local delay, a policy gradient update formula is constructed to optimize the local delay of the label. The policy gradient update formula is as follows:

[0049] ,

[0050] It is the action function value, representing the value at the time of action. Actions used in a state The action function value, D is the action function value. It is the state at time g, and e is the state. The action is executed at time g, where f is the action and g represents the time index. It is an assignment operation. For learning rate, In order to complete the action The timely reward received later, where h represents the timely reward. The discount factor is used to measure the present value of future returns. To find the next state All possible actions peak value To perform the action The next state that is reached later. For all possible actions.

[0051] Step 3: Perform playback verification based on the ledger and frame records; quantify the business effect and conduct offline training.

[0052] Specifically, the replay verification steps based on the ledger and frame records are as follows: First, the seed resource package, frame record set, and interaction event record set within the session scope are read by calling the lightweight ledger query interface to obtain the ledger records required for replay. Second, public key verification is performed on the signature fields in the ledger records, and the signature validity of each ledger record is verified to obtain the signature verification result, and signature anomalies are recorded in the audit log. The signature anomalies include: incomplete or untrusted signature certificate chains, revoked signature certificates, etc. Third, the resource package list and version number recorded in the ledger are verified, and the corresponding resource package is called from the resource repository to obtain the data set used for recalculation rendering. Fourth, the rendering seeds in the ledger are verified and signed to obtain trusted rendering seeds. Fifth, under the same software version and resource set conditions, the rendering seeds and frame numbers are recalculated using a deterministic rendering function to obtain recalculated frames. Sixth, the hash value of the recalculated frames is calculated and... Using the initial seed, a hash is calculated for each recalculated frame, and a recalculated hash chain is constructed sequentially according to the recursive relationship to obtain a hash chain used for item-by-item comparison and replay verification, thus obtaining the recalculated hash chain. By comparing the recalculated hash chain with the corresponding frame record in the ledger item by item, when the recalculated hash chain and the corresponding frame record in the ledger are not equal, the audit node marks the frame as inconsistent and records the inconsistency position and difference type. By globally sorting the event record set according to the timestamp order based on the sorting key, an event sequence arranged in global order is obtained. By retrieving the write-ahead and commit records of each key transaction in the ledger and verifying the commit records of key transactions in the ledger, the number of commit records returned by different nodes is confirmed. If the number of returned transactions is not less than a preset threshold, it indicates that the two commits of the transaction are consistent. By comparing the sort key of any concurrent transaction with the ledger confirmation time order, the global sequence is determined according to the ledger confirmation time. The sort key of the concurrent transactions represents events that occur at the same time. The confirmation time order is sorted according to the order of timestamps. By reading the local timestamps of each record in the ledger and using a clock offset correction method to map each local time to a unified reference clock, a unified timeline for playback is obtained. By synthesizing the recalculated frame sequence with the corresponding audio stream according to the unified timeline and generating a playback media file at a predetermined playback rate, playback material that can be automatically reviewed is obtained.

[0053] The specific steps for quantifying business performance and conducting offline training are as follows: The original event sequence of the session organization is obtained by reading transaction records, frame records, and scheduling commands from the lightweight distributed event ledger; sample units aggregated by time windows are obtained by aligning the original event sequence with the time windows of the recalculated frames; each sample unit contains three elements: state, behavior, and consequence; a clean dataset for training is obtained by performing desensitization, missing value imputation, and noise reduction on the original samples; the model input feature vector is obtained by encoding and normalizing candidate features; the candidate features include: audience profile embedding, short-term interaction density, latency estimation, historical conversion rate, etc.; by defining a reward signal, the business metrics obtained from playback are used as positive revenue, and perceived latency and the lack of consistency in actions or audio for digital humans in multiple room live streams are used as penalty terms, thus obtaining the target value required for supervised learning; by defining key business metrics, a quantitative measurement standard is obtained; the key business metrics include: conversion rate, revenue per unit time, average dwell time, etc.; by... The simulation of candidate strategies is performed in the replay environment of historical records, and importance sampling is used to correct the bias problem of historical strategies, thereby obtaining the performance of historical business after correction. The simulation of candidate strategies is carried out by re-enacting the historical interactions of digital human live broadcast in the replay environment and calculating the expected business indicators under various scenarios, thereby providing data support for offline training. The importance sampling corrects the bias of historical strategies by weighting the candidate strategies generated from historical strategies in the replay data according to the ratio of historical strategies to candidate strategies. The historical decision bias problem is: selection bias and insufficient coverage caused by sampling under finite action distribution, confusion bias caused by causal confusion and unobserved variables, loss bias caused by sample loss, and missing behavioral information in the log, which causes systematic estimation bias in the replay data. By calculating various business indicators on the validation set and performing statistical significance tests, the criteria for judging whether the candidate improvement is significant are obtained. By linking the evaluation results, replay logs and ledger evidence and recording them in the audit log, an auditable evaluation evidence chain is obtained.

[0054] After quantifying the business results, offline training is performed on de-identified training samples collected from playback and ledgers. An optimized model for online lightweight prediction and scheduling is obtained by constructing an offline model training formula. This improves the consistency and auditability of digital humans across multiple live streaming rooms, reduces perceived latency, enhances business conversion, and reduces online trial-and-error risks. The offline model training formula is as follows:

[0055] ,

[0056] Let J be the offline model function, representing the predicted value obtained after w training iterations from the input state feature vector R. Here, J is the predicted value, R is the state vector, and w is the number of training iterations. This is the initial model, representing a constant estimate of the trough of the total loss on the training set. J is the constant estimate of the trough of the total loss, Q is the observed value of the state vector R, and S is the learning rate. Let t be the learning function of the state vector R in the w-th round, and t be the base learner.

[0057] After constructing the training model, to test whether the offline-trained model can improve the live streaming effect of digital humans, the version number, binary representation, and validation sample list of the offline training model are written into a lightweight distributed event ledger to obtain an auditable validation baseline and validation credentials. In the replay environment, inferences are performed on the offline training model on the validation set and the reserved test set, and key indicators are calculated to obtain the offline performance metrics of the offline training model. These key indicators include: revenue, latency impact, etc. The offline performance metrics are calculated on the validation set using key business and technical indicators, combined with significance testing to quantify and evaluate the performance and stability of the candidate model in an offline environment. The obtained offline training model is run as an experimental strategy in a controlled low-traffic room, and key business and consistency indicators are collected in real time, allowing for the observation of the performance observation value U of the digital human multi-live streaming synchronous live streaming under the offline training model in a near-realistic environment. A consistency performance threshold L is set. If the performance of the time-representing numbers improves after using the offline model, then the data will be distributed. If the performance does not improve, the system will revert to the previous version. If the live streaming of the digital human does not improve after three consecutive offline training sessions, the administrator will be notified.

[0058] like Figure 2 The diagram illustrates the technical effects of a method for simultaneous live streaming across multiple live streaming rooms based on AI digital humans. This invention demonstrates significant improvements over existing technologies in key performance dimensions, including enhanced consistency across nodes, perceived latency, business conversion, auditability, and bandwidth utilization in multi-live streaming. The black areas represent the technical effects of this invention, while the gray areas represent the technical effects within the technical background.

[0059] Example 2, based on Example 1, is a method for simultaneous live streaming in multiple live rooms using AI digital humans, specifically including the following scheme:

[0060] In step one, edge rendering nodes are deployed in layers and seeds are initialized.

[0061] Specifically, the steps for deploying edge rendering nodes in a layered manner are as follows: by connecting edge rendering nodes and converging them to the regional coordinator according to region, a layered edge topology structure of the region is obtained; the regional coordinator divides the connected edge rendering nodes according to region, and after the division is completed, the connected edge rendering nodes are registered to obtain a management structure that can respond quickly.

[0062] The specific steps of seed initialization are as follows: The central orchestrator reads the session identifier, script version number, resource package identifier and timestamp from the script resource package and calculates the security hash, and writes the rendering seed signature, seed metadata and initial node list into the lightweight distributed event ledger, thereby obtaining the signed seed baseline and controlled node set for cross-node deterministic rendering.

[0063] Step two: Real-time edge deterministic rendering and local optimization are performed during the live broadcast of the digital human;

[0064] Specifically, the steps for real-time edge deterministic rendering and local optimization during digital human live streaming are as follows: Frames are generated using the rendering seed and deterministic rendering function of the edge nodes, and hashes and hash chains are calculated. The rendering seed, deterministic rendering function, generated frames, and hash chains of the edge nodes are written into the regional ledger, allowing for rapid verification of rendering records and data backtracking and querying. The regional coordinator issues local compensation commands through a hybrid decision-making process combining prior rules and a lightweight online prediction model, and writes the commands and effects back to the ledger to obtain closed-loop feedback. This reduces perceived latency and improves performance while ensuring consistency and auditability. Stability; the aforementioned advance rule triggers buffering according to a delay threshold; the aforementioned online prediction model is a small neural network that predicts the latency and asynchrony during future digital human live streams based on existing real-time received data, thereby providing predictive input to the regional regulator, switching compensation actions of pre-rendered keyframes to reduce the perceived latency for viewers and improve audio-visual synchronization; after issuing compensation actions, the issued commands and the effects achieved after issuing commands are written back to the ledger to obtain closed-loop feedback, thereby reducing perceived latency and improving the stability of simultaneous live streaming of digital humans in multiple live rooms while ensuring consistency and auditability.

[0065] Step 3: After the digital human's live stream ends, a replay verification is performed, and offline supervised training is conducted.

[0066] Specifically, the steps for replay verification and offline supervised training after the digital human live stream ends are as follows: By reading the ledger and recalculating the replay frames, unifying the timestamps, and comparing the frame hash chains item by item, a consistency and difference localization report for the digital human's live streams across multiple rooms is obtained; a training set is formed by combining the anonymized aggregated features obtained from the replay with business tags, and a gradient boosting decision tree is used to train the training set offline to obtain a prediction model; the training summary and model version are written into the ledger as an audit record; if the prediction model enables the digital human to meet the preset acceptance threshold in low-traffic live streams, it is issued; otherwise, it is rolled back or the administrator is notified for review; the anonymized aggregated features include: user interaction and session logs, device metrics, and network performance data; the prediction model uses the anonymized aggregated feature data to make real-time predictions of end-to-end latency estimates and audio-visual asynchrony probabilities within a short future time window, thereby outputting suggested compensation amounts to provide suggested support for the regional coordinator's real-time scheduling decisions.

[0067] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for simultaneous live streaming across multiple live streaming rooms based on AI digital humans, characterized in that: Deploy system templates; build audience profiles by collecting user information; build script templates based on the products sold in the live stream and the live stream content; render the seed, specifically: the central orchestrator reads the session identifier, script version number and timestamp from the script resource package, concatenates the read data in the order of reading, and performs a secure hash calculation after concatenation to obtain the rendering seed; A seed package is constructed by packaging the rendering seed and seed metadata; a signature information is obtained by digitally signing the seed package, and the signature information is attached to the seed package to obtain a seed package containing the signature; By calling the write interface of the lightweight distributed event ledger, the seed package is written as the baseline record into the lightweight distributed event ledger; Edge rendering nodes receive a signed seed packet from the central orchestrator and verify its validity using a public key; Under the same software version conditions, generate the same frame data, and combine the generated frame data into a set to obtain a deterministic frame sequence; Edge rendering nodes write the frame hash and hash chain value of each frame into the lightweight distributed event ledger by calling the ledger interface. The central orchestrator compares the frame hash and hash chain value written by at least two edge rendering nodes under the same rendering seed and determines whether they are consistent. The lightweight distributed event ledger is then initialized. During live streaming, edge deterministic rendering is performed by querying signed rendering seeds through the central orchestrator, verifying them, and labeling them with a trust tag. Edge nodes render the current frame by calling a deterministic rendering function to render the current seed; the central script generates the text and speech events corresponding to the current frame to obtain the audio stream and phoneme sequence; and the digital human's lip movements are aligned with the audio frame by applying dynamic time warping to the audio stream and phoneme sequence. The aligned rendering result is obtained by fine-tuning the output of the deterministic rendering function based on the alignment result; a distributed ledger is used for recording, and sorting is performed according to the lexicographical priority of the keys by defining sorting keys, and key interactions must carry a unique transaction identifier. To construct a policy gradient formula for local perception optimization, specifically: edge nodes exchange timestamps with other reference nodes, and then subtract the obtained timestamp from the local standard time. The difference in error is within... The time within is normal, exceeding it is not. If a value is marked as delayed, it is then marked as local delay. The local delay is optimized by constructing a policy gradient update formula, which is: ， It is the action function value, representing the value at the time of action. Actions used in a state The action function value, D is the action function value. It is the state at time g, and e is the state. The action is executed at time g, where f is the action and g represents the time index. It is an assignment operation. For learning rate, In order to complete the action The timely reward received later, where h represents the timely reward. The discount factor is used to measure the present value of future returns. To find the next state All possible actions peak value To perform the action The next state that is reached later. For all possible actions; Replay verification is performed based on records and frame records from the distributed ledger; business effects are quantified and offline model training formulas are constructed for offline training.

2. The method for simultaneous live streaming in multiple live rooms based on AI digital humans according to claim 1, characterized in that: The specific steps for building a script template based on the products sold in the live stream and the live stream content are as follows: Based on the products sold and the content of the live stream, determine the process nodes and compliance requirements of the live stream, and obtain the writing specifications and constraints of the script template; write the global script according to the writing specifications of the script template, and obtain the main script text covering the live stream process. By extracting localizable paragraphs into fragments and setting parameter placeholders, a variable fragment template is obtained; by merging the variable fragment template, localized thesaurus, greetings, and discount expressions, and comparing and correcting them with the compliance thesaurus, a localized dictionary for replacement is obtained; by marking the replacement of the main script text and the variable fragment template, compliance opinions are obtained; by incorporating the global script, variable fragments, localized dictionary, and compliance opinions into the template repository and implementing version control, a script resource package that can be distributed and traced back is obtained; by distributing the script resource package to the edge testing environment and performing replay simulation and item verification, a verification report of the script rendering and replacement logic is obtained.

3. The method for simultaneous live streaming in multiple live rooms based on AI digital humans according to claim 1, characterized in that: The specific steps for recording using a distributed ledger are as follows: By defining sorting keys and sorting them according to lexicographical priority, an event ordering rule for cross-node comparison is obtained; by stipulating that key interactions must carry a unique transaction identifier, a convention for transaction idempotency and traceability is obtained; by digitally signing the detected interaction events with its own private key to obtain signature information, and by calling the Lightweight Ledger interface to submit a write request, evidence of the write request submission is obtained. The ledger node confirms the write request through the authorized consensus mechanism selected by the ledger node and returns a confirmation credential. The edge node obtains the credential that the write request has taken effect by receiving the returned confirmation credential.

4. The method for simultaneous live streaming in multiple live rooms based on AI digital humans according to claim 1, characterized in that: The specific steps for replay verification based on the records and frame records of the distributed ledger are as follows: By calling the lightweight ledger query interface to read the seed resource package, frame record set, and interaction event record set within the session scope, the ledger records required for playback are obtained; by performing public key verification on the signature field of the ledger records and verifying the signature validity of each ledger record, the signature verification result is obtained; by verifying the resource package list and version number recorded in the ledger and calling the corresponding resource package from the resource repository, the data set used for recalculation rendering is obtained; by verifying and signing the rendering seeds in the ledger, a trusted rendering seed is obtained. Under the same software version and resource set conditions, the rendering seed and frame number are recalculated using a deterministic rendering function to obtain a recalculated frame; By calculating the hash value of the recalculated frame, and using Using the initial seed, a hash is calculated for each recalculated frame, and the recalculated hash chain is constructed sequentially according to the recursive relationship to obtain a hash chain used for item-by-item comparison and replay verification. By reading the local timestamps of each record in the ledger and mapping each local time to a unified reference clock using a clock offset correction method, a unified timeline for playback is obtained. By synthesizing the recalculated frame sequence with the corresponding audio stream according to the unified timeline and generating a playback media file at a predetermined playback rate, playback material that can be automatically reviewed is obtained.

5. The method for simultaneous live streaming in multiple live rooms based on AI digital humans according to claim 1, characterized in that: The specific steps for quantifying business results are as follows: By reading transaction records, frame records, and scheduling commands from the lightweight distributed event ledger, the original event sequence of the session organization is obtained; by aligning the original event sequence with the time window of the recalculated frame, sample units aggregated by time window are obtained; by performing desensitization, missing value imputation, and noise reduction on the original samples, a clean dataset for training is obtained; by encoding and normalizing candidate features, the model input feature vector is obtained; by defining a reward signal, the business indicators obtained from playback are used as positive revenue, and the perceived latency and the lack of consistency in actions or audio of the digital human in multiple rooms during live broadcasts are used as penalty terms, thus obtaining the target value required for supervised learning; by defining key business indicators, a quantitative measurement standard is obtained; by simulating the candidate strategy in the playback environment of historical records, and by using importance sampling to correct the bias problem of historical strategies, the performance of the corrected historical business is obtained; By calculating various business indicators on the validation set and performing statistical significance tests, we obtain the criteria for determining whether candidate improvements are significant; by linking the evaluation results, playback logs, and ledger evidence and recording them in the audit log, we obtain an auditable chain of evaluation evidence.

Images