Resource scheduling method for real-time interactive digital human and related device

Abstract

The invention provides a resource scheduling method for real-time interactive digital humans and a related device, and relates to the technical field of artificial intelligence such as human-computer interaction, digital humans, end-cloud integration and large models. The method comprises the following steps: receiving real-time interaction requests initiated by different users through different digital human live broadcast room entrances; and for each real-time interaction request, matching and allocating a corresponding backplane video resource from a pre-generated digital human backplane video library to form different digital human backplane videos in the digital human backplane video library to respectively correspond to the digital human showing different actions, the backplane video resources allocated to the at least two different real-time interaction requests are multiplexed from the same digital human backplane video; a corresponding digital human response video is generated based on the backplane video resource allocated for each real-time interaction request. According to the method, the calculation overhead and the resource consumption for independently generating the basic video material for the real-time interaction request initiated by each user are remarkably reduced through cross-request multiplexing of the bottom plate video resources.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image generation, in particular to the technical field of artificial intelligence such as human-computer interaction, digital human, end-cloud integration, and large model, and more particularly to a resource scheduling method and device for real-time interactive digital human, electronic equipment, computer readable storage medium, and computer program product. BACKGROUND

[0002] In the current technical field of digital human interaction, as the application scenarios evolve from one-way content broadcasting to large-scale one-on-one real-time interaction, traditional technical solutions expose serious cost and performance bottlenecks.

[0003] Although mainstream pure generative solutions can achieve personalized interaction, the "single-path exclusive" resource allocation mode leads to the need for the system to independently call complete speech synthesis, lip matching, and video rendering links for each user, resulting in a cost of more than 100 yuan per thousand interactions. In high-concurrency scenarios with tens of millions of daily active users, the computing power demand grows exponentially, causing low resource utilization and limited system throughput, making it difficult to provide large-scale services to a wide range of users. SUMMARY

[0004] Embodiments of the present disclosure provide a resource scheduling method and device for real-time interactive digital human, electronic equipment, computer readable storage medium, and computer program product.

[0005] In a first aspect, embodiments of the present disclosure provide a resource scheduling method for real-time interactive digital human, comprising: receiving real-time interaction requests initiated by different users through different digital human live streaming room entrances respectively; for each real-time interaction request, matching and allocating corresponding digital human baseboard video resources from a pre-generated digital human baseboard video library; wherein different digital human baseboard videos in the digital human baseboard video library respectively correspond to digital humans exhibiting different actions, and the baseboard video resources allocated for at least two different real-time interaction requests are reused from the same digital human baseboard video; generating a corresponding digital human response video based on the baseboard video resources allocated for each real-time interaction request.

[0006] In a second aspect, the embodiments of the present disclosure provide a resource scheduling apparatus for real-time interactive digital person, comprising: a real-time interaction request receiving unit configured to receive real-time interaction requests initiated by different users through different digital person live streaming room entrances respectively; a bottom plate video resource allocating unit configured to match and allocate corresponding bottom plate video resources from a pre-generated digital person bottom plate video library for each real-time interaction request; wherein different digital person bottom plate videos in the digital person bottom plate video library respectively correspond to digital persons performing different actions, and the bottom plate video resources allocated for at least two different real-time interaction requests are multiplexed from the same digital person bottom plate video; and a digital person response video generating unit configured to generate corresponding digital person response videos based on the bottom plate video resources allocated for each real-time interaction request.

[0007] In a third aspect, the embodiments of the present disclosure provide an electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to implement the resource scheduling method for real-time interactive digital person as described in the first aspect when executed.

[0008] In a fourth aspect, the embodiments of the present disclosure provide a non-transitory computer-readable storage medium storing computer instructions for enabling a computer to implement the resource scheduling method for real-time interactive digital person as described in the first aspect when executed.

[0009] In a fifth aspect, the embodiments of the present disclosure provide a computer program product comprising a computer program, which, when executed by a processor, can implement the steps of the resource scheduling method for real-time interactive digital person as described in the first aspect.

[0010] The resource scheduling scheme for real-time interactive digital person provided by the present disclosure can significantly reduce the computational overhead and resource consumption for generating basic video materials for each user-initiated real-time interaction request through cross-request multiplexing of bottom plate video resources, and can efficiently process massive concurrent requests while ensuring real-time response, thereby optimizing overall resource utilization and effectively controlling generation cost, and laying a technical foundation for large-scale application of real-time interactive digital person technology to a wide range of users.

[0011] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0012] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings: Figure 1 is an exemplary system architecture to which the present disclosure can be applied; Figure 2 A flowchart of a resource scheduling method for real-time interactive digital humans provided by an embodiment of the present disclosure; Figure 3 A flowchart of a method for determining resource adjustment modes for resource pools provided by an embodiment of the present disclosure; Figure 4 A flowchart of a method for generating digital human response videos provided by an embodiment of the present disclosure; Figure 5 A flowchart of a digital human response video delivery method provided by an embodiment of the present disclosure; Figure 6 A flowchart of a resource scheduling method for real-time interactive digital humans in an application scenario provided by an embodiment of the present disclosure; Figure 7 A structural block diagram of a resource scheduling apparatus for real-time interactive digital humans provided by an embodiment of the present disclosure; Figure 8 A structural diagram of an electronic device suitable for executing a resource scheduling method for real-time interactive digital humans provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0013] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0014] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solutions comply with relevant laws and regulations and do not violate public order and good customs.

[0015] Figure 1 An exemplary system architecture 100 of an embodiment of a resource scheduling method, apparatus, electronic device and computer readable storage medium for real-time interactive digital humans to which the present disclosure can be applied is shown.

[0016] As Figure 1As shown, the system architecture 100 can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0017] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. The terminal devices 101, 102, 103 and the server 105 can be installed with various applications for realizing information communication between them, such as resource scheduling applications, digital human real-time interaction applications, instant messaging applications, etc.

[0018] The terminal devices 101, 102, 103 and the server 105 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-mentioned electronic devices, and can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here.

[0019] The server 105 can provide various services through various built-in applications. Taking a resource scheduling application for implementing a real-time interactive digital human as an example, the server 105 can achieve the following effects when running the resource scheduling application: first, receiving real-time interaction requests initiated by different users using their respective terminal devices 101, 102, 103 through different digital human live streaming room entrances through the network 104; then, for each real-time interaction request, matching and allocating corresponding base video resources from a pre-generated digital human base video library, wherein different digital human base videos in the digital human base video library respectively correspond to digital humans performing different actions, and the base video resources allocated for at least two different real-time interaction requests are reused from the same digital human base video; finally, generating a corresponding digital human response video based on the base video resources allocated for each real-time interaction request.

[0020] It should be noted that the real-time interaction request can be pre-stored in the server 105 locally in various ways in addition to being temporarily acquired from the terminal device 101, 102, 103 through the network 104. Therefore, when the server 105 detects that the local has already stored these data (for example, the stored pending request before starting processing), it can be selected to directly acquire these data from the local, and in this case, the exemplary system architecture 100 can also not include the terminal device 101, 102, 103 and the network 104.

[0021] The resource scheduling method for real-time interactive digital humans provided by subsequent embodiments of the present disclosure is generally executed by the server 105 with strong computing power, more computing resources and resource management capabilities. Correspondingly, the resource scheduling device for real-time interactive digital humans is generally also arranged in the server 105.

[0022] It should be understood that Figure 1 The number of terminal devices, networks and servers in

[0023] Please refer to Figure 2 , Figure 2 A flowchart of a resource scheduling method for real-time interactive digital humans provided by an embodiment of the present disclosure is shown, wherein the flow 200 includes the following steps: Step 201: receiving real-time interaction requests initiated by different users through different digital human live streaming room entrances respectively; This step is intended to receive real-time interaction requests initiated by different users through different digital human live streaming room entrances respectively by the execution subject of the resource scheduling method for real-time interactive digital humans (for example, the server 105 shown in Figure 1 The server 105). Among them, the reception is a proactive and continuous listening process, that is, the above-mentioned execution subject is always ready to capture user intentions from countless independent sessions through the interface service deployed in each live streaming room entrance (different live streaming room entrances can be scattered in different application programs). And these requests are not simple data packets, but carry user identity, entrance channel, interaction timestamp and original, unprocessed voice or text instructions.

[0024] The different digital human live room entrances can be embodied in practice as virtual portals set according to different industries, service types or digital human roles. For example, one entrance can be a "legal consultation digital human", and another can be a "children's education digital human". Each entrance is associated with a set of independent configurations, such as the professional knowledge base to be used by the digital human, the style of voice tone, and the range of action templates allowed to be called. When an interaction request arrives, the above execution subject immediately performs request analysis to extract key information such as user ID and specific entrance identifier. This entrance identifier is crucial as it determines which "bottom video resource pool" will be matched subsequently, and the generation strategy of the reply content. For example, a request from the "tax assistant" entrance will be preferentially directed to the tax regulations knowledge base and generate a reply in a formal tone. Further, the same digital human can also open multiple different live room entrances on different application programs, for example, it can be set on the live interface of an instant interaction application, or it can be set or presented on a video library interface.

[0025] At the same time, in order to cope with the huge flow that rushes in at a moment, the above execution subject can use message queue as a buffer layer to convert disordered and pulsed requests into smooth and orderly data flow, preventing the system from being overwhelmed. Subsequently, according to the load balancing strategy, the request will be distributed to a computing node with corresponding resources and currently relatively idle for processing.

[0026] Step 202: For each real-time interaction request, match and assign the corresponding bottom video resource from the pre-generated digital human bottom video library; On the basis of step 201, this step aims to select the most suitable digital human bottom video from the pre-established digital human bottom video library to constitute the bottom video resource according to the context semantics of each real-time interaction request by the above execution subject, wherein the different digital human bottom videos constituting the digital human bottom video library correspond to digital humans exhibiting different actions, and the bottom video resources assigned to at least two different real-time interaction requests are reused from the same digital human bottom video.

[0027] The technical principle that enables this step is to abstract the digital human's motions and expressions into standardized "components" in advance, which are stored in a library in video form, with each component corresponding to an independent motion unit, such as waving, nodding, or smiling. The execution body analyzes the request content, such as keywords or emotional tendencies, to determine which motion segment should be invoked. For example, when the user asks "Can you introduce yourself?", a base frame video containing a smile and a brief gesture may be matched as the basis for the response. In practice, when the user initiates interaction through the digital human live room entrance, the execution body will immediately start a matching process similar to the query mechanism of a search engine, but targeting video motion segments. That is, the execution body can scan the metadata tags in the base video library in real time, which describe the motion type, emotional intensity, and applicable scenarios of each video segment. The matching algorithm considers the semantics of the request, historical interaction records, and the role setting of the digital human to ensure that the selected video is highly consistent with the current conversation in terms of theme and style.

[0028] The different base videos in the digital human base video library are actually pre-made segments of the digital human in different motion and expression states. Each base video encapsulates a complete visual motion sequence, but it is a "semi-finished product" because it lacks fine mouth movements synchronized with specific voice content, which means that the base video only contains the digital human's static image and preset body movements without binding any specific audio track. This approach is to achieve high flexibility and reusability, as the same base video can be combined with multiple different voice responses to generate seemingly unique response videos. In terms of technical implementation, these base videos have been structurally processed during the production phase, such as adding descriptive labels to each segment, including motion categories, duration, emotional labels, and possible applicable contexts. For example, a "nodding smile" base video can be reused in various positive response scenarios, whether confirming an order or answering a question.

[0029] The key of the technical solution provided in this step is the feature of "multiplexing the same digital human base video", that is, the above-mentioned execution subject intentionally allocates the same base video to two or more different user requests. For example, user A and user B may both receive responses generated based on the same "crossed hands listening" base video, although their questions are completely different. The core advantage of this multiplexing mechanism is that it can distribute the expensive video generation cost to multiple interactions, because generating a high-quality base video requires significant computing power, and only a small amount of synthesis overhead is required when it is reused. In specific operation, the above-mentioned execution subject can maintain an index structure to quickly find a candidate video set according to the request characteristics, and then select the best matching item through a sorting algorithm, thereby breaking through the limitation of the traditional "one person one video" mode and realizing the large-scale use of resources.

[0030] An implementation mode, including but not limited to, can be: according to a preset single-path multi-time slice multiplexing mechanism, matching and allocating corresponding base video resources from the digital human base video library for each real-time interaction request. Specifically, the single-path multi-time slice multiplexing mechanism can be manifested as: analyzing each digital human base video into multiple independent time slices; for each real-time interaction request, one or more are selected from the multiple time slices based on the context semantics of the real-time interaction request, and a matched base video resource is assembled. The analysis operation can include at least one of forward playing, reverse playing, cropping, segmenting, and combining.

[0031] This implementation mode deconstructs and reconstructs a complete base video from the dimension of "timeline", that is, instead of regarding a video as an indivisible whole, it regards it as a sequence of independent and programmable "action frames". The above-mentioned execution subject can decompose a linearly played video into an independent "action dictionary" or "vocabulary" through a preset analysis rule, and each time slice carries a minimum unit of meaningful character action or expression, such as a complete "eyebrow raising" action, a "nodding" action, or a "smiling" action from appearing to converging. This deconstruction will enable the above-mentioned execution subject to "read" and "use" the video in a way that transcends normal time flow, thereby creating almost unlimited expression possibilities within limited resources. The core idea is intelligent retrieval and assembly based on context semantics.

[0032] At the specific practice level, for the "analysis" step, the above execution subject can digitally "dissect" each piece of digital human baseboard video in the warehouse, cutting it into multiple independent time segments according to frames or action nodes. Among them, "forward play" can be understood as the most basic mode, that is, using the segment in the time order of the original video recording, suitable for regular sequential response scenarios; "reverse play" is a clever operation, for example, a "pick up the water cup" action segment, when played in reverse, can cleverly represent "put down the water cup", achieving the advanced effect of simulating two logical actions with one physical resource; "cutting" and "segmentation" allow the execution subject to extract only the most expressive core frames from a long action, or to extract the most representative middle 3 seconds from a 10-second hand waving action, in order to achieve the purpose of simplification and highlighting the key points; the "combination" operation is to assemble the segments processed in the above different ways, like assembling Lego blocks, according to the logic required by the current interaction, to form a new, coherent baseboard video resource. For example, a "slightly tilting head" segment and a "doubtful frown" segment can be spliced together to form a complete "expressing confusion" complex expression.

[0033] Further, the idea of reinforcement learning can also be introduced into the segment selection strategy. Considering that the current matching is largely based on predefined rules and labels, reinforcement learning can establish an intelligent editing agent that takes the user's continuous interaction satisfaction (such as whether the conversation is terminated or whether a good review is given) as a long-term reward signal, and optimizes its decision to choose and assemble from a large number of segments through learning. For example, when the above execution subject finds that the response composed of "Segment A (smile) + Segment B (nod)" can bring longer user retention than "Segment C (simple nod)", it will prefer to use this combination in similar situations in the future. This makes the reuse mechanism evolve from "static, rule-based matching" to "dynamic, profit maximization-based decision-making", thereby systematically improving the effectiveness and appeal of the overall interaction experience.

[0034] Further, a global dynamic video resource library can be built, which not only matches according to the current request, but also predicts the future interaction trend and preloads related time segments that may be used into the cache in advance. Just like a skilled chef who does not start preparing dishes only after the customer orders, but can prepare several most popular soup bases in advance based on experience and customer flow prediction. This prediction mechanism will be more than capable of handling unexpected high-frequency problems, thereby further reducing response delay.

[0035] Step 203: Based on the baseboard video resources allocated for each real-time interaction request, generate the corresponding digital human response video.

[0036] On the basis of step 202, the present step aims to generate a corresponding digital human response video based on the floor video resource allocated for each real-time interaction request by the above-mentioned execution subject, so as to respond to the real-time interaction request issued by the user.

[0037] The essence of the scheme provided by the present step is to dynamically and strongly content-relatedly activate the static digital human floor video, and the core problem solved thereby is how to make a general and voiceless action segment look like thinking and answering a specific question of a user. The key lies in: first, accurate synchronization of voice and lip shape, and second, realizing local update without regenerating the entire picture, that is, an audio stream representing “said content” needs to be aligned and mixed with a video stream representing “who is speaking” on a frame-by-frame basis in the timeline.

[0038] A specific implementation manner can be: aligning and synthesizing the floor video resource allocated for each real-time interaction request with a reply voice to generate a corresponding digital human response video; wherein the reply voice is converted based on a reply text for replying to the inquiry raised by the real-time interaction request. That is, the above-mentioned execution subject can first obtain the floor video resource allocated for the current request in the last step, and the “reply voice” converted by a language model from a “reply text” for the user's question through TTS (Text To Speech), which can be regarded as a perfect action body (floor video) and soul content (reply voice). The next step is to combine the two, and the key operation is alignment and synthesis. That is, by analyzing the “reply voice” to extract the phoneme sequence therein, the floor video is also analyzed to find the image of the mouth region, and then each phoneme is mapped to the corresponding lip shape visual change in real time, and the lips of the digital human in the floor video are driven to move, so that it looks like it is saying the words.

[0039] In order to improve rendering efficiency and shorten generation time, frame-level difference rendering technology can be used, that is, instead of changing the entire video frame, only the pixels in a small area around the mouth are recalculated and drawn, and then the “patch” is perfectly fused into the original floor video frame, and the final smooth video stream is output.

[0040] Further, in the entire process, in order to ensure that the digital human looks like the same person no matter how different floor segments are spliced, face embedding vector locking technology can also be used, which can be understood as a digital DNA for locking core features such as facial features, skin quality, and light and shadow, so as to ensure visual consistency.

[0041] The resource scheduling method for real-time interactive digital humans provided by the embodiments of the present disclosure can significantly reduce the computational overhead and resource consumption of generating basic video materials for each user-initiated real-time interaction request through cross-request multiplexing of the baseboard video resources. In addition, through the dynamic matching and allocation mechanism, a large number of concurrent requests can be efficiently processed while ensuring real-time response, thereby optimizing the overall resource utilization and effectively controlling the generation cost, and laying a technical foundation for the large-scale application of real-time interactive digital human technology to a wide range of users.

[0042] On the basis of the above-mentioned embodiments, each real-time interaction request can also be divided according to the type of digital human it is directed to, to obtain an interaction request set corresponding to each type of digital human. In this case, the solution provided by step 202 can be further improved as follows: using a resource pool previously constructed for each type of digital human, for each real-time interaction request in the interaction request set corresponding to the corresponding type of digital human, matching and allocating the corresponding baseboard video resource from the pre-generated digital human baseboard video library.

[0043] The solution provided by the present embodiment aims to further realize fine grouping management of real-time interaction requests, that is, on the basis of the multiplexing mechanism, it is fully recognized that the resource characteristics, knowledge base and action style relied on by different types of digital humans (for example, serious legal consultants and lively children's education instructors) during interaction may differ significantly, and in order to better develop the strengths of different types of digital humans and avoid resource competition between different types, the present solution adopts a "pooling" strategy. From a technical principle point of view, the present solution aims to divide exclusive "resource territories" for different task types to prevent a busy digital human type from exhausting all shared resources, causing service interruption or quality degradation of other types of digital humans. It essentially sets up a logical boundary to build a highly customized and independently running "virtual studio" or "resource pool" for each type of digital human role. Each resource pool is optimized for the needs of a certain type of digital human, for example, a digital human resource pool for legal consultants may preload more calm and affirmative action clips, while an education instructor's resource pool may contain more encouraging and heuristic expression templates. The advantage of this approach is that it can achieve deep optimization for specific roles, as all resources within the pool, including computing nodes, cached memory, and even GPU computing cores, are adjusted as much as possible to be most suitable for processing this type of interaction. And by directing all multiple entries for the same digital human image to the same resource pool, resource utilization can also be improved.

[0044] At the level of specific practice, the division operation can be implemented through a classifier, i.e., the classifier can automatically classify the real-time interaction request into the corresponding digital human category according to the information in the real-time interaction request (such as the specific live room entry identifier entered by the user), forming a clear "interaction request set"; then when building the resource pool, the composition preference of the bottom plate video library can also be adjusted in combination with the category of the digital human, for example, for the "financial analyst" digital human, a fixed number of high-performance GPU nodes, a dedicated financial knowledge base, and a template library containing actions such as chart analysis and serious listening can be allocated to it, all these elements together form a highly autonomous resource pool that belongs to the "financial analyst". When a request belonging to this set comes, the scheduler will directly schedule in the exclusive resource pool corresponding to this category, which can significantly reduce the complexity of scheduling decisions on the one hand, and also improve the cache hit rate because the resources in the pool are repeatedly used to process similar businesses.

[0045] On the basis of the previous embodiment, the resource adjustment mode for each resource pool can also be determined according to the request characteristics of the real-time interaction requests received by different resource pools in a preset period, and then the resources of each resource pool in the next period are adjusted according to the resource adjustment mode; wherein the resources can include at least one of the maximum channel number, the number of nodes, the shared cache between nodes, GPU resources, and the upper limit of read-write performance. That is, the embodiment aims to realize the transition from static resource allocation to dynamic resource allocation based on data driving, the core of which is no longer to arbitrarily allocate fixed resources to a category of digital humans, but to dynamically adjust according to the actual "workload" and "work efficiency".

[0046] From the technical principle level, the essence of this adjustment mechanism is a closed-loop feedback control system, which no longer assumes that a certain resource pool is always busy or idle, but recognizes that there is an objective tide law in traffic and actively adapts to it. That is, the above-mentioned execution subject takes a preset time period (such as every minute or every five minutes) as the beat, continuously collects the sign data reflecting the running state of each resource pool - request characteristics, and then determines the resource adjustment mode (such as increasing resources and reducing resources) based on the request characteristics, and finally adjusts according to the resource adjustment mode in the next period.

[0047] To deepen the understanding of how to determine the resource adjustment mode according to the request characteristics, please refer to Figure 3 , Figure 3 A flowchart of a method for determining the resource adjustment mode for each resource pool is provided for the embodiments of the present disclosure, wherein the flow 300 includes the following steps: Step 301: Determine the request concurrency intensity and the request distribution of different priorities according to the number of real-time interaction requests received by different resource pools in a preset period; This step aims to focus on analyzing the "traffic characteristics" by the above-mentioned execution subject, that is, to calculate the request concurrency strength which can reflect the size of external pressure by counting the number of requests, and to analyze the request sources or content tags of these requests to determine the distribution of requests of different priorities, which can indicate the nature of the pressure, and then distinguish between regular traffic and urgent requests that need to be processed urgently.

[0048] Step 302: According to the bottom plate video resources allocated by different resource pools to the received real-time interactive requests within a preset period, determine the bottom plate video multiplexing rate; Unlike step 301, this step aims to evaluate the "resource efficiency" by the above-mentioned execution subject, that is, to measure the economy of internal resources by calculating the bottom plate video multiplexing rate. That is, it is necessary to trace all the requests handled by each resource pool within the period, and analyze how many different bottom plate videos these requests ultimately point to. A high multiplexing rate means that the resource pool can serve a large number of requests with very few core assets (bottom plate videos), which is a model of high efficiency; otherwise, it indicates that the resources may be idle or wasted.

[0049] Step 303: According to the request concurrency strength, the distribution of requests of different priorities, and the bottom plate video multiplexing rate of different resource pools within a preset period, determine the resource adjustment mode for each resource pool.

[0050] On the basis of steps 301 and 302, this step aims to combine the traffic characteristics and efficiency characteristics by the above-mentioned execution subject, rather than looking at them in isolation, to form a comprehensive resource health report, and to generate accurate adjustment instructions accordingly. This is essentially an assessment of each resource pool, and a decision on whether to "invest more" (i.e. increase resources) or "cut budget" (i.e. reduce resources) based on the assessment results.

[0051] The scheme provided by this step is equivalent to the core link of the performance adjustment mechanism, which performs a multi-factor weighted decision algorithm. For example, through analysis, it is found that the resource pool of "virtual teacher A" meets both high concurrency strength and high multiplexing rate, which is like a store with both high customer flow and high turnover rate, so it is reasonable to "assign more staff" and "expand the store", that is, to increase the number of GPU nodes and improve the shared cache capacity for it in the next period; while for a resource pool with high concurrency and low multiplexing, it can be concluded that the matching algorithm needs to be optimized, and instead, it is prioritized to allocate higher-quality CPU computing resources to try to improve the matching speed.

[0052] Further, considering that the dynamic adjustment mechanism provided in the above scheme is automatic, but is still a post-response in nature, a proactive resource planning based on time series prediction can also be considered, that is, a machine learning model can be trained to learn the unique "work and rest rules" of each resource pool, for example, for the resource pool corresponding to the "fitness coach" digital person, the morning and evening peak can be predicted using this prediction model one or two cycles before the expected traffic peak, and resources can be gradually injected into the target resource pool in advance.

[0053] To deepen the understanding of how to generate digital human response videos, please refer to Figure 4 , Figure 4 A flowchart of a method for generating digital human response videos provided by an embodiment of the present disclosure, wherein the flow 400 comprises the following steps: Step 401: Obtain the base video resource and the reply voice corresponding to each real-time interaction request; Step 402: Based on the preset phoneme-viseme mapping relationship, adjust the mouth shape change of the mouth area in the base video resource according to the reply voice, and generate the corresponding digital human response video.

[0054] The core principle of the scheme provided in this step is based on the intersection of linguistics and computer vision. Among them, phoneme (Phoneme) is the smallest unit of sound that can distinguish meaning in human language, such as initial and final in Chinese, while viseme (Viseme) is the mouth shape and facial muscle movement pattern corresponding to one or more phonemes in vision.

[0055] This step first performs phoneme recognition on the reply voice, decomposing it into a series of phoneme sequences arranged in time, and then converts each phoneme in real time to a specific, continuous mouth movement trajectory on the digital human face according to the preset phoneme-viseme mapping table. In specific practice, this process can be performed by a specially optimized mouth shape rendering model that receives real-time mouth area images from the base video, but does not redraw the entire face. Instead, it uses frame-level difference rendering technology to accurately and locally modify and cover the image pixels of the lips and their surrounding muscles on the original video frames, thereby generating a digital human response video with synchronized mouth shapes in each frame, in order to make the digital human's speech as realistic and credible as possible.

[0056] On the basis of any of the above embodiments, please refer to Figure 5 , Figure 5 A flowchart of a method for delivering digital human response videos provided by an embodiment of the present disclosure, wherein the flow 500 comprises the following steps: Step 501: Deliver the generated digital human response video to the terminal device used by the corresponding user through the edge node under the content distribution network. The step is to deliver the digital human response video to the terminal device of each user by the content delivery network (CDN) and each edge node constituting the CDN.

[0057] That is, when the above execution subject completes the synthesis of the digital human response video, it does not send it directly to the remote user, but preferentially pushes it to these edge nodes. When the user requests arrive, the system intelligently guides the user to the nearest node that stores the video he needs through DNS resolution and other technologies, thereby shortening the data transmission distance and reducing the probability of network congestion and packet loss, ensuring smooth video streaming and low-latency experience.

[0058] Step 502: Control the edge node to temporarily store the received digital human response video for delivery by the edge node to the terminal device of other users with the same needs.

[0059] On the basis of step 501, the step is to control the edge node to temporarily store the received digital human response video by the above execution subject. That is, the above execution subject instructs the edge node not only to serve the current requesting user, but also to keep the video in its local storage for a period of time.

[0060] The "temporary storage" described in this step is a dynamic and intelligent process, and the storage duration is not fixed but proportional to a key indicator: the number of reuses. It is a feedback loop: the more a video is requested, the more likely it is to be requested again, so it will be kept on the edge node for a longer period of time. In practice, a popularity ranking can even be formed, and the remaining storage time can be adjusted by tracking the request frequency of each video in real time.

[0061] On the basis of the previous embodiment, considering that the current caching strategy is still reactive in nature, i.e., the caching time of a video is extended only after it is frequently requested. Based on this, an intelligent predictive caching and global resource map can be tried, i.e., using machine learning models to analyze user behavior data to predict which questions or content will become hot. For example, after a major news event occurs, it can be predicted that question and answer videos related to the event will be deployed in advance to edge nodes in various regions, and even lightweight transcoding processing can be performed in advance to adapt to different network environments. When the model predicts that the attention of a certain topic (such as a new policy or product release) will rise sharply, it will proactively preheat related content videos to the cache.

[0062] With the wide application of AI digital human technology, the industry is experiencing a major mode transition from "unified content live broadcast" to "massive 1-to-1 real-time interaction". In the traditional 1vN (one digital human playing the same content to all users) live broadcast mode, the system only needs to output one video stream, and all users receive the same video content, with controllable production cost. However, when the interactive scene evolves from "one-way play" to "personalized response and real-time interaction", the input of each user will trigger independent speech synthesis, lip synchronization, expression and action control, and video rendering links. At this time, the system not only faces the challenge of exponential growth of computing power demand, but also needs to generate visually consistent and emotionally natural digital human response content in a very short time, resulting in problems such as skyrocketing interaction cost, low resource utilization, and limited system throughput.

[0063] Taking AI customer service, virtual lecturer, virtual host, interactive live broadcast and other scenarios as examples, the semantic input of users is very different, and the system must have the ability to "thousand faces for a thousand people", while also completing the response within milliseconds of delay. If the traditional "single-path exclusive" generation scheme is continued, a complete generation channel (including speech driving, video generation and rendering resources) must be allocated independently for each user, which will directly lead to uncontrollable cost and limited concurrency capability, severely restricting the landing of digital human technology in large-scale real-time interaction scenarios.

[0064] To solve the above problems, the embodiment proposes a digital human video efficient generation and real-time response technology scheme based on multiplexing and dynamic scheduling suitable for large-scale user concurrency scenarios, to cope with the computing power pressure and cost challenges brought by the full-interactive mode of "one person one video".

[0065] That is, the embodiment constructs a multiplexing and scheduling mechanism for digital human content generation and playback, based on the three-layer core mechanism of "baseboard video + polymorphic multiplexing + dynamic scheduling", through structural optimization in the generation, rendering and scheduling three links, to realize the maximum utilization and real-time distribution of video resources. The following describes each core mechanism in detail (see Figure 6 the schematic diagram): I. Single-path multi-time slice multiplexing mechanism After generating the digital human baseboard video, the system disassembles it into multiple time slices, and differentiates the local combination based on context semantics, action flow and expression template, so as to realize "one video" multi-period and multi-user multiplexing without repeating the generation of baseboard materials.

[0066] For example, a standardized green screen digital human baseboard video (such as 5-second smile and nod action) can be used as a welcome action in user A's conversation and as a response action in user B's Q&A scenario, realizing multi-time and space multiplexing of the same video segment, and significantly reducing the cost of single generation.

[0067] II. Multiplexing and tidal dynamic allocation mechanism

[0068] To cope with high concurrency access, the embodiment splits the digital human video generation and rendering task into multiple service nodes to build a "resource group pool". The system can dynamically adjust the occupation and release of different channels according to current concurrency intensity, task priority, content multiplexing rate and other indicators, using a "tidal lane" like allocation logic, so that GPU, cache and I / O resources can be allocated on demand.

[0069] During the low concurrency period, the system will concentrate tasks in a high multiplexing node to save energy; during the peak period, the node will be quickly expanded and the board cache will be shared to maximize resource utilization and dynamically elastic scaling.

[0070] III. Rendering and distribution optimization combined with live broadcast middleware

[0071] Through deep integration with the live broadcast middleware technology stack, the application introduces cache acceleration and frame-level multiplexing strategies at the transmission layer and rendering layer. After the board video cache is stored in the edge node, user request response can be completed within milliseconds, and based on the phoneme-viseme (Phoneme-Viseme) driving relationship, dynamic superposition of voice and lip shape is realized to achieve high-synchronization real-time interaction. Frame-level difference coding mechanism further reduces repeated rendering and reduces bandwidth consumption.

[0072] The core innovation of the scheme provided by the embodiment is to deeply integrate the content layer multiplexing of the video generation model with the system layer computing power scheduling, realizing the architecture transformation from "generative independent computing" to "multiplexing dynamic sharing". The main innovations include: 1) Building a "single multi-time slice multiplexing" mechanism, breaking the limitation of "one video for one task", greatly improving the utilization rate of the board material; 2) Proposing a "tidal resource dynamic allocation" strategy, making GPU and cache resources allocated on demand among multiple tasks, improving system flexibility and concurrency throughput capacity; 3) Through the board cache and frame-level difference coding mechanism, low-latency high-synchronization real-time interactive experience is realized; 4) Forming a standardized architecture that can seamlessly connect with video generation large models, speech synthesis systems and live broadcast middleware, with scalability and industry generality.

[0073] In summary, the scheme provided by the embodiment realizes the capability transition of the digital person from "pre-generated content" to "real-time generated content" by mechanizing and systematizing the reuse mechanism of the bottom plate video and rendering computing resources on the basis of ensuring the interaction personalization and response real-time. Therefore, the scheme provided by the embodiment can be applied to scenarios such as virtual customer service, digital person explanation, intelligent question and answer, AI emotional accompaniment, virtual anchor interaction, e-commerce live broadcast, enterprise training, and the like. In these applications, the embodiment can realize personalized interaction of "thousand faces for a thousand people", significantly reduce system computing power and video generation cost, make the real-time interactive digital person have the feasibility of large-scale deployment, and promote the comprehensive evolution of the digital person technology from "content playing type" to "real-time service type".

[0074] Further referring to Figure 7 , as an implementation of the method shown in the above figures, the disclosure provides an embodiment of a resource scheduling apparatus for a real-time interactive digital person. The apparatus embodiment corresponds to the method embodiment shown in Figure 2 , and the apparatus can be specifically applied to various electronic devices.

[0075] As shown in Figure 7 , the resource scheduling apparatus 700 for a real-time interactive digital person of the embodiment can include a real-time interaction request receiving unit 701, a bottom plate video resource allocation unit 702, and a digital person response video generation unit 703. The real-time interaction request receiving unit 701 is configured to receive real-time interaction requests initiated by different users through different digital person live broadcast room entrances respectively. The bottom plate video resource allocation unit 702 is configured to match and allocate corresponding bottom plate video resources from a pre-generated digital person bottom plate video library for each real-time interaction request. Different digital person bottom plate videos in the digital person bottom plate video library respectively correspond to digital persons performing different actions, and the bottom plate video resources allocated for at least two different real-time interaction requests are reused from the same digital person bottom plate video. The digital person response video generation unit 703 is configured to generate a corresponding digital person response video based on the bottom plate video resources allocated for each real-time interaction request.

[0076] In the embodiment, in the resource scheduling apparatus 700 for a real-time interactive digital person: the specific processing of the real-time interaction request receiving unit 701, the bottom plate video resource allocation unit 702, and the digital person response video generation unit 703 and the technical effects brought by the specific processing can be respectively referred to the related descriptions of steps 201-203 in the corresponding embodiment, which will not be described here again. Figure 2 The corresponding embodiment corresponds to the related description of steps 201-203, which will not be described here again.

[0077] In some other optional implementation manners of the embodiment, the bottom plate video resource allocation unit 702 includes: The single-path multi-time slice multiplexing subunit is configured to match and assign corresponding digital human baseboard video resources from the digital human baseboard video library for each real-time interaction request according to a preset single-path multi-time slice multiplexing mechanism.

[0078] In some other optional implementations of the embodiment, the single-path multi-time slice multiplexing subunit includes: The parsing module is configured to parse each digital human baseboard video into a plurality of independent time slices; The semantic selection and assembly module is configured to, for each real-time interaction request, select one or more from the plurality of time slices based on the context semantics of the real-time interaction request, and assemble to obtain a matched baseboard video resource.

[0079] In some other optional implementations of the embodiment, the parsing module is further configured to: parse each digital human baseboard video according to at least one of the following preset parsing processing operations to obtain a plurality of independent time slices: forward playback, reverse playback, cropping, segment interception, and combination.

[0080] In some other optional implementations of the embodiment, the resource scheduling apparatus 700 for real-time interactive digital humans can further include: The digital human division unit is configured to divide each real-time interaction request according to the type of digital human it is directed to, to obtain an interaction request set corresponding to each type of digital human; Correspondingly, the baseboard video resource allocation unit 702 is further configured to: match and assign corresponding baseboard video resources from the pre-generated digital human baseboard video for each real-time interaction request in the interaction request set corresponding to each type of digital human, using a resource pool corresponding to each type of digital human constructed in advance; wherein the number of resource pools is the same as the number of types of digital humans.

[0081] In some other optional implementations of the embodiment, the resource scheduling apparatus 700 for real-time interactive digital humans can further include: The resource adjustment mode determination unit is configured to determine the resource adjustment mode for each resource pool according to the request characteristics of real-time interaction requests received by different resource pools within a preset period; The resource adjustment unit is configured to adjust the resources of each resource pool in the next period according to the resource adjustment mode; wherein the resources include at least one of the maximum number of channels pointing to the resource pool, the number of nodes, the shared cache between nodes, GPU resources, and the upper limit of read-write performance.

[0082] In some other optional implementations of the embodiment, the resource adjustment mode determination unit is further configured to: Based on the number of real-time interactive requests received by different resource pools within a preset period, the request concurrency intensity and the distribution of requests with different priorities are determined. The baseboard video reuse rate is determined based on the baseboard video resources allocated by different resource pools to the received real-time interactive requests within a preset period. Based on the concurrent request intensity of different resource pools, the distribution of requests with different priorities, and the reuse rate of baseboard video within a preset period, the resource adjustment method for each resource pool is determined.

[0083] In some other optional implementations of this embodiment, the digital human response video generation unit 703 includes: The audio-visual alignment subunit is configured to perform alignment and synthesis processing on the background video resources allocated for each real-time interactive request and the response speech to generate the corresponding digital human response video; wherein, the response speech is obtained by converting the response text used to answer the questions raised in the real-time interactive request.

[0084] In some other optional implementations of this embodiment, the audio-visual alignment subunit is further configured as follows: Acquire the baseboard video resources and response voice corresponding to each real-time interaction request; Based on the preset phoneme-visual pixel mapping relationship, the mouth shape changes in the mouth area of ​​the base plate video resource are adjusted according to the response voice to generate the corresponding digital human response video.

[0085] In some other optional implementations of this embodiment, the resource scheduling device 700 for real-time interactive digital humans may further include: The distribution unit is configured to distribute the generated digital human response video to the terminal device used by the corresponding user through the edge node of the content delivery network. The edge node temporary storage control unit is configured to control the edge node to temporarily store the digital human response video received in transit, so that the edge node can directly send the digital human response video to the terminal devices used by other users with the same needs; wherein, the storage duration of the digital human response video stored in the edge node is proportional to the number of times it is reused.

[0086] This embodiment exists as a device embodiment corresponding to the above method embodiment. The resource scheduling device for real-time interactive digital humans provided in this embodiment firstly reduces the computational overhead and resource consumption of generating basic video materials separately for each user's real-time interactive request by reusing the baseboard video resources across requests. Secondly, through the dynamic matching and allocation mechanism, it can efficiently handle massive concurrent requests while ensuring real-time response, thus optimizing the overall resource utilization and effectively controlling the generation cost, thereby laying the technical foundation for the large-scale application of real-time interactive digital human technology to a wide range of users.

[0087] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the resource scheduling method for real-time interactive digital humans described in any of the above embodiments.

[0088] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the resource scheduling method for real-time interactive digital humans described in any of the above embodiments when executed.

[0089] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the resource scheduling method for real-time interactive digital humans described in any of the above embodiments.

[0090] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0091] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0092] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0093] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as a resource scheduling method for a real-time interactive digital human. For example, in some embodiments, the resource scheduling method for a real-time interactive digital human can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the resource scheduling method for a real-time interactive digital human described above can be performed. Alternatively, in other embodiments, computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a resource scheduling method for real-time interactive digital humans.

[0094] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0095] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0096] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0098] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0099] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0100] The beneficial effects of the technical solutions according to the embodiments of this disclosure are repeated.

[0101] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0102] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A resource scheduling method for real-time interactive digital humans, comprising: receiving real-time interaction requests initiated by different users through different digital human live streaming room entrances respectively; for each of the real-time interaction requests, matching and assigning a corresponding baseboard video resource from a pre-generated digital human baseboard video library; wherein different digital human baseboard videos constituting the digital human baseboard video library respectively correspond to digital humans exhibiting different actions, and the baseboard video resources assigned to at least two different real-time interaction requests are multiplexed from the same digital human baseboard video; generating a corresponding digital human response video based on the baseboard video resource assigned to each of the real-time interaction requests.

2. The method of claim 1, wherein, The method further comprises: for each of the real-time interaction requests, matching and assigning a corresponding baseboard video resource from the digital human baseboard video library according to a preset single-path multi-time slice multiplexing mechanism.

3. The method of claim 2, wherein, The method further comprises: analyzing each of the digital human baseboard videos into a plurality of independent time slices; for each of the real-time interaction requests, selecting one or more time slices from the plurality of time slices based on the context semantics of the real-time interaction request to assemble a matched baseboard video resource.

4. The method of claim 3, wherein, The method further comprises: analyzing each of the digital human baseboard videos according to at least one of the following preset analysis processing operations to obtain a plurality of independent time slices: forward playback, reverse playback, cropping, segment interception, and combination.

5. The method of claim 1, further comprising: dividing each of the real-time interaction requests according to the type of digital human to which it is directed to obtain an interaction request set corresponding to each type of digital human; correspondingly, the method further comprises: for each real-time interaction request in the interaction request set corresponding to each type of digital human, matching and assigning a corresponding baseboard video resource from the pre-generated digital human baseboard video library using a resource pool pre-built for each type of digital human; wherein the number of resource pools is the same as the number of types of digital humans.

6. The method of claim 5, further comprising: determining a resource adjustment method for each of the resource pools according to the request characteristics of the real-time interaction requests received by the resource pool within a preset period; adjusting the resources of each of the resource pools for the next period according to the resource adjustment method; wherein the resources include at least one of the maximum number of channels, the number of nodes, the shared cache between nodes, GPU resources, and the upper limit of read-write performance.

7. The method of claim 6, wherein, The method further comprises: determining a resource adjustment method for each of the resource pools according to the request characteristics of the real-time interaction requests received by the resource pool within a preset period. determine request concurrency intensity and request distribution of different priorities according to the number of real-time interaction requests received by different resource pools within the preset period; determine board video multiplexing rate according to the board video resources allocated by different resource pools to the received real-time interaction requests within the preset period; determine resource adjustment mode for each resource pool according to request concurrency intensity, request distribution of different priorities and board video multiplexing rate of different resource pools within the preset period.

8. The method of claim 1, wherein, generate a corresponding digital human response video based on the board video resources allocated for each real-time interaction request, including: align and synthesize the board video resources allocated for each real-time interaction request with the reply voice to generate a corresponding digital human response video; wherein the reply voice is converted based on the reply text for replying to the inquiry raised by the real-time interaction request.

9. The method of claim 8, wherein, align and synthesize the board video resources allocated for each real-time interaction request with the voice generated according to the request reply text to generate a corresponding digital human response video, including: obtain the board video resources and the reply voice corresponding to each real-time interaction request; adjust the mouth shape change of the mouth area in the board video resources according to the reply voice based on the preset phoneme-visual mapping relationship to generate a corresponding digital human response video.

10. The method of any one of claims 1-9, further comprising: distributing the generated digital human response video to the terminal device used by the corresponding user through the edge node under the content distribution network; controlling the edge node to temporarily store the received digital human response video for direct distribution of the digital human response video to the terminal device used by other users with the same demand by the edge node; wherein the storage duration of the digital human response video temporarily stored in the edge node is proportional to the number of multiplexing times.

11. A resource scheduling device for real-time interactive digital human, comprising: a real-time interaction request receiving unit configured to receive real-time interaction requests initiated by different users through different digital human live streaming room entrances respectively; a board video resource allocation unit configured to match and allocate corresponding board video resources from a pre-generated digital human board video library for each real-time interaction request; wherein different digital human board videos constituting the digital human board video library respectively correspond to digital humans exhibiting different actions, and the board video resources allocated for at least two different real-time interaction requests are multiplexed from the same digital human board video; a digital human response video generation unit configured to generate a corresponding digital human response video based on the board video resources allocated for each real-time interaction request.

12. An electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the resource scheduling method for real-time interactive digital human of any one of claims 1-10.

13. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the resource scheduling method for real-time interactive digital human of any one of claims 1-10.

14. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the resource scheduling method for real-time interactive digital human of any one of claims 1-10.

Images