Digital human video generation method and device for high concurrency call, and medium

Abstract

The application discloses a high-concurrency calling digital human video generation method and device, and a medium. The method comprises the following steps: in response to a user request, sending a task request corresponding to the user request to a task queue of a task scheduler through an API gateway; taking a computing node with an idle working state in a distributed computing resource pool as an idle node, and actively pulling the task request from the task queue through the idle node; performing inference on the task request through the idle node, updating a task state of the task request to a completed state through the task scheduler, and returning a query response through the API gateway. The embodiment of the application can meet the demand of a high-concurrency scene.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of data processing technology, and in particular to a method, device, and medium for generating digital human videos with high concurrency. Background Technology

[0002] With the rise of applications such as metaverse, virtual anchors, and digital employees, the demand for high-quality, real-time interactive digital human video generation is increasing. This application scenario requires the system to be able to handle massive requests from a large number of users simultaneously, generating lip-synced, naturally performing digital human videos. Therefore, the backend system architecture supporting digital human generation, its concurrent processing capabilities, response speed, and resource utilization have become key technical indicators determining user experience and commercial viability.

[0003] Currently, the industry has numerous technical models in the field of digital human video generation, especially in lip-sync generation, where models such as Wav2Lip and HeyGem have achieved significant results. However, existing technical models mainly focus on improving the quality and accuracy of a single generation task. At the system deployment level, the most common implementation is a monolithic service architecture. Specifically, as shown in the publicly available official Wav2Lip implementation codebase, it typically exists as a Python or simple web service (such as using Flask or FastAPI frameworks). This service is deployed on a single server equipped with a GPU, receives uploaded audio and video files, sequentially calls the model for inference, and finally returns the generated result.

[0004] However, existing solutions generally adopt a synchronous blocking request-response model. Since digital human generation is a computationally intensive long task, a single task can completely occupy GPU resources for tens of seconds or even minutes, causing the service process to be completely blocked during task execution. The concurrency capability is essentially limited to "1" (or equal to the number of GPUs). When faced with a large number of instantaneous requests, the system throughput is almost zero, which cannot meet the basic requirements of high concurrency scenarios. Summary of the Invention

[0005] This application provides a method, device, and medium for generating digital human videos with high concurrency, which can meet the needs of high-concurrency scenarios.

[0006] In a first aspect, embodiments of this application provide a method for generating digital human videos with high concurrency calls, applied to a digital human video generation system, the digital human video generation system including an API gateway, a task scheduler, and a distributed computing resource pool, the method including: In response to a user request, the task request corresponding to the user request is sent to the task queue of the task scheduler through the API gateway; The idle computing nodes in the distributed computing resource pool are designated as idle nodes, and the task requests are actively pulled from the task queue by the idle nodes. The idle node performs inference on the task request, updates the task status of the task request to the completed status through the task scheduler, and then returns a query response through the API gateway.

[0007] Secondly, an electronic device provided according to an embodiment of this application includes: At least one processor; At least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the digital human video generation method with high concurrency calls as described in any of the first aspects is implemented.

[0008] Thirdly, according to the embodiments of the application, a computer-readable storage medium is provided, storing computer-executable instructions, which are used to execute the digital human video generation method that implements any of the high-concurrency calls described in the first aspect.

[0009] In summary, the high-concurrency digital human video generation method of the above embodiments of this application includes: responding to a user request, sending the task request corresponding to the user request to the task queue of the task scheduler through the API gateway; designating idle computing nodes in the distributed computing resource pool as idle nodes, and actively pulling task requests from the task queue through the idle nodes; performing inference on the task request through the idle nodes, updating the task status of the task request to the completed status through the task scheduler, and then returning a query response through the API gateway. By employing an asynchronous decoupling mechanism between the API gateway and the task scheduler, the reception of user requests and the computationally intensive, long-running task of digital human video generation are completely separated in time. This allows massive requests in high-concurrency scenarios to be responded to quickly and buffered in the task queue, avoiding direct impact on backend computing resources. Furthermore, idle nodes actively pull tasks from the task queue, achieving precise load balancing based on their actual load, avoiding allocation errors caused by state lag in centralized dispatching. Moreover, adding or removing nodes does not require modifying the scheduling logic; processing capacity scales linearly with the number of nodes, and unfinished tasks on faulty nodes can be automatically taken over by other nodes, inherently possessing elastic scaling and fault recovery capabilities. Simultaneously, the entire task processing process uses task identifiers as indexes for status tracking, allowing users to query the complete progress of tasks from pending to processing to completion in real time, ultimately obtaining the generated results through the API gateway. This ensures high-concurrency processing capabilities while maintaining user transparency and system maintainability. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the steps of a high-concurrency digital human video generation method according to an embodiment of this application; Figure 2 This application provides an embodiment of a digital human video generation system architecture diagram; Figure 3 This is a hardware schematic diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] It is understandable that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0013] With the rise of applications such as metaverse, virtual anchors, and digital employees, the demand for high-quality, real-time interactive digital human video generation is increasing. This application scenario requires the system to be able to handle massive requests from a large number of users simultaneously, generating lip-synced, naturally performing digital human videos. Therefore, the backend system architecture supporting digital human generation, its concurrent processing capabilities, response speed, and resource utilization have become key technical indicators determining user experience and commercial viability.

[0014] Currently, the industry has numerous technical models in the field of digital human video generation, especially in lip-sync generation, where models such as Wav2Lip and HeyGem have achieved significant results. However, existing technical models mainly focus on improving the quality and accuracy of a single generation task. At the system deployment level, the most common implementation is a monolithic service architecture. Specifically, as shown in the publicly available official Wav2Lip implementation codebase, it typically exists as a Python or simple web service (such as using Flask or FastAPI frameworks). This service is deployed on a single server equipped with a GPU, receives uploaded audio and video files, sequentially calls the model for inference, and finally returns the generated result.

[0015] However, existing solutions generally adopt a synchronous blocking request-response model. Since digital human generation is a computationally intensive long task, a single task can completely occupy GPU resources for tens of seconds or even minutes, causing the service process to be completely blocked during task execution. The concurrency capability is essentially limited to "1" (or equal to the number of GPUs). When faced with a large number of instantaneous requests, the system throughput is almost zero, which cannot meet the basic requirements of high concurrency scenarios.

[0016] Based on this, embodiments of this application provide a method, device, and medium for generating digital human videos with high concurrency, which can meet the needs of high-concurrency scenarios.

[0017] This application provides a high-concurrency digital human video generation method, applied to a digital human video generation system. The digital human video generation system includes an API gateway, a task scheduler, and a distributed computing resource pool. (Refer to...) Figure 1 and Figure 2 As shown, the method includes, but is not limited to, the following steps: Step S100: In response to the user request, the task request corresponding to the user request is sent to the task queue of the task scheduler through the API gateway.

[0018] For example, when a user initiates a digital human video generation request (e.g., uploading a driver audio file and a target person video file) through a front-end page or other upstream services, the API gateway of the digital human video generation system, as the unified entry point of the system, receives the user request. Then, the API gateway can directly send the task request corresponding to the user request to the task scheduler, or it can preprocess the user request and then send it to the task scheduler. It then returns a response containing the task identifier to the user, informing the user that the task has been successfully received, but does not return the generation result at this time.

[0019] For example, preprocessing includes performing necessary parameter validation on the request (such as verifying whether the file format and file size meet the requirements), generating a unique task identifier (task_id), initializing the task status to a pending state, and sending the task request along with the task identifier to the task queue of the task scheduler.

[0020] Step S110: Idle computing nodes in the distributed computing resource pool are designated as idle nodes, and task requests are actively pulled from the task queue by the idle nodes.

[0021] For example, an idle node actively pulls a task request from the task queue and updates the task status of the request to "processing".

[0022] Step S120: Perform inference on the task request through the idle node, update the task status of the task request to the completed status through the task scheduler, and then return the query response through the API gateway.

[0023] For example, after receiving a task request, the idle node decodes and preprocesses the input file corresponding to the task request; it then calls the lip-sync model integrated within the idle node (such as the Wav2Lip model or the HeyGem model) to perform lip-sync inference; and finally, it encodes the digital human video output from the inference. Once the idle node completes the entire inference process, it reports the task completion status to the task scheduler. Upon receiving the report, the task scheduler updates the task status of the task request in the task queue to the completed status. Users or their upstream services can initiate a task status query request to the API gateway by carrying a task identifier (task_id). The API gateway retrieves the latest task status corresponding to the task identifier from the task scheduler and returns a query response to the user. When the task status is completed, the query response further includes the method for obtaining the generated digital human video file (e.g., a download link), allowing the user to obtain the final digital human video result.

[0024] For example, in this embodiment of the application, the API gateway serves as the unified entry point of the system, adopts a RESTful API design, and provides two core interfaces: / api / task (for submitting tasks) and / api / task / {task_id} (for querying task status). When the API gateway receives a task, it does not process the task directly, but sets the task to a "pending" state, puts it into a persistent task queue, and hands it over to the backend system for processing.

[0025] The task scheduler in this embodiment is responsible for the service registration and discovery of idle nodes, and stores the static metadata of the nodes (such as IP address, supported model type, GPU model, video memory size, etc.).

[0026] In this embodiment, the persistent task queue (Redis) of the task scheduler serves as a persistent and globally ordered task buffer for the system, responsible for storing all pending task messages. Its role is that of a reliable storage intermediary, without undertaking complex scheduling logic. A cluster of multiple GPU-equipped servers is managed using containerization technology (Docker). Each container instance is an intelligent computing node, internally encapsulating a Wav2Lip or HeyGem model and integrating the following quality and efficiency optimization pipeline. The node is enhanced with autonomous decision-making capabilities, proactively pulling tasks from the persistent task queue that match its own configuration (such as model type), achieving elastic task allocation. The node provides an HTTP API for status reporting and external control.

[0027] This application embodiment receives user requests instantly via an API gateway and quickly returns task identifiers. The task scheduler queues and buffers task requests, allowing user requests to receive responses without waiting for video generation to complete. This avoids connection timeouts caused by prolonged processing. Massive requests with high concurrency are buffered in the task scheduler's task queue instead of directly impacting computing resources, achieving traffic smoothing and ensuring system stability in high-concurrency scenarios. The task scheduler monitors the working status of each computing node in the distributed computing resource pool in real time based on a heartbeat detection mechanism. It adopts a task acquisition strategy where idle nodes actively pull tasks, ensuring full utilization of all available computing power in the cluster. Multiple idle nodes can execute different task requests in parallel, fundamentally breaking through the bottleneck of concurrency limited by the processing power of a single GPU in a monolithic architecture. The overall concurrency processing capacity of the system increases linearly with the horizontal expansion of computing nodes. From the moment a request is submitted, the user can query the task status in real time via the task identifier. The process from pending processing to processing to final completion is transparent and visible, avoiding the pain point of passively waiting after submitting a request in traditional solutions without knowing any progress. This significantly improves user experience and system maintainability.

[0028] In some embodiments, sending a task request corresponding to a user request to the task scheduler via an API gateway includes: performing parameter verification on the user request via the API gateway; generating a task identifier corresponding to the task request after successful verification, setting the task request to a pending state; and placing the pending task request and its corresponding task identifier into the task queue of the task scheduler.

[0029] Because existing technologies lack a pre-filtering mechanism for the legitimacy of user requests, illegal or incorrectly formatted requests may only be exposed during inference execution on subsequent idle nodes, resulting in inefficient use of computing resources. Furthermore, the lack of a unified task identifier generation mechanism makes it impossible to pinpoint the specific task request during subsequent task status tracking and result retrieval. Upon receiving a user request, the API gateway first performs parameter validation. Specific validation may include: verifying whether the audio file format uploaded by the user is a system-supported audio format (e.g., WAV, MP3), verifying whether the target person's video file format is a system-supported video format (e.g., MP4, AVI), and verifying whether the file size exceeds the system's preset upper limit threshold. If any validation item fails, the API gateway directly returns a parameter error response to the user, rejecting the request, and the request will not proceed to the next step.

[0030] For valid user requests that pass verification, the API gateway generates a globally unique task identifier (task_id). This task identifier can be a string generated based on a Universally Unique Identifier (UUID) algorithm or a Snowflake algorithm, uniquely identifying the task request corresponding to this user request. Simultaneously with generating the task identifier, the API gateway initializes the task status of the request to a pending state (PENDING), indicating that the task request has been successfully received but has not yet been assigned to any idle node.

[0031] The API gateway persists pending task requests and their corresponding task identifiers as a complete task message into the task scheduler's task queue. This task queue can be implemented using message queue middleware, such as RabbitMQ, Kafka, or Redis Streams, to ensure that task messages are not lost in the event of system exceptions. The task message must contain at least: the task identifier, the storage path of the input file corresponding to the task request, the user-specified model type identifier, and the current task status. After being placed in the task queue, the API gateway returns a response containing the task identifier to the user, who can then use this identifier to query the task processing progress at any subsequent time.

[0032] By intercepting illegal requests such as those with format errors and file limits before computing resources are actually used, invalid requests are prevented from entering the task queue and only being exposed during the inference phase on idle nodes, thus saving network bandwidth, storage resources, and GPU computing resources.

[0033] In some embodiments, actively pulling task requests from the task queue via an idle node includes: The task scheduler registers the compute nodes and sends heartbeat signals to them for management. The idle node actively pulls task requests that are in a pending state from the task queue.

[0034] The task scheduler sends heartbeat requests to every compute node in the distributed computing resource pool at a fixed frequency (e.g., every 5 seconds). This heartbeat request uses a lightweight health check interface (e.g., GET / health) via the HTTP protocol. Upon receiving the heartbeat request, each compute node immediately returns its current working status information, which includes at least an idle state (IDLE, indicating that the idle node is not currently executing any inference tasks and can receive new tasks) and a busy state (BUSY, indicating that the compute node is currently executing an inference task and cannot temporarily receive new tasks). Based on the status information returned by each compute node, the task scheduler maintains a real-time updated node status table, recording the node identifier and corresponding working status of each compute node, thereby determining the real-time working status of all idle nodes in the distributed computing resource pool.

[0035] It is understood that the task scheduler in this application embodiment is used for node management.

[0036] In some embodiments, sending heartbeat detection to idle nodes for management includes: when the task scheduler detects that the heartbeat of an idle node has timed out or receives an error message reported by a compute node, marking the compute node as an abnormal state; resetting unfinished task requests on compute nodes in an abnormal state to a pending state, and putting them back into the task queue of the task scheduler, waiting to be picked up by an idle node.

[0037] In the actual operation of a distributed computing resource pool, computing nodes may malfunction due to reasons such as memory overflow, dependency library conflicts, GPU hardware failure, or network interruption. At this time, computing nodes in an abnormal state are unable to return heartbeat signals to the task scheduler or perform inference. However, if the task scheduler lacks the ability to identify abnormal nodes, subsequent tasks will fail continuously. Simultaneously, task requests currently being executed but not yet completed on the abnormal node will remain in a "processing" state indefinitely without being taken over, and users will never receive the processing results. Therefore, in this embodiment, the task scheduler, while continuously sending heartbeat detection signals to each computing node, determines the corresponding computing node to be in an abnormal state (UNHEALTHY) if any of the following abnormal situations occur: Situation 1: The task scheduler does not receive a heartbeat response from a computing node within a preset timeout period (e.g., two consecutive heartbeat detection cycles, i.e., 10 seconds), and determines that the computing node's heartbeat has timed out; Situation 2: The task scheduler receives error information actively reported by a computing node, such as a GPU memory overflow error, a model loading failure error, or an inference process crash report. Once a compute node is determined to be in an abnormal state, the task scheduler immediately marks it as abnormal in the node status table and suspends the allocation of new task requests to that node, thus automatically isolating it and preventing subsequent task failures. After marking a compute node as abnormal, the task scheduler queries the node for any incomplete task requests—those currently in the processing state and recorded as being executed by that node. The scheduler resets the status of these incomplete task requests from "processing" to "pending" and puts them back into the task queue. Simultaneously, the scheduler records the number of retries for each task to prevent infinite retry loops caused by the same task being repeatedly picked up by an abnormal node. The reset task requests will be picked up and executed by idle nodes in subsequent heartbeat detection cycles.

[0038] Thus, this embodiment of the application, through dual detection methods of heartbeat timeout monitoring and error information reception, enables the task scheduler to discover and mark abnormal nodes in the shortest possible time (e.g., within two heartbeat cycles) after an abnormality occurs in an idle node. This breaks the chain of influence of the faulty node on subsequent task requests, achieving domain isolation of single-point failures and avoiding the cascading effect of a single node's abnormality causing widespread task failures. By resetting unfinished task requests on abnormal nodes to a pending state and putting them back into the task queue, task requests that were interrupted due to node failure have the opportunity to be picked up and executed again by other healthy idle nodes. For users, this task recovery process is automatically completed by the system, without requiring users to resubmit requests, making node failures transparent to users and ensuring the high availability of the system. This allows the task scheduler to make appropriate scheduling responses regardless of whether the idle node is operating normally or experiencing a sudden abnormality, ensuring the stability and reliability of the digital human video generation service in high-concurrency scenarios.

[0039] In some embodiments, performing inference on a task request through an idle node includes: sequentially decoding, detecting and preprocessing the input file corresponding to the task request through the idle node; calling the lip-sync model integrated within the idle node to perform lip-sync inference and encoding on the preprocessed input file to obtain the output file.

[0040] The idle node calls the corresponding decoder to decompress the compressed input file into the original video frame sequence and audio waveform data. After decoding, the idle node can use a face detection model based on a convolutional neural network (such as MTCNN or RetinaFace) to locate the face bounding box coordinates in each frame and further identify facial key points. Preprocessing operations include: cropping the face region sub-image based on the face bounding box, removing interference information from the background region, further extracting the mouth region of interest based on the key point coordinates, performing size scaling on the cropped face or mouth region to uniformly adjust it to the fixed input size required by the lip-sync model, and filtering low-quality frames to remove frames with excessive blur, severe face occlusion, or excessive pose deviation to avoid the negative impact of low-quality input on model inference.

[0041] After preprocessing, the idle node invokes its integrated lip-sync model. Using the preprocessed face region image sequence and corresponding audio waveform data as input, it employs deep learning models such as Wav2Lip and HeyGem to perform synchronous inference based on audio features and the spatiotemporal context information of the face region in the video frame. Frame-by-frame, it generates a mouth region image matching the audio pronunciation of that frame. The idle node then performs encoding operations on the inference results. The encoder compresses and encodes the processed video frame sequence according to the target output format and performs audio-video synchronization encapsulation with the original audio stream, ultimately generating the output file.

[0042] In some embodiments, the process of calling the lip-sync model integrated within the idle node to perform lip-sync inference and post-encoding on the preprocessed input file to obtain the output file includes: dynamically combining video frames from multiple task-requested preprocessed input files into an inference batch; calling the lip-sync model integrated within the idle node using half-precision floating-point format; performing cross-task batch lip-sync inference on the video frames within the batch using the lip-sync model to generate mouth region images corresponding to each frame; and determining the output file based on the mouth region images.

[0043] Idle nodes maintain a batch assembly buffer. When the number of preprocessed video frames from multiple different task requests in the buffer reaches a preset batch size threshold, or when the waiting time since the first frame entered the buffer reaches a preset timeout limit (e.g., 50 milliseconds), the idle node dynamically combines all current frames in the buffer into an inference batch. The idle node loads its internally integrated lip-sync model weights using half-precision floating-point format and sends the assembled inference batch to the GPU memory all at once. The lip-sync model performs cross-task batch forward inference on all video frames within the batch. The GPU processes all frames in the batch in parallel using matrix operations. A single forward propagation simultaneously generates the mouth region image corresponding to each frame in the batch. The inference results for each frame are output separately according to the task identifier, without confusion. By dynamically combining video frames from multiple tasks into inference batches, the GPU can perform parallel matrix multiplication and addition calculations on all frames within the batch, significantly improving computational saturation and hardware utilization. Furthermore, batch inference merges multiple frames into a single forward propagation, and the model weights only need to be loaded once to serve all frames within the batch. Memory access latency is amortized across multiple frames, significantly reducing the average inference latency per frame.

[0044] In some embodiments, determining the output file based on the mouth region image includes: performing facial landmark localization on the original high-definition video frames in the input file to determine the mouth target region to be replaced in the original high-definition video frames; using image fusion technology to replace the mouth region image with the mouth target region, fusing it with the original high-definition video frames, and then encoding it to obtain the output file.

[0045] The idle node obtains the original high-definition video frame from the input file corresponding to the task request, calls the facial key point detection model, locates the coordinates of the facial key points on the original high-definition video frame, and accurately delineates the mouth target region to be replaced in the original high-definition video frame based on the mouth contour feature points among the identified facial key points. This region is the polygonal bounding box or its outer mask where the original mouth is located.

[0046] The idle node uses the gradient field of the generated mouth region as a guide to seamlessly embed the generated mouth region into the target region while maintaining the overall tone and lighting distribution of the original high-definition video frame, eliminating color differences and splicing artifacts at the replacement boundary. After fusion, the idle node encodes the fused complete video frame sequence to generate the final output file.

[0047] For example, in this embodiment, the first step is to locate the driving audio file and the target person video file corresponding to the task request. The target person video file is decoded into a continuous sequence of image frames, and the driving audio file is decoded into audio waveform data. The decoded image frame sequence undergoes operations such as facial key region cropping and size normalization. The audio waveform data undergoes preprocessing operations such as sampling rate unification, silence segment removal, or Mel-spectrum feature extraction to ensure the input data conforms to the input format and size specifications required by the subsequent lip-sync model. The idle node calls its internally integrated lip-sync model, inputting the preprocessed audio data and the preprocessed video frame sequence into the lip-sync model. The model performs lip-alignment inference frame by frame, generating an output video frame sequence with synchronized lip correction. After inference, the idle node performs video encoding on the output video frame sequence (e.g., using H.264 or H.265 encoding format), encapsulating the frame sequence into a video file of a specified format to obtain the final output file, which is the digital human video synchronized with the driving audio.

[0048] In some embodiments, invoking the lip-sync model integrated within the idle node includes: obtaining the task identifier of the task request; selecting the final invoking model from the lip-sync model according to the mapping relationship corresponding to the task identifier, wherein the mapping relationship is a one-to-one correspondence between the task identifier and the lip-sync model.

[0049] It is understood that, in this embodiment of the application, before preparing to invoke the lip-sync model, the idle node first obtains the task identifier corresponding to the received task request from the task request. The task identifier is the unique credential for locating and distinguishing different task requests. Specifically, the mapping relationship is a one-to-one correspondence between the task identifier and the lip-sync model; that is, each task identifier is pre-associated with a uniquely determined lip-sync model type at the system level. The lip-sync model can be a Wav2Lip or HeyGem model. This mapping relationship can be established by the API gateway during the task identifier generation stage based on the model preferences implicitly or explicitly stated in the user request and transmitted along with the task request. Based on the mapping relationship corresponding to the task identifier, the final model to be invoked is selected from the internally integrated lip-sync models.

[0050] In some embodiments, returning a query response through the API gateway includes: when the task scheduler receives a task status report from an idle node that is in a completed state, the task scheduler actively pulls the generated completed output file from the idle node and performs unified management and storage of the output file; after the task scheduler completes the unified storage of the output file, it generates a download link and associates it with the task identifier of the task request; and the API gateway queries and obtains the download link corresponding to the task request based on the task identifier.

[0051] Understandably, when the task scheduler receives a task request reported by an idle node with a completed status, the scheduler does not simply update the task status to completed and stop. Instead, it actively initiates an output file retrieval request to the idle node. Specifically, the task scheduler uses the HTTP file retrieval interface provided by the idle node to retrieve the completed output file (i.e., the digital human video file encoded according to claim 5) from the idle node's local file system to the task scheduler's storage space. Subsequently, the task scheduler renames the output file to correspond to the task identifier and transfers the file to a unified shared storage area (e.g., Network Attached Storage (NAS) or Object Storage Service), thereby achieving centralized storage of the output file and freeing it from dependence on the local storage of a single idle node. After completing the unified management and storage of the file, when the API gateway receives a user's status query request for a task request, if the task status is completed, the API gateway includes a download link for the corresponding output file in the returned query response. The user can then obtain the final output file through this download link.

[0052] After the task scheduler completes the unified storage of the output file—that is, after the output file has been fully written to the shared storage area and its integrity has been verified—it generates a download link for the output file that can be accessed externally. Simultaneously, the task scheduler associates the download link with the task identifier of the task request, establishing a one-to-one correspondence between the task identifier and the download link. This association is stored in a task status record table maintained by the task scheduler. Each task identifier, in addition to recording the current task status, is also associated with a download link field after the task is completed. This design ensures that the download link is only generated after the output file is securely written to disk, rather than being generated before the file is actually available. Thus, when a user initiates a task status query request through the API gateway, carrying the task identifier, the API gateway queries the task scheduler for the current task status of the requested task based on the task identifier. When the task scheduler returns a completed task status, the API gateway further retrieves the download link associated with that task identifier from the task scheduler and includes the download link in the query response, returning it to the user. The user can then download the output file using this download link. Throughout the process, the API gateway only needs to transmit the task identifier to obtain all associated information, without needing to maintain additional mapping relationships.

[0053] For example, a high-concurrency digital human video generation method can be implemented through the following steps: Step 1: Task Submission and Asynchronous Reception: The upstream service uploads the driver audio and target person video by calling the unified RESTful API provided by the system (such as POST / api / tasks), and immediately receives a response containing a unique task_id generated by the system. However, the generated result is not returned at this time, thus realizing asynchronous request and traffic shaping. Implementation: After receiving a request, the API gateway performs necessary parameter validation (such as file format and size). Upon successful validation, a task ID is generated, the task status is initialized to PENDING, and the task information (including metadata such as input file and required model type) is persisted as a message to the persistent task queue.

[0054] Step 2: Elastic task acquisition and allocation: Idle nodes actively pull tasks that match their capabilities from the persistent task queue. The system's processing capacity increases linearly with the increase of computing nodes. There is no central bottleneck in task allocation. Tasks are pulled as soon as nodes are idle, resulting in rapid response and high resource utilization.

[0055] Implementation: Each intelligent idle node runs a task acquisition loop. When a node is idle and healthy, it sends a "pull request" to the persistent task queue. The node can specify filters in the request, such as only pulling tasks with `model_type == "Wav2Lip"`, achieving natural "model routing". The queue returns one or more tasks that meet the criteria to the node. After the node receives a task, it updates the task status to `PROCESSING`.

[0056] Step 3: Integrated Quality and Efficiency Optimization of Task Execution: After an idle node pulls a task, it executes an inference pipeline that integrates optimization strategies. Through preprocessing, precise fusion, and post-processing super-resolution, the output video has clear lip movements and high-definition image quality, solving the blurring problem of the original model. Quantization, batch processing, and caching mechanisms significantly reduce the resource consumption and latency of a single task, improving the overall throughput of the system.

[0057] Implementation: The node extracts keyframes and performs quality screening on the input video, filtering out low-quality frames; it also performs lightweight super-resolution or sharpening preprocessing on the face region (especially the mouth ROI) to improve input quality; it uses FP16 precision for inference, accelerating speed and reducing power consumption, and performs dynamic batch processing: the node can combine video frame sequences from multiple tasks into a batch and send them to the GPU for cross-task batch processing inference, greatly improving GPU utilization; the preprocessing results of frequently used target person videos and the loaded models reside in GPU memory to avoid repeated loading; it only uses the mouth region generated by Wav2Lip / HeyGem, locates facial key points, and uses image fusion technology (such as Poisson fusion) to seamlessly paste it back into the original high-definition video; optionally, it uses a lightweight face super-resolution model to enhance the final image quality of the entire output video or face region; the node directly writes the task execution status (progress, result) to a shared result storage (such as a database or object storage).

[0058] Step 4: Health monitoring and fault recovery: The system handles node failures through heartbeat and timeout mechanisms, achieving high availability, and fault recovery is perfectly integrated with the system architecture.

[0059] Implementation: If a node becomes disconnected (heartbeat timeout), an independent monitoring process will re-mark the task it is processing as PENDING and put it back into the task queue, so that other healthy nodes can retrieve it again; a faulty node is marked as unhealthy until it recovers and passes the health check.

[0060] Step 5: Result Query: The upstream service obtains the task status and results by querying the API (GET / api / tasks / {task_id}). The status query is decoupled from the task execution, resulting in a clear architecture.

[0061] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned high-concurrency digital human video generation method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0062] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 302 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called and executed by the processor 301 to execute the high-concurrency digital human video generation method of the embodiments of this application. Input / output interface 303 is used to implement information input and output; The communication interface 304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 305 transmits information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304); The processor 301, memory 302, input / output interface 303, and communication interface 304 are connected to each other within the device via bus 305.

[0063] In some embodiments, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described high-concurrency digital human video generation method.

[0064] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0065] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0066] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0069] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0070] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0072] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for generating digital human videos with high concurrency, characterized in that, The method, applied to a digital human video generation system, which includes an API gateway, a task scheduler, and a distributed computing resource pool, comprises: In response to a user request, the task request corresponding to the user request is sent to the task queue of the task scheduler through the API gateway; The idle computing nodes in the distributed computing resource pool are designated as idle nodes, and the task requests are actively pulled from the task queue by the idle nodes. The idle node performs inference on the task request, updates the task status of the task request to the completed status through the task scheduler, and then returns a query response through the API gateway.

2. The digital human video generation method with high concurrency calls according to claim 1, characterized in that, The step of sending the task request corresponding to the user request to the task scheduler's queue through the API gateway includes: The API gateway performs parameter validation on the user request. After the verification is successful, the task identifier corresponding to the task request is generated, and the task request is set to the pending state. The task request that is in the pending state and its corresponding task identifier are placed into the task queue of the task scheduler.

3. The digital human video generation method with high concurrency calls according to claim 1, characterized in that, The step of actively pulling the task request from the task queue through the idle node includes: The task scheduler registers the computing nodes and sends heartbeat detection signals to the computing nodes for management. The idle node actively pulls the task requests that are in a pending state from the task queue.

4. The digital human video generation method with high concurrency calls according to claim 3, characterized in that, The process of sending heartbeat detections to the computing node for management includes: When the task scheduler detects that the heartbeat of the computing node has timed out or receives an error message reported by the computing node, it marks the computing node as an abnormal state. The unfinished task requests on the computing node in the abnormal state are reset to the pending state and put back into the task queue of the task scheduler, waiting to be picked up by the idle node.

5. The digital human video generation method with high concurrency calls according to claim 1, characterized in that, The step of performing inference on the task request through the idle node includes: The idle node sequentially decodes, detects, and preprocesses the input file corresponding to the task request; The lip-sync model integrated within the idle node is invoked to perform lip-sync inference and encoding on the preprocessed input file, resulting in the output file.

6. The digital human video generation method with high concurrency calls according to claim 5, characterized in that, The step of calling the lip-sync model integrated within the idle node to perform lip-sync inference and encoding on the preprocessed input file yields the following output file: The video frames of the preprocessed input files from multiple task requests are dynamically combined into an inference batch, and the lip-sync model integrated within the idle node is called using a half-precision floating-point format. The lip-sync model is used to perform cross-task batch lip-sync inference on the video frames in the inference batch, generating mouth region images corresponding to each frame. The output file is determined based on the mouth region image.

7. The high-concurrency digital human video generation method according to claim 6, characterized in that, Determining the output file based on the mouth region image includes: Facial key point localization is performed on the original high-definition video frames in the input file to determine the mouth target area to be replaced in the original high-definition video frames; Image fusion technology is used to replace the mouth region image with the mouth target region, and after fusion with the original high-definition video frame, the image is encoded to obtain the output file.

8. The digital human video generation method with high concurrency calls according to claim 1, characterized in that, The query response returned through the API gateway includes: When the task scheduler receives a report from the idle node that the task status is "completed", the task scheduler actively pulls the generated output file from the idle node and performs unified management and storage of the output file. After the task scheduler completes the unified storage of the output file, it generates the download link and associates it with the task identifier of the task request. The API gateway queries and retrieves the download link corresponding to the task request based on the task identifier.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the digital human video generation method with high concurrency calls as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the digital human video generation method with high concurrency calls as described in any one of claims 1 to 8.

Images