A method and system for elastic scaling scheduling of digital human services with GPU memory resident

Abstract

This invention discloses a method and system for elastic scaling scheduling of a GPU-resident digital human service. The method includes: upon service startup, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads an AI model into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen to put the digital human instance in a preheated-ready state. Upon receiving a user's session startup request, the server allocates an idle digital human instance in the preheated-ready state from the GPU resource pool and returns the RTC room information corresponding to the idle digital human instance to the user client. The user client joins the room based on the RTC room information and interacts with the digital human in real time. After the session ends, the server performs context cleanup on the digital human instance, resets its state to idle, and reclaims it from the GPU resource pool. This invention achieves preheated-readiness, dynamic reuse, and intelligent scheduling of digital human instances through GPU memory resident and elastic scaling strategies.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for elastic scaling scheduling of digital human services with GPU memory resident. Background Technology

[0002] Current AI digital humans require the integration of multiple technologies, including Automatic Speech Recognition (ASR), Large Language Model (LLM), Text-to-Speech (TTS), digital human rendering, and Real-Time Communication (RTC). Digital human rendering refers to the complete process of using computer graphics technology to transform a 3D digital human model (including its geometry, texture, skeleton, facial expressions, etc.) into realistic or stylized 2D images or video sequences displayed on a screen. Digital human rendering is a computationally intensive task, requiring GPUs for deep learning inference, lip-sync prediction, face rendering, and video encoding. A single digital human requires 2-4GB of GPU memory, and loading the model into memory takes 5-10 seconds.

[0003] The existing technology has the following disadvantages (1) High startup latency: Each user request requires reallocating GPU resources, loading the model into the video memory, and creating an RTC room. The entire startup process takes 5-10 seconds, resulting in a poor user experience.

[0004] (2) Low resource utilization: Digital human instances are bound one-to-one with user sessions. Once the session ends, the instance is destroyed. Loaded models and created RTC rooms cannot be reused. Frequent allocation and release of GPU resources result in waste.

[0005] (3) Severe hardware wear and tear: Traditional solutions either start on demand (high latency) or run 24 hours a day (GPU continuously at full load of 80-85°C, that is, in the traditional 24-hour continuous solution, the GPU needs to continuously perform digital human rendering calculations, including: deep learning model inference (lip-reading prediction, face rendering), video encoding (H.264 / H.265), real-time audio and video processing. These computing tasks keep the GPU utilization rate at 80-100% for a long time, causing the GPU core temperature to run continuously in the high temperature range of 80-85°C. Long-term high temperature will accelerate silicon crystal aging and solder joint fatigue, shortening the GPU lifespan from the normal 10-15 years to 2-3 years), resulting in a GPU lifespan of only 3-4 years.

[0006] (4) Lack of elasticity and scalability: The fixed pool size cannot adapt to fluctuations in business traffic, resulting in insufficient resources during peak periods and wasted resources during off-peak periods.

[0007] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0008] The main objective of this invention is to provide a method and system for elastic scaling scheduling of digital human services with GPU memory resident, aiming to solve the problems of high startup latency, low GPU resource utilization, severe hardware loss and lack of elastic scaling capability in the existing technology when rendering digital humans.

[0009] To achieve the above objectives, the present invention provides a method for elastic scaling scheduling of digital human services with GPU memory resident, the method comprising the following steps: When the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads AI models into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen, so that the digital human instance is in a preheating and ready state. When a user's session start request is received, an idle digital human instance in a preheating-ready state is allocated from the GPU resource pool, and the RTC room information corresponding to the idle digital human instance is returned to the user. The user joins a room based on the RTC room information and interacts with the digital human in real time. After the session ends, the context of the digital human instance is cleaned up, and the state of the digital human instance is reset to idle and then recycled back to the GPU resource pool.

[0010] Optionally, the GPU memory-resident digital human service elastic scaling scheduling method, wherein when the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads an AI model into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen, putting the digital human instance in a preheating-ready state, specifically includes: When the service starts, the scheduling server reads the preset initial instance count configuration of the GPU resource pool; The scheduling server determines whether the number of currently initialized digital human instances has reached the initial number of instances; If the target is not met, the scheduling server will allocate independent GPU resources to the new digital human instance, load the required AI model into the GPU memory, create the corresponding real-time communication (RTC) room, and render the default standby screen. Mark the initialized digital human instance as ready and add it to the idle instance pool of the GPU resource pool until the number of initialized instances reaches the preset initial instance number.

[0011] Optionally, in the GPU memory-resident digital human service elastic scaling scheduling method, the AI ​​model includes an audio feature extraction model, a lip-sync generation model, an image encoding / decoding model, a face detection model, and an expression-driven model.

[0012] Optionally, the GPU memory-resident digital human service elastic scaling scheduling method, wherein when a user's session initiation request is received, an idle digital human instance in a preheating-ready state is allocated from the GPU resource pool, and the RTC room information corresponding to the idle digital human instance is returned to the user, specifically includes: When the scheduling server receives a user's session start request, it determines whether the current number of active sessions in the system has reached the set maximum concurrency limit. If the limit is not exceeded, then find a digital human instance with an IDLE state from the idle instance pool of the GPU resource pool; If an idle instance exists, change the status of the idle instance from IDLE to OCCUPIED, and bind the idle instance to the current user information; Based on the scenario information carried in the session initiation request, update the prompt words of the Large Language Model (LLM), bind the corresponding knowledge base, and set the timbre parameters of the TTS speech synthesis and ASR speech recognition. Generate the corresponding RoomId for the Real-Time Communication (RTC) room and the temporary Token required for the user to join the RTC room, and return the RoomId and Token to the user. If there are no available instances in the idle instance pool, asynchronous scaling is initiated, and the current session request is added to the waiting queue.

[0013] Optionally, in the GPU memory-resident digital human service elastic scaling scheduling method, the asynchronous expansion includes: When a user request arrives, if there are insufficient available instances, a response is returned immediately. The background process initiates a scaling task, waking up dormant instances or creating new ones. Once the scaling is complete, a notification is pushed out, and users can join the room after receiving the notification.

[0014] Optionally, the GPU memory-resident digital human service elastic scaling scheduling method, wherein the user client joins a room based on the RTC room information and interacts with the digital human in real time, and after detecting the end of the session, performs context cleanup on the digital human instance, resets the state of the digital human instance to idle, and reclaims it into the GPU resource pool, specifically includes: When the user session is detected to have ended, the scheduling server changes the status of the digital human instance to RECYCLING. Perform a context cleanup operation to clear the large language model (LLM) dialogue history and knowledge base retrieval context of the digital human instance, and delete the bound user information; Perform resource quota release operations to reduce the global concurrent session count, scene concurrent count, and device concurrent count, so as to allow new sessions to use the released quota; Control the rendering of the digital human instance and switch it to a preset standby screen to maintain visual activity; The state of the digital human instance is reset from RECYCLING to IDLE, and the digital human instance is recycled to the idle instance pool of the GPU resource pool for allocation and reuse in subsequent session requests.

[0015] Optionally, the GPU memory-resident digital human service elastic scaling scheduling method further includes: The scheduling server starts a scheduled task to periodically obtain the current status of the GPU resource pool; Based on the current status, determine whether the number of idle instances is less than a preset threshold; If so, wake up the specified number of dormant instances from the dormant instance pool and add them to the idle instance pool to complete the expansion; If not, a specified number of idle instances will be transferred to the hibernation pool to release the occupied GPU resources and complete the scaling down. The scheduling server records detailed information about each scaling operation in the log.

[0016] Optionally, in the GPU memory-resident digital human service elastic scaling scheduling method, the scheduled task is a background task automatically executed by the system at fixed time intervals, used to monitor the resource pool status and trigger elastic scaling.

[0017] Optionally, in the GPU memory-resident digital human service elastic scaling scheduling method, the logs are used for operation and maintenance monitoring, performance analysis, cost optimization, and problem troubleshooting. The operation and maintenance monitoring includes real-time viewing of scaling operations, detection of abnormal scaling behavior, and alarm notifications; The performance analysis includes statistical analysis of expansion frequency, analysis of reduction effect, and optimization of threshold configuration; The cost optimization includes calculating resource utilization, evaluating the benefits of elastic scaling, and adjusting time-period strategies. The troubleshooting process includes tracing back historical operations, identifying performance bottlenecks, and reproducing the problem scenario.

[0018] In addition, to achieve the above objectives, the present invention also provides a GPU memory-resident digital human service elastic scaling scheduling system, wherein the GPU memory-resident digital human service elastic scaling scheduling system includes: a scheduling server, a GPU resource pool, and a user terminal; When the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads an AI model into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen to put the digital human instance into a preheating and ready state. When a user's session start request is received, the scheduling server allocates an idle digital human instance in a preheating-ready state from the GPU resource pool and returns the RTC room information corresponding to the idle digital human instance to the user client. The user joins a room based on the RTC room information and interacts with the digital human in real time. After the session ends, the scheduling server cleans up the context of the digital human instance, resets the state of the digital human instance to idle, and then reclaims it into the GPU resource pool.

[0019] In this invention, upon service startup, the scheduling server initializes multiple digital human instances in the GPU resource pool. For each digital human instance, an AI model is preloaded into the GPU memory, an independent real-time communication (RTC) room is created, and a standby screen is rendered, putting the digital human instance in a preheated-ready state. When a user's session startup request is received, an idle digital human instance in the preheated-ready state is allocated from the GPU resource pool, and the RTC room information corresponding to the idle digital human instance is returned to the user. The user joins the room based on the RTC room information and interacts with the digital human in real time. After the session ends, the context of the digital human instance is cleaned up, and its state is reset to idle before being returned to the GPU resource pool. This invention achieves preheating-readiness, dynamic reuse, and intelligent scheduling of digital human instances through GPU memory resident and elastic scaling strategies. Attached Figure Description

[0020] Figure 1 This is a flowchart of a preferred embodiment of the GPU memory-resident digital human service elastic scaling scheduling method of the present invention; Figure 2 This is a schematic diagram illustrating the overall interaction between the scheduling server, GPU resource pool, and user terminal in the GPU memory-resident digital human service elastic scaling scheduling system of this invention. Figure 3 This is a flowchart of the resource pool initialization process in a preferred embodiment of the GPU memory-resident digital human service elastic scaling scheduling method of the present invention. Figure 4This is a flowchart of user session allocation in a preferred embodiment of the GPU memory-resident digital human service elastic scaling scheduling method of the present invention. Figure 5 This is a flowchart of session termination and instance reclamation in a preferred embodiment of the GPU memory-resident digital human service elastic scaling scheduling method of the present invention; Figure 6 This is a flowchart of the elastic scaling process in a preferred embodiment of the GPU memory-resident digital human service elastic scaling scheduling method of the present invention. Figure 7 This is an instance state transition diagram in a preferred embodiment of the GPU memory-resident digital human service elastic scaling scheduling method of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0022] The preferred embodiment of the present invention describes a method for elastic scaling scheduling of digital human services with persistent GPU memory, such as... Figure 1 and Figure 2 As shown, the method for elastic scaling scheduling of the digital human service with GPU memory resident includes the following steps: Step S10: When the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads AI models into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen, so that the digital human instance is in a preheating-ready state.

[0023] Specifically, such as Figure 3 As shown, when the service starts, the scheduling server reads the preset initial instance count configuration of the GPU resource pool; the scheduling server determines whether the number of currently initialized digital human instances has reached the initial instance count; if it has, the initialization is complete; if it has not, the scheduling server allocates independent GPU resources to the new digital human instance, loads the required AI model into the GPU memory, creates the corresponding real-time communication (RTC) room, and renders the default standby screen; the initialized digital human instance is marked as ready and added to the idle instance pool of the GPU resource pool until the number of initialized instances reaches the preset initial instance count.

[0024] The initial number of instances (i.e., the initial active count) can be 10, which can be dynamically adjusted according to business needs; the initial active count refers to the number of digital human instances that are pre-created and kept active when the service starts.

[0025] Configuration example: { "poolConfig": { "totalCapacity": 30, / / Total capacity of the resource pool "initialActiveCount": 10, / / Initial active count "minActiveCount": 2, / / Minimum number of active users "maxActiveCount": 30, / / Maximum number of active users } } The initial active number is determined based on factors including: historical traffic data analysis, peak business concurrency demands, total GPU hardware resources, and the balance between cost and performance. Different initial active numbers (initial values) can be configured for different business scenarios: small applications: 2-5 initial paths; medium applications: 10-15 initial paths; large applications: 20-30 initial paths.

[0026] When there are insufficient idle instances in the GPU resource pool, expansion is required. The specific expansion scenarios are explained below: Scenario 1: There are idle instances (normal situation): Ten instances were initialized in the pool. Eight were used by users, leaving two idle. When another user accesses the system, the system directly allocates one instance from the idle pool (response within seconds), without needing to reallocate GPUs, load models, or create rooms.

[0027] Scenario 2: No idle instances (requires expansion): Ten instances were initialized and are all occupied. When a new user accesses the system, the number of idle instances is 0, triggering the expansion condition. The system wakes up instances from the hibernation pool or creates new instances, which requires the following steps: allocate GPU → load model → create room (10-20 seconds).

[0028] Therefore, "allocating GPUs, loading models, and creating rooms" are operations that need to be performed only during expansion, not with every user request.

[0029] Idle GPUs are randomly allocated to load the digital human engine model for real-time rendering. The GPU allocation strategy involves randomly selecting one GPU from the reclaimed idle GPUs. Allocation algorithm: Available GPU list = [GPU0, GPU1, GPU2, ..., GPU_N]; Idle GPU = Filtered (Status == IDLE); Select GPU = Randomly select (idle GPU); A load balancing strategy can also be used: Round Robin, prioritizing the fewest connections and the lowest GPU temperature.

[0030] The loaded models, the digital human engine models include several core models listed in Table 1: Table 1: Correspondence between Digital Human Engine Models and Functions, and Graphics Memory Usage

[0031] These models work together to achieve real-time rendering of digital humans: receiving user voice input; extracting audio features; predicting corresponding lip movements and facial expressions; rendering and generating digital human video frames; and then encoding and pushing them to the user via RTC. The entire process needs to be completed in milliseconds to ensure audio-visual synchronization and a real-time interactive experience.

[0032] The standby screen serves several purposes: when the instance is idle, the digital human remains in standby mode (blinking, breathing, and other natural movements); users see the digital human is "alive" before entering the instance, enhancing the experience; and it avoids black screens or static displays.

[0033] The technical implementation includes: Backend: After the digital human instance creates an RTC room, it continuously pushes standby video streams; Frontend: Before the user enters the room, a standby screen preview can be selectively displayed; the standby screen is a pre-recorded video that loops or a real-time generated idle animation.

[0034] By proactively and in batches completing the most time-consuming operations such as GPU allocation, model loading, and room creation during service startup, the "cold start" process, which originally needed to be executed with each user request (5-10 seconds), is moved to the service startup phase. This ensures that the digital human instance is in a "hot ready" state before a user request arrives, allowing for direct allocation upon request and reducing response time to the millisecond level (milliseconds to seconds).

[0035] By pre-creating and maintaining a pool of ready instances (idle instance pool), computing resources (GPUs, models, RTC channels) are transformed from "create-on-demand" to "pre-allocated and reused." This avoids resource allocation overhead and fragmentation caused by frequent instance creation and destruction, significantly improving the utilization of scarce resources such as GPUs, while ensuring stable system responsiveness under high concurrency. User-perceived startup latency is greatly reduced, resulting in an instant-response interactive experience. On the system side, by avoiding repeated initialization overhead, overall throughput is improved, and buffer resources are provided to handle sudden traffic surges.

[0036] Step S20: When a user's session start request is received, an idle digital human instance in a preheating-ready state is allocated from the GPU resource pool, and the RTC room information corresponding to the idle digital human instance is returned to the user.

[0037] Specifically, such as Figure 4 As shown, when the scheduling server receives a user's session initiation request, it determines whether the current number of active sessions in the system has reached the set maximum concurrency limit. If it has not exceeded the limit, it searches for a digital human instance in the idle instance pool of the GPU resource pool with an IDLE state. If an idle instance exists, it changes the state of the idle instance from IDLE to OCCUPIED and binds the idle instance to the current user information. Based on the scene information carried in the session initiation request, it updates the prompt words of the Large Language Model (LLM), binds the corresponding knowledge base, sets the timbre parameters of the TTS speech synthesis and the ASR speech recognition parameters, generates the corresponding RTC room RoomId and the temporary Token required for the user to join the RTC room, and returns the RoomId and Token to the user. If there are no available instances in the idle instance pool, it initiates asynchronous expansion and adds the current session request to the waiting queue.

[0038] Concurrency exceeding the limit refers to the number of concurrent sessions exceeding the system's maximum concurrency limit. Concurrency limits are divided into three levels: Global concurrency limit: The maximum number of sessions that the entire system can support simultaneously, determined by the total amount of GPU hardware resources. Example: Total number of GPUs: 10 RTX 4090s; per GPU: 3 digital human lanes; global concurrency limit: 30 lanes.

[0039] Scenario Concurrency Limit: The maximum number of concurrent connections for a single business scenario (such as smart buildings or customer service centers), used for resource isolation and fair allocation. Examples: Scenario A (Smart Buildings): Maximum 10 connections; Scenario B (Customer Service Centers): Maximum 15 connections; Scenario C (Exhibition Hall Guidance): Maximum 5 connections.

[0040] Device concurrency limit: A single device (such as a large screen) can only initiate one session at a time to prevent resource abuse. Example: Device ID: device_001; Current concurrency: 1; Attempt to initiate a second session → Rejected (Device concurrency exceeded).

[0041] The over-limit handling strategy is shown in Table 2: Table 2: Correspondence between Exceeding Limit Types and Handling Methods

[0042] The complete allocation process is as follows: Step 1: Allocate instance (0.1 seconds).

[0043] - Select an instance in the IDLE state from the idle pool; - Mark instance status: IDLE → OCCUPIED; - Bind user information (device ID, user ID).

[0044] Step 2: Apply configuration (0.5-1 second).

[0045] - Update LLM prompts (based on the scenario); - Bind to a knowledge base (such as a building knowledge base); - Set the TTS voice (male / female); - Configure ASR parameters.

[0046] Step 3: Return voucher (0.1 seconds).

[0047] - Generate RoomId + Token; - Return to the user.

[0048] The total time is approximately 0.7-1.2 seconds, far less than the 10-30 seconds of traditional solutions.

[0049] For application configuration, scenario-based customization is achieved by dynamically updating instance runtime parameters. Configuration content includes: LLM prompt word configuration: { "systemPrompt": "You are the AI ​​assistant for the smart building, responsible for answering visitors' questions about building facilities, meeting room bookings, parking, etc." question...", "personality": "Professional, friendly, and efficient" "constraints": ["Do not discuss topics unrelated to the building", "Protect user privacy"] } Knowledge base binding: { "knowledgeBaseId": "kb_building_001", "retrievalTopK": 5, "similarityThreshold": 0.75 } TTS sound configuration: { "voiceId": "zh_female_qingxin", "speed": 1.0, "pitch": 0, "volume": 1.0 } ASR parameter configuration: { "language": "zh-CN", "enablePunctuation": true, "enableITN": true } The application configuration process includes: (1) Read the scene configuration template: Scene ID → Configure Template Library → Get Configuration JSON.

[0050] (2) Call the instance configuration interface: POST / instance / {instanceId} / config; Body: Configure JSON.

[0051] (3) Internal updates within the instance: - LLM service: Update system prompt; - Knowledge base service: Switch search source; - TTS service: Switch voice model; - ASR service: Update identification parameters.

[0052] (4) Configuration takes effect: Time taken: 0.5-1 second.

[0053] The configurations of different users are completely isolated and do not affect each other: User A: Using the building knowledge base + female voice; User B: Using the customer service knowledge base + male voice; The configuration is cleared and restored to the default state when the instance is recycled.

[0054] Here, RoomId is the unique identifier of the RTC room, and Token is the temporary credential for users to join the room. RoomId (Room ID) is a globally unique identifier for RTC rooms, used to distinguish different call rooms. Format example: room_20260122_abc123def456; composition: Prefix: room_; Timestamp: 20260122 (Year, Month, Day); Random string: abc123def456.

[0055] A token (access token) is a temporary authorization credential for a user to join an RTC room, containing the following information: User ID; Room ID; Permissions (publish / subscribe to audio and video); Validity period (usually 1-2 hours).

[0056] Token generation algorithm: Token = HMAC-SHA256( data: userId + RoomId + expireTime + permissions, key: appSecret ) The usage process is as follows: (1) After the backend allocates an instance, it generates a RoomId and a Token. return:{ "roomId": "room_20260122_abc123", "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...", "expireTime": 1737532800 } (2) The front end uses RoomId and Token to join the RTC room; rtcClient.joinRoom(roomId, token).

[0057] (3) The user establishes an audio and video connection with the digital human: - Receive digital human video streams; - Send user audio stream.

[0058] (4) Tokens can be renewed before they expire, but must be obtained again after they expire.

[0059] Scalability refers to the system's ability to dynamically increase the number of active instances. Scalability refers to the "number of active instances," which are digital human instances in an IDLE or OCCUPIED state that are ready to provide services immediately or soon.

[0060] Source of expansion: Source 1: Wake up a sleeping instance (recommended, 10-20 seconds); Select from instances of the SLEEPING state; Reload the model into GPU memory; Recreate the RTC room; State transition: SLEEPING → IDLE.

[0061] Source 2: Create a new instance (slower, 15-30 seconds); When the hibernation pool is also empty; Complete initialization process; State transition: None → INIT → IDLE.

[0062] Expansion conditions: Expansion is triggered when all of the following conditions are met: 1. Number of idle instances < expansion threshold (e.g., 3); 2. Total number of active instances < maximum value for the current time period; 3. Total number of active instances < total resource pool capacity.

[0063] Example: Current status: - Active instances: 28 (25 occupied, 3 idle); - Hibernation instance: 2-way; - Total capacity: 30 channels; - Current maximum value for the current time period: 30 routes; - Expansion threshold: Idle capacity < 3.

[0064] Judgment: Idle capacity = 3, expansion not triggered (critical value); With a new user joining, the available slots become 2. Judgment: Idle < 3 and Active < 30 → Trigger expansion; Two dormant instances are awakened, increasing the number of active instances to 30.

[0065] Asynchronous scaling refers to scaling operations being performed in the background without blocking user requests. The synchronous scaling process is: user request → check for no free instances → wait for scaling to complete (10-20 seconds) → allocate instances → return; User experience: waiting 10-20 seconds, synchronous scaling is not recommended. This invention uses asynchronous scaling, the process of which is: user request → check for no free instances → immediately return to preparing → background scaling → notify user upon completion; User experience: immediate feedback, background preparation.

[0066] The asynchronous expansion process is as follows: 1. User request arrives; 2. Insufficient available instances detected; 3. Return a response immediately (0.1 seconds); { "status": "preparing" Message: "Digital human is being prepared, please wait..." "estimatedTime": 15 } 4. Start the scaling task in the background (asynchronously); - Wake up sleeping instances; - Or create a new instance; 5. A notification will be sent after the expansion is completed; WebSocket push: { "event": "instanceReady", "roomId": "room_xxx", "token": "token_xxx" } 6. The user receives a notification and joins the room.

[0067] Advantages of asynchronous scaling: users will not wait for a long time without response; progress prompts can be displayed to improve the user experience; the system can scale up in batches to improve efficiency; and it can gracefully degrade when scaling fails.

[0068] Step S30: The user joins the room based on the RTC room information and interacts with the digital human in real time. After the session ends, the context of the digital human instance is cleaned up, and the state of the digital human instance is reset to idle and then recycled back to the GPU resource pool.

[0069] Specifically, such as Figure 5 As shown, when the user session is detected to be ending, the scheduling server changes the state of the digital human instance to RECYCLING; performs a context cleanup operation, clearing the large language model (LLM) dialogue history and knowledge base retrieval context of the digital human instance, and deleting the bound user information; performs a resource quota release operation, reducing the global concurrent session count, scene concurrent count, and device concurrent count to allow new sessions to use the released quota; controls the rendering of the digital human instance and switches it to a preset standby screen to maintain visual activity; resets the state of the digital human instance from RECYCLING to IDLE, and reclaims the digital human instance to the idle instance pool of the GPU resource pool for allocation and reuse in subsequent session requests.

[0070] Users join an RTC room using RoomId and Token, establish an audio-visual connection with the digital human, and begin real-time dialogue interaction. The complete interaction process includes: Step 1: The user obtains the RoomId and Token; Frontend call: POST / api / session / start; Returns: {roomId, token}.

[0071] Step 2: The user joins the RTC room; rtcClient.joinRoom(roomId, token).

[0072] Step 3: Subscribe to the digital human video stream; rtcClient.subscribe(digitalHumanStream); Users see a digital human image.

[0073] Step 4: Publish the user's audio stream; rtcClient.publish(userAudioStream); Digital humans can hear what users say.

[0074] Step 5: Start the conversation; User: "Hello, where is the meeting room?" ASR recognition: Text = "Hello, where is the meeting room?"; LLM understanding + knowledge base retrieval: generating answers; TTS Synthesis: Voice = "Hello, the meeting room is on the 3rd floor..."; Digital human rendering: lip movements + facial expressions + voice; RTC streaming: Users see a digital human speaking.

[0075] During the dialogue, the system features: real-time performance (end-to-end latency <1 second), audio-visual synchronization (lip movements and speech are precisely aligned), natural interaction (supports interruption and multi-turn dialogue), and context preservation (remembers the dialogue history).

[0076] Cleaning up the context deletes a user's conversation history and state data, while releasing the quota returns the concurrency count, allowing new users to use it. Cleaning up the context includes: Clean up LLM conversation history: # Before cleaning conversationHistory = [ {"role": "user", "content": "Where is the meeting room?"} {"role": "assistant", "content": "The meeting room is on the 3rd floor..."} {"role": "user", "content": "Do I need to make an appointment?"} {"role": "assistant", "content": "Yes, advance booking is required..."} ] # After cleaning conversationHistory = [] # Clear.

[0077] Clean up the knowledge base retrieval context: # Before cleaning retrievalContext = { "lastQuery": "Meeting room reservation" "retrievedDocs": [...] "relevanceScores": [...] } # After cleaning retrievalContext = None # Clear.

[0078] Clear user binding information: # Before cleaning instanceBinding = { "userId": "user_123", "deviceId": "device_456", "sessionId": "session_789", "startTime": 1737532800 } # After cleaning instanceBinding = None # Unbind.

[0079] The released quota includes: Global concurrency count: globalConcurrentCount -= 1 # 30 → 29; Scene concurrency count: sceneConcurrentCount["Smart Building"] -= 1 # 10→9; Device concurrency count: deviceConcurrentCount["device_456"] -= 1 # 1→0; After the quota is released, new users can use these quotas to initiate sessions.

[0080] The necessity of cleaning includes: Privacy protection: Prevent the next user from seeing the previous user's conversation; State isolation: Ensures that each user gets a clean instance; Resource reclamation: freeing up memory and concurrency quotas; Fast reuse: Cleanup time is less than 100ms, which does not affect the reuse speed.

[0081] Switching to the standby screen is to allow the instance to return to an idle state, waiting for the next user, while maintaining the visual effect of the digital human being "alive".

[0082] The functions of the standby screen include: Visual continuity: After a user leaves, the digital human does not suddenly disappear or become still; instead, it naturally returns to a standby state (blinking, breathing, smiling); the next user sees the "alive" digital human when they enter.

[0083] Minimize resource consumption: The standby screen is a lightweight video that plays in a loop; GPU utilization is <10% (80-100% during conversations); reduce power consumption and temperature.

[0084] Quick Response Preparation: Instances remain online; no re-initialization is required when new users log in; switch directly from standby to conversation mode (<0.5 seconds).

[0085] The standby screen can be implemented in the following ways: Method 1: Loop playback of pre-recorded video.

[0086] Standby video: 10-second loop; Content: The digital human stands naturally, occasionally blinking and smiling; Encoding: H.264, bitrate 500kbps.

[0087] Method 2: Generate idle animation in real time.

[0088] Drive parameters: - Blinking frequency: every 3-5 seconds; - Breathing amplitude: slight chest rise and fall; - Slight head movements: slight swaying from side to side.

[0089] State transitions include: Dialogue state → Standby state; - Stop receiving user audio; - Stop LLM inference; - Stop TTS synthesis; - Switch to standby animation driver; - GPU utilization: 80% → 10%.

[0090] Standby mode → Dialogue mode; - Start receiving user audio; - Initiate LLM inference; - Start TTS synthesis; - Switch to dialogue animation driven; - GPU utilization: 10% → 80%.

[0091] Furthermore, as shown in step 6, the scheduling server starts a scheduled task to periodically obtain the current status of the GPU resource pool. Based on the obtained current status, including key indicators such as the number of instances, distribution, and load, it determines whether the number of idle instances is less than a preset threshold. If so, it wakes up a specified number of dormant instances from the dormant instance pool and adds them to the idle instance pool to complete the expansion. If not, it transfers a specified number of idle instances to the dormant pool to release the occupied GPU resources and complete the shrinkage. The scheduling server records detailed information of each scaling operation to the log.

[0092] The scheduled task is a background task that the system automatically executes at fixed time intervals to monitor the resource pool status and trigger elastic scaling. Scheduled task configuration: { "scheduledTasks": { "elasticScaling": { "enabled": true, "interval": "1m", / / Execute once every 1 minute Description: "Flexibility and stretching check" }, "healthCheck": { "enabled": true, "interval": "30s", / / Executes every 30 seconds "description": "Instance health check" }, "metricsCollection": { "enabled": true, "interval": "10s", / / Executes every 10 seconds "description": "Performance Metrics Collection" } } } The workflow for elastic scaling scheduled tasks is as follows: Triggered once per minute: 1. Get the current resource pool status; - Total number of active instances; - Number of free instances; - Number of instances used; - Number of hibernating instances; 2. Retrieve the current time period configuration; - Current time: 14:30; - Time period: 14:00-18:00 (afternoon peak); - Minimum number of active users: 10; - Maximum active users: 30; 3. Determine if expansion is needed; if (number of idle instances < 3 && total number of active instances < maximum value): Wake up N sleeping instances; 4. Determine if volume reduction is necessary; If (number of idle instances > 10 && lasts for 5 minutes && total number of active instances > minimum value): Sleep N idle instances; 5. Record scaling logs; - Scalability type: Expand / Shrink; - Number of extension channels: +3 / -5; - Triggering reason: Insufficient idle time / Excessive idle time; - Execution time: 2026-01-22 14:31:00.

[0093] The advantages of scheduled tasks include: Automation: No human intervention required; Timeliness: Responds to load changes within 1 minute; Smoothness: Avoids sudden scaling up and down; Predictability: Prepares in advance based on time-based strategies.

[0094] The current status includes key metrics such as instance count, distribution, and load. The status data structure is as follows: { "timestamp": "2026-01-22T14:30:00Z", "poolStatus": { "totalCapacity": 30 "activeInstances": 15 "sleepingInstances": 15, "instancesByState": { "IDLE": 2, "OCCUPIED": 13, "RECYCLING": 0, "SLEEPING": 15, "INIT": 0 } }, "currentTimeSlot": { "period": "14:00-18:00", "minActiveCount": 10, "maxActiveCount": 30 }, "concurrency": { "global": 13, "byScene": { "Smart Buildings": 8 Customer Service Center: 5 } }, "gpuMetrics": { "averageUtilization": 65, "averageTemperature": 58, "averageMemoryUsage": 75 } Key indicators are explained in Table 3: Table 3: Correspondence between Key Indicators and Their Explanations and Uses

[0095] Record detailed scaling operations for auditing, analysis, and optimization. The scaling log content is as follows: { "logId": "scale_20260122_143100_001", "timestamp": "2026-01-22T14:31:00Z", "operation": "scale_out", "trigger": { "type": "idle_insufficient", "condition": "idleCount<3", "actualValue": 2 }, "before": { "activeInstances": 28 "idleInstances": 2, "occupiedInstances": 26, "sleepingInstances": 2 }, "action": { "type": "wake_up", "count": 2, "instanceIds": ["inst_029", "inst_030"] }, "after": { "activeInstances": 30 "idleInstances": 4, "occupiedInstances": 26, "sleepingInstances": 0 }, "duration": 18.5, "result": "success" } Logs are used for the following purposes: Operations and maintenance monitoring: Real-time viewing of scaling operations, detection of abnormal scaling behavior, and alarm notifications.

[0096] Performance analysis: Statistical analysis of expansion frequency, analysis of reduction effect, and optimization of threshold configuration.

[0097] Cost optimization: Calculate resource utilization, evaluate the benefits of elastic scaling, and adjust time-based strategies.

[0098] Troubleshooting: Review historical operations to pinpoint performance bottlenecks and reproduce the problem scenario.

[0099] The time-period strategy configuration is shown in Table 4: Table 4: Time Strategy Configuration Correspondence Table

[0100] Minimum and maximum active counts refer to the number of instances in an active state. Active Instances: Instances in the IDLE, OCCUPIED, or RECYCLING states; models have been loaded into GPU memory; RTC rooms have been created; and services can be provided immediately or soon. Minimum Active Count: The minimum number of active instances guaranteed by the system. Even with many idle instances, the system will not scale down below this value to ensure basic responsiveness. Maximum Active Count: The maximum number of active instances allowed by the system. Even with insufficient idle instances, the system will not scale up beyond this value to prevent excessive resource consumption.

[0101] Total capacity = Active instances + Dormant instances; Active instances = IDLE + OCCUPIED + RECYCLING; Constraints: Minimum number of active instances ≤ Active instances ≤ Maximum number of active instances ≤ Total capacity.

[0102] Configuration example: Scenario 1: Small application.

[0103] { "totalCapacity": 10, "minActiveCount": 2, "maxActiveCount": 10 } Two channels remain active at night, while the other eight are dormant; during peak hours, up to 10 channels are active.

[0104] Scenario 2: Large-scale applications.

[0105] { "totalCapacity": 100, "timeSlots": [ { "period": "09:00-18:00", "minActiveCount": 30, "maxActiveCount": 80 }, { "period": "18:00-09:00", "minActiveCount": 5, "maxActiveCount": 20 } ] } During working hours: Maintain 30-80 active channels; during non-working hours: Maintain 5-20 active channels.

[0106] Dynamic adjustment strategy: Based on historical data and predictive models, dynamically adjust the minimum / maximum number of active users: #Monday morning rush hour: if (weekday == 1 and hour == 9): minActiveCount = 50 # Increase the minimum value maxActiveCount = 100 #weekend: if (weekday in [6, 7]): minActiveCount = 5 # Decrease the minimum value maxActiveCount = 30.

[0107] Furthermore, such as Figure 7 As shown in Table 5, the different states and descriptions are as follows: Table 5: Correspondence between instance status and description

[0108] Figure 7 This invention demonstrates the complete lifecycle state transition process of a digital human instance, the core of which is a state machine containing five key states. The specific transition process is as follows: 1. Initialization (INIT).

[0109] Starting point: When the system starts up or performs an expansion operation, a new digital human instance is created.

[0110] Process: The system allocates independent GPU resources to the instance, loads the AI ​​model into the GPU memory, creates the corresponding RTC room, and renders the standby screen.

[0111] Next state: After all the above initialization operations are completed, the instance state changes from INIT to IDLE.

[0112] 2. Idle / Ready for Service.

[0113] Note: The instance has completed its warm-up and is in a "hot ready" state. It has been stored in the idle resource pool and is waiting for user requests.

[0114] circulation: When a user session request is received and the instance is successfully allocated, the state changes from IDLE to OCCUPIED.

[0115] When the system decides to scale down based on an elastic scaling strategy (such as during off-peak business periods), the state changes from IDLE to SLEEPING.

[0116] 3. Occupied service.

[0117] Note: The instance has been assigned to a specific user and is currently in the process of real-time audio and video interaction.

[0118] Transition: When a user session ends (the user leaves), the state changes from OCCUPIED to RECYCLING.

[0119] 4. Recycling.

[0120] Note: This refers to the cleanup of the transition state after the session ends.

[0121] Process: In this state, the system cleans up the instance's LLM dialogue history, knowledge base context, user binding information and other session data, and releases the concurrency quota.

[0122] Next state: After the cleanup operation is completed, the state changes from RECYCLING to IDLE, and the instance returns to the idle pool, waiting for the next allocation (this is a key step in realizing resource reuse).

[0123] 5. Sleeping.

[0124] Note: This is a low-power state entered under low load to conserve GPU resources. The instance's model may be partially unloaded, GPU computation may be paused, but core metadata is retained.

[0125] Transition: When the system decides to expand its capacity based on the elastic scaling strategy (such as during peak business periods), the state changes from SLEEPING to INIT, and the initialization process is re-executed to wake up and restore it to the ready state.

[0126] This state machine clearly defines the entire lifecycle of an instance from creation, service, recycling to hibernation. It achieves core resource reuse through the loop of IDLE -> OCCUPIED -> RECYCLING -> IDLE, and realizes elastic scaling through the transition of IDLE <-> SLEEPING. It is the core logic of the entire scheduling method.

[0127] The achieved results are shown in Table 6: Table 6: Comparison of the effects of prior art and the present invention on different comparison items

[0128] The startup delay breakdown is shown in Table 7: Table 7: Comparison of Time Consumption of Prior Technology and This Invention for Different Stages of Startup Delay Decomposition

[0129] Table 8 shows the GPU resource consumption reference for a single-channel digital human (based on the open-source MuseTalk solution): Table 8: Correspondence between unused components and video memory usage

[0130] The benefits of flexible scaling are shown in Table 9: Table 9: Comparison of Existing Technology and the Invention Regarding the Benefits of Elastic Stretchability

[0131] The key innovations of this invention are as follows: 1. A preheating mechanism for keeping GPU memory resident.

[0132] The deep learning model is loaded into GPU memory once during service startup and never unloaded afterward. When a user makes a request, there is no need to wait for the model to load; a ready instance is allocated directly, reducing startup latency from 5-10 seconds to the second level.

[0133] Key technical features: (1) Model preloading: The model is loaded into the GPU memory when the service starts; (2) RTC Room Pre-creation: Digital humans join the room in advance and push the standby screen; (3) Instance reuse: After the session ends, the context is cleaned up but the model is retained, and the instance is returned to the pool for reuse.

[0134] 2. Load-aware elastic scaling strategy.

[0135] The number of active instances is dynamically adjusted based on real-time load. When there are insufficient idle instances, the instance pool is woken up to expand the pool. When there are too many idle instances, the instance pool is shrunk to release GPU resources.

[0136] Key technical features: (1) Expansion trigger: Automatically wake up dormant instances when the number of idle instances is lower than the threshold; (2) Shrinkage trigger: When the number of idle instances exceeds the threshold and continues for a certain period of time, the excess instances will be put to sleep; (3) Time period strategy: Configure different minimum / maximum active numbers for different time periods.

[0137] 3. Session context isolation and quick cleanup.

[0138] The session contexts of different users are completely isolated. When a user leaves, the LLM conversation history, knowledge base retrieval context, etc. are quickly cleaned up to ensure that the next user gets a clean instance.

[0139] Key technical features: (1) Context isolation: Each session has its own independent dialogue history and state; (2) Quick cleanup: The context is cleaned up in milliseconds when the user leaves; (3) Seamless switching: It can be assigned to a new user immediately after cleaning.

[0140] Possible design changes or modifications to this invention are as follows: 1. Predictive expansion.

[0141] Predict future traffic based on historical data and expand capacity in advance. For example, preheat instances at 7:45 am on weekdays so that they are ready when the peak arrives at 8 am.

[0142] 2. Staged preheating.

[0143] Different preheating depths are used based on the predicted probability: (1) High probability: Full warm-up (model loading + RTC room creation + standby screen); (2) Medium probability: partial preheating (only loading the model, and creating the room when the user triggers it); (3) Low probability: Start-up on demand.

[0144] 3. Affinity scheduling.

[0145] The same user will be given priority to be assigned previously used instances, which can retain some personalized configurations and improve the user experience.

[0146] 4. Multiple image pools.

[0147] Resource pools are established for each digital human avatar, and instances of different avatars are managed independently to avoid additional costs when switching avatars.

[0148] This invention achieves pre-warming, dynamic reuse, and intelligent scheduling of digital human instances through GPU memory resident and elastic scaling strategies. When the service starts, the model is pre-loaded into GPU memory and an RTC room is created. When a user makes a request, an idle instance is directly allocated, achieving a response time within seconds. After a session ends, instances are recycled and reused instead of being destroyed. The system dynamically scales up and down based on business load, releasing GPU resources by hibernating instances during off-peak periods and waking up instances for expansion during peak periods. Through elastic scaling, this invention keeps most instances hibernating during off-peak periods, reducing the average GPU temperature to 50-60°C, effectively extending hardware lifespan.

[0149] The core effects of the overall technical solution of this invention are as follows: (1) Achieve millisecond-level real-time response and significantly improve user experience: By pre-completing GPU resource allocation, model loading and RTC room creation when the service starts, the digital human instance is in a "hot ready" state. When a user makes a request, an idle instance is directly allocated, skipping the 5-10 second cold start process in the traditional solution, and achieving session opening in seconds (or even milliseconds), completely eliminating user waiting delay.

[0150] (2) Achieving efficient resource reuse and extended hardware lifespan: A digital human instance resource pool is established. After the session ends, the instance is cleaned up and returned to the idle pool for recycling, avoiding the overhead of frequent resource creation and destruction in the traditional one-to-one mode, and greatly improving the utilization rate of expensive resources such as GPUs. At the same time, combined with the elastic scaling strategy, idle instances are put into hibernation during off-peak periods to release GPU load, avoiding the long-term high-temperature full load (80-85°C) operation of GPUs caused by the traditional resident solution, thereby restoring the lifespan of GPUs from 2-3 years under severe wear and tear to near normal lifespan.

[0151] (3) Provide intelligent elastic scaling capabilities to ensure service stability and economy: Based on real-time monitoring of business traffic, dynamically adjust the size of the resource pool: automatically expand during peak periods (wake up dormant instances) to ensure service capacity and stability; automatically shrink during off-peak periods (put instances into dormancy) to release resources and reduce energy consumption. This mechanism enables the system to smoothly cope with traffic fluctuations, achieving optimal resource cost allocation while ensuring service quality.

[0152] Furthermore, such as Figure 2 As shown, based on the above-mentioned GPU memory-resident digital human service elastic scaling scheduling method, this invention also provides a GPU memory-resident digital human service elastic scaling scheduling system, wherein the GPU memory-resident digital human service elastic scaling scheduling system includes: a scheduling server, a GPU resource pool, and a user terminal (i.e., Figure 2 (users in the middle) When the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads an AI model into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen to put the digital human instance into a preheating and ready state. When a user's session start request is received, the scheduling server allocates an idle digital human instance in a preheating-ready state from the GPU resource pool and returns the RTC room information corresponding to the idle digital human instance to the user client. The user joins a room based on the RTC room information and interacts with the digital human in real time. After the session ends, the scheduling server cleans up the context of the digital human instance, resets the state of the digital human instance to idle, and then reclaims it into the GPU resource pool.

[0153] In summary, this invention provides a method and system for elastic scaling scheduling of a GPU-resident digital human service. The method includes: upon service startup, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads an AI model into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen, putting the digital human instance in a preheating-ready state; when a user's session startup request is received, an idle digital human instance in the preheating-ready state is allocated from the GPU resource pool, and the RTC room information corresponding to the idle digital human instance is returned to the user; the user joins the room based on the RTC room information and interacts with the digital human in real time; after the session ends, the context of the digital human instance is cleaned up, and the state of the digital human instance is reset to idle and then recycled back to the GPU resource pool. This invention achieves preheating-readiness, dynamic reuse, and intelligent scheduling of digital human instances through GPU memory resident and elastic scaling strategies.

[0154] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0155] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0156] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for elastic scaling scheduling of digital human services with GPU memory resident, characterized in that, The method for elastic scaling scheduling of the GPU memory-resident digital human service includes: When the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads AI models into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen, so that the digital human instance is in a preheating and ready state. When a user's session start request is received, an idle digital human instance in a preheating-ready state is allocated from the GPU resource pool, and the RTC room information corresponding to the idle digital human instance is returned to the user. The user joins a room based on the RTC room information and interacts with the digital human in real time. After the session ends, the context of the digital human instance is cleaned up, and the state of the digital human instance is reset to idle and then recycled back to the GPU resource pool.

2. The method for elastic scaling and scheduling of digital human services with persistent GPU memory resident according to claim 1, characterized in that, When the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads an AI model into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen, putting the digital human instance into a preheating-ready state. Specifically, this includes: When the service starts, the scheduling server reads the preset initial instance count configuration of the GPU resource pool; The scheduling server determines whether the number of currently initialized digital human instances has reached the initial number of instances; If the target is not met, the scheduling server will allocate independent GPU resources to the new digital human instance, load the required AI model into the GPU memory, create the corresponding real-time communication (RTC) room, and render the default standby screen. Mark the initialized digital human instance as ready and add it to the idle instance pool of the GPU resource pool until the number of initialized instances reaches the preset initial instance number.

3. The method for elastic scaling and scheduling of digital human services with persistent GPU memory resident according to claim 2, characterized in that, The AI ​​model includes an audio feature extraction model, a lip-sync generation model, an image encoding / decoding model, a face detection model, and an expression-driven model.

4. The method for elastic scaling and scheduling of digital human services with GPU memory resident as described in claim 1, characterized in that, When a user's session initiation request is received, an idle digital human instance in a preheated and ready state is allocated from the GPU resource pool, and the RTC room information corresponding to the idle digital human instance is returned to the user client. Specifically, this includes: When the scheduling server receives a user's session start request, it determines whether the current number of active sessions in the system has reached the set maximum concurrency limit. If the limit is not exceeded, then find a digital human instance with an IDLE state from the idle instance pool of the GPU resource pool; If an idle instance exists, change the status of the idle instance from IDLE to OCCUPIED, and bind the idle instance to the current user information; Based on the scenario information carried in the session initiation request, update the prompt words of the Large Language Model (LLM), bind the corresponding knowledge base, and set the timbre parameters of the TTS speech synthesis and ASR speech recognition. Generate the corresponding RoomId for the Real-Time Communication (RTC) room and the temporary Token required for the user to join the RTC room, and return the RoomId and Token to the user. If there are no available instances in the idle instance pool, asynchronous scaling is initiated, and the current session request is added to the waiting queue.

5. The method for elastic scaling and scheduling of digital human services with GPU memory resident as described in claim 1, characterized in that, The asynchronous expansion includes: When a user request arrives, if there are insufficient available instances, a response is returned immediately. The background process initiates a scaling task, waking up dormant instances or creating new ones. Once the scaling is complete, a notification is pushed out, and users can join the room after receiving the notification.

6. The method for elastic scaling scheduling of digital human services with GPU memory resident as described in claim 1, characterized in that, The user client joins a room based on the RTC room information and interacts with the digital human in real time. Upon detecting the end of the session, the digital human instance undergoes context cleanup, and its state is reset to idle before being recycled back to the GPU resource pool. Specifically, this includes: When the user session is detected to have ended, the scheduling server changes the status of the digital human instance to RECYCLING. Perform a context cleanup operation to clear the large language model (LLM) dialogue history and knowledge base retrieval context of the digital human instance, and delete the bound user information; Perform resource quota release operations to reduce the global concurrent session count, scene concurrent count, and device concurrent count, so as to allow new sessions to use the released quota; Control the rendering of the digital human instance and switch it to a preset standby screen to maintain visual activity; The state of the digital human instance is reset from RECYCLING to IDLE, and the digital human instance is recycled to the idle instance pool of the GPU resource pool for allocation and reuse in subsequent session requests.

7. The method for elastic scaling scheduling of digital human services with GPU memory resident as described in claim 1, characterized in that, The method for elastic scaling and scheduling of digital human services with GPU memory resident also includes: The scheduling server starts a scheduled task to periodically obtain the current status of the GPU resource pool; Based on the current status, determine whether the number of idle instances is less than a preset threshold; If so, wake up the specified number of dormant instances from the dormant instance pool and add them to the idle instance pool to complete the expansion; If not, a specified number of idle instances will be transferred to the hibernation pool to release the occupied GPU resources and complete the scaling down. The scheduling server records detailed information about each scaling operation in the log.

8. The method for elastic scaling scheduling of digital human services with persistent GPU memory resident according to claim 7, characterized in that, The scheduled task is a background task that the system executes automatically at fixed time intervals to monitor the status of the resource pool and trigger elastic scaling.

9. The method for elastic scaling scheduling of digital human services with GPU memory resident as described in claim 7, characterized in that, The logs are used for operation and maintenance monitoring, performance analysis, cost optimization, and problem troubleshooting. The operation and maintenance monitoring includes real-time viewing of scaling operations, detection of abnormal scaling behavior, and alarm notifications; The performance analysis includes statistical analysis of expansion frequency, analysis of reduction effect, and optimization of threshold configuration; The cost optimization includes calculating resource utilization, evaluating the benefits of elastic scaling, and adjusting time-period strategies. The troubleshooting process includes tracing back historical operations, identifying performance bottlenecks, and reproducing the problem scenario.

10. A GPU memory-resident digital human service elastic scaling scheduling system, characterized in that, The GPU memory-resident digital human service elastic scaling scheduling system includes: a scheduling server, a GPU resource pool, and a user terminal; When the service starts, the scheduling server initializes multiple digital human instances in the GPU resource pool, preloads an AI model into the GPU memory for each digital human instance, creates an independent real-time communication (RTC) room, and renders a standby screen to put the digital human instance into a preheating and ready state. When a user's session start request is received, the scheduling server allocates an idle digital human instance in a preheating-ready state from the GPU resource pool and returns the RTC room information corresponding to the idle digital human instance to the user client. The user joins a room based on the RTC room information and interacts with the digital human in real time. After the session ends, the scheduling server cleans up the context of the digital human instance, resets the state of the digital human instance to idle, and then reclaims it into the GPU resource pool.

Images