Multi-modal AI intelligent human edge computing intelligent server and dynamic load balancing scheduling method

Abstract

The application belongs to the technical field of edge computing dynamic load balancing, and discloses a multi-modal AI intelligent person edge computing intelligent server and a dynamic load balancing scheduling method, which comprises a multi-modal task arrangement module, a heterogeneous resource sensing module, an interactive load state prediction module, a candidate scheduling evaluation module, a whole-process feedback learning module and a scheduling execution and continuous service module. Through splitting multi-modal interactive tasks, constructing task dependency relationships, generating interactive load state occupation coefficients, predicting the execution delay and resource occupation of candidate scheduling actions, and combining whole-process execution feedback to dynamically update the scheduling strategy, fine allocation and cross-node collaborative scheduling of heterogeneous computing resources are realized, voice response delay, video frame drop and video memory overflow risk are reduced, and the real-time interaction quality, resource utilization and continuous service capability of the multi-modal AI intelligent person are improved.

Description

Technical Field

[0001] This invention belongs to the field of edge computing dynamic load balancing technology, specifically relating to a multimodal AI intelligent human edge computing intelligent server and a dynamic load balancing scheduling method. Background Technology

[0002] Multimodal AI intelligent humans in shopping malls, exhibition halls, parks, and business halls need to process user voice, image, text, video, and digital human motion data simultaneously. A single interaction typically generates tasks such as speech recognition, image recognition, target tracking, multimodal semantic reasoning, action generation, speech synthesis, and video rendering in sequence. Different tasks have significant differences in response time, computational load, memory requirements, model accuracy, and dependencies. When multiple AI intelligent human terminals are accessed concurrently, the edge computing server also needs to allocate tasks in real time among the central processing unit, graphics processing unit, neural network processor, different model replicas, and multiple edge computing nodes. Therefore, it is necessary to establish a dynamic load balancing scheduling method that can adapt to changes in the multimodal task structure and heterogeneous resource load.

[0003] (1) Existing edge computing load balancing methods usually treat an AI intelligent human interaction request as a whole task for node allocation. The execution position is mainly selected based on processor utilization, queue length and whether the current node is idle. It does not fully express the pre-dependent relationships between speech recognition results, image recognition results, multimodal semantic reasoning results, action generation results, speech synthesis results and video rendering results. It also does not distinguish the different response requirements corresponding to the first speech response, semantic reasoning, video frame rate and high-priority interaction. When the pre-task scheduling is unreasonable, subsequent tasks are likely to wait continuously, resulting in slow speech response, asynchronous digital human actions and stuttering rendering screen.

[0004] (2) Existing dynamic scheduling methods usually use fixed resource thresholds and fixed scoring weights to judge the server load status. They are difficult to adapt to the concentrated arrival of low-latency voice tasks, the increase in throughput of visual tasks, the increase in memory usage of large model inference, the continuous increase in video rendering load, and the load distribution changes caused by sudden concurrent access. At the same time, after the task is completed, feedback is mostly based on whether it is successful and whether it times out. The first response time, peak memory usage, model output confidence, number of voice stutters, number of video frame loss, number of task retries, and waiting time of subsequent tasks are not associated with the preceding scheduling actions. As a result, the scheduling strategy cannot accurately identify the actual impact of a certain resource allocation on the complete interaction chain, and it is difficult to take into account the real-time interaction quality, the utilization rate of heterogeneous resources, and the continuous service capability of the server. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing edge server load balancing methods, which allocate tasks based solely on processor utilization, queue length, and task timeout. These methods struggle to express the pre-dependencies between speech recognition, visual recognition, semantic reasoning, action generation, speech synthesis, and video rendering tasks in multimodal AI human interaction requests. Furthermore, they cannot adapt to the continuous changes in the proportion of different task types, memory usage, task arrival rates, and response times. Additionally, they lack the ability to continuously adjust scheduling strategies based on initial response time, peak memory usage, speech stuttering, video frame drops, model confidence, and subsequent task delays. This invention provides a multimodal AI human edge computing intelligent server and a dynamic load balancing scheduling method.

[0006] The technical solution adopted in this invention is as follows: a multimodal AI intelligent human edge computing intelligent server and dynamic load balancing scheduling method, including a multimodal task queue construction step, a heterogeneous resource status acquisition step, an interactive load status occupancy calculation step, a candidate scheduling load prediction step, a rich feedback scheduling scoring step, a scheduling execution step, and a full-process feedback update step.

[0007] The multimodal task queue construction step receives interactive requests uploaded by multiple AI intelligent human terminals, breaks down the interactive requests into speech recognition tasks, image recognition tasks, target tracking tasks, multimodal semantic reasoning tasks, action generation tasks, speech synthesis tasks, and video rendering tasks, establishes the pre-dependencies between each task, and records the request number, terminal number, task type, input data volume, target response time limit, terminal priority, and number of subsequent tasks for each task.

[0008] The heterogeneous resource status acquisition steps obtain the utilization rates of the central processing unit, graphics processing unit, and neural network processor in multiple edge computing nodes according to a unified acquisition time scale, and obtain the remaining amount of video memory, the remaining amount of running memory, the remaining network bandwidth between nodes, the status of the model instance queue, the number of concurrent model instances, the task arrival rate, the historical execution time, the historical latency residual, and the historical video memory residual, and construct the task resource input vector.

[0009] The interactive load state occupancy calculation steps calculate the interactive load state occupancy coefficients corresponding to the low latency dominant state of speech, the high throughput dominant state of vision, the memory dominant state of semantic reasoning, the rendering frame rate dominant state, and the burst concurrency congestion state based on the proportion of task types, the task arrival rate, the task response time limit, the remaining amount of heterogeneous computing resources, the status of model instance queues, and historical prediction residuals. This allows multiple interactive load states to continuously overlap and dynamically migrate within the same resource status window.

[0010] The candidate scheduling load prediction step generates multiple candidate scheduling actions for each task to be scheduled. Each candidate scheduling action includes the target edge computing node, target computing unit, target model instance, model replica accuracy, and cross-node transmission method. Based on the current resource status, interactive load state occupancy coefficient, and task characteristics, the execution latency, peak memory usage, queue waiting time, task failure probability, video frame rate loss, voice stuttering probability, model accuracy loss, and energy consumption increment corresponding to each candidate scheduling action are predicted.

[0011] The rich feedback scheduling scoring step dynamically adjusts the degree of influence of execution latency, peak video memory, queue waiting time, task failure probability, video frame rate loss, voice stuttering probability, model accuracy loss, and energy consumption increment on the scheduling score based on the occupancy coefficient of each interactive load state. It also calculates the predicted latency increment of the current candidate scheduling action on subsequent tasks based on the task dependency graph and generates a comprehensive scheduling score by combining terminal priority.

[0012] The scheduling execution steps eliminate candidate scheduling actions that fail to meet the target response time limit, have peak prediction memory exceeding the remaining memory, are incompatible with the model instance and task input, or have insufficient network bandwidth. From the remaining candidate scheduling actions, the scheduling action with the lowest comprehensive scheduling score is selected, and the task to be scheduled is sent to the corresponding edge computing node, computing unit and model instance. If there are no executable candidate scheduling actions, the following actions are performed: reducing the accuracy of the model replica, adjusting the rendering resolution, delaying low-priority tasks and cross-node collaborative scheduling.

[0013] The entire process feedback update step collects the actual execution delay, first response time, peak memory usage, model output confidence, task timeout status, number of video frame losses, number of audio stutters, number of task retries, and user interaction interruption status after the task is completed. It forms a rich feedback scheduling trajectory according to the task dependency graph and inversely associates the delay, stuttering, frame drops, and retry feedback generated by subsequent tasks with the corresponding preceding scheduling actions.

[0014] The full-process feedback update step generates a full-process feedback reference distribution based on the actual scheduling trajectory and historical candidate scheduling actions. It performs distribution correction on the scheduling strategy based on the actual resource status accessed by the current scheduling strategy, and updates the interactive load state prediction parameters, candidate scheduling load prediction parameters, and scheduling strategy parameters based on the rich feedback scheduling trajectory.

[0015] The edge computing intelligent server includes a multimodal task orchestration module, a heterogeneous resource perception module, an interactive load state prediction module, a candidate scheduling evaluation module, a full-process feedback learning module, and a scheduling execution and continuous service module. The multimodal task orchestration module performs multimodal task decomposition and task dependency graph construction; the heterogeneous resource perception module performs heterogeneous resource status acquisition; the interactive load state prediction module generates interactive load state occupancy coefficients; the candidate scheduling evaluation module completes candidate scheduling load prediction and rich feedback scheduling scoring; the full-process feedback learning module updates the scheduling strategy based on the rich feedback scheduling trajectory; and the scheduling execution and continuous service module completes task allocation, task migration, and continuous service in the state of node disconnection.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) By using task dependency graphs and fine-grained subtask orchestration, AI intelligent human interaction requests are split into speech recognition, image recognition, target tracking, multimodal semantic reasoning, action generation, speech synthesis and video rendering tasks, and response time limits, priorities and pre-dependent relationships are recorded respectively, so as to realize the independent scheduling of key tasks and subsequent tasks in the multimodal interaction chain, and solve the problems of excessive overall request scheduling granularity and the inability of the delay of the preceding task to be passed on to the subsequent interaction process.

[0017] (2) By using interactive load state soft occupancy and heterogeneous resource probability prediction, multiple interactive load state occupancy coefficients are generated based on task type ratio, resource remaining amount, queue status and historical prediction residuals. The execution latency, memory peak and interactive quality loss corresponding to different computing nodes, computing units, model instances and model replicas are predicted to realize continuous state recognition and candidate scheduling comparison under load distribution change conditions, and solve the problem that fixed threshold scheduling cannot adapt to the continuous drift of speech, vision, semantic reasoning and rendering load.

[0018] (3) By using rich feedback throughout the process and the reverse allocation of the impact of subsequent tasks, the first response time, peak memory usage, voice stuttering, video frame drops, model confidence, task retry and user interaction interruption status are combined into a rich feedback scheduling trajectory. The feedback of subsequent tasks is then linked to the preceding scheduling actions, so that the scheduling strategy is continuously updated according to the complete interactive experience. This solves the problem that setting a single feedback based on whether the task is completed cannot identify the actual impact of the preceding resource allocation on the quality of subsequent interactions. Attached Figure Description

[0019] The accompanying drawings are provided to further understand the present invention and form part of the specification. They are used together with the embodiments of the present invention to explain the invention and do not constitute a limitation thereof.

[0020] Figure 1 This is an overall architecture diagram of the multimodal AI intelligent human edge computing intelligent server and dynamic load balancing scheduling method of the present invention. Detailed Implementation

[0021] Example 1, see Figure 1 This invention provides a multimodal AI intelligent human edge computing intelligent server and a dynamic load balancing scheduling method. The edge computing intelligent server includes a multimodal task orchestration module, a heterogeneous resource perception module, an interactive load state prediction module, a candidate scheduling evaluation module, a full-process feedback learning module, and a scheduling execution and continuous service module.

[0022] This embodiment is applied to an AI-powered intelligent human shopping guide project in a commercial complex. The commercial complex is equipped with a total of 32 AI intelligent human terminals, including 20 fixed shopping guide terminals, 8 activity display terminals and 4 mobile tour terminals. Each AI intelligent human terminal can receive user voice, user images, user input text and terminal camera video, and output response text, synthesized speech, digital human actions and real-time rendered images.

[0023] The project sets up three edge computing nodes in the data center of the commercial complex. The three edge computing nodes are connected via a 25Gbps industrial Ethernet. Each edge computing node includes a dual 32-core central processing unit, 256GB of RAM, two 48GB graphics processors, two neural network processors, local solid-state storage, and a dual-port network adapter. The three edge computing nodes are referred to as the first edge computing node, the second edge computing node, and the third edge computing node.

[0024] The first edge computing node deploys speech recognition model instances, speech synthesis model instances, multimodal semantic reasoning model instances, and digital human rendering model instances. The second edge computing node deploys image recognition model instances, target tracking model instances, multimodal semantic reasoning model instances, and digital human rendering model instances. The third edge computing node deploys spare instances of various models, compact model copies, and task migration receiving areas.

[0025] The edge computing intelligent server executes the following steps: multimodal task queue construction, heterogeneous resource status acquisition, interactive load status occupancy calculation, candidate scheduling load prediction, rich feedback scheduling scoring, scheduling execution, and full-process feedback update.

[0026] The multimodal task queue construction step receives interactive requests uploaded by AI intelligent human terminals, breaks down the interactive requests into speech recognition tasks, image recognition tasks, target tracking tasks, multimodal semantic reasoning tasks, action generation tasks, speech synthesis tasks, and video rendering tasks, and records the request number, terminal number, task type, input data volume, target response time limit, terminal priority, and prerequisite dependencies for each task.

[0027] The heterogeneous resource status acquisition steps read the CPU core utilization, GPU computing utilization, neural network processor utilization, GPU video memory utilization, RAM utilization, inter-node network bandwidth utilization, model instance queue length, model instance concurrency, and historical execution time of various tasks for each edge computing node according to a unified acquisition time stamp.

[0028] The interactive load state occupancy calculation steps generate interactive load state occupancy coefficients corresponding to the following states: voice low latency dominant state, vision high throughput dominant state, semantic reasoning memory dominant state, rendering frame rate dominant state, and sudden concurrency congestion state, based on the arrival rate of various tasks, the proportion of task types, the remaining amount of graphics processor memory, the remaining computing power of neural network processor, the length of model instance queue, historical latency residuals, and terminal priority.

[0029] The candidate scheduling load prediction step generates multiple candidate scheduling actions for the task to be scheduled. Each candidate scheduling action includes the target edge computing node, target computing unit, target model instance, model replica accuracy, and cross-node transmission method. It also predicts the execution latency, peak memory usage, queue waiting time, task failure probability, video frame rate loss, voice stuttering probability, model accuracy loss, and energy consumption change corresponding to each candidate scheduling action.

[0030] The rich feedback scheduling scoring step calculates the comprehensive scheduling score of each candidate scheduling action based on the interactive load state occupancy coefficient, candidate scheduling load prediction results, task response time limit, terminal priority, and the impact of subsequent tasks, and eliminates candidate scheduling actions that do not meet resource constraints and response time limits.

[0031] The scheduling execution steps select the candidate scheduling action with the lowest comprehensive scheduling score, send the task to be scheduled to the corresponding edge computing node, computing unit and model instance, and trigger the subsequent task to enter the schedulable state according to the task's pre-dependency relationship.

[0032] The full-process feedback update step collects actual execution latency, first response time, peak memory usage, model output confidence, task timeout status, number of video frame losses, number of audio stutters, number of task retries, and user interaction interruption status after the task is completed, forming rich feedback scheduling samples, and updating the interactive load state prediction parameters, candidate scheduling load prediction parameters, and scheduling strategy parameters based on the rich feedback scheduling samples.

[0033] Through the above processing, this method decomposes multimodal interaction requests into multiple subtasks with pre-dependent relationships, predicts the execution results of candidate scheduling actions based on the continuously changing task structure and heterogeneous resource status, and inversely correlates the multidimensional feedback during the execution of each subtask with the preceding scheduling actions. This solves the problem that traditional load balancing methods, which only allocate nodes based on the CPU utilization and whether the task has timed out, cannot adapt to real-time interaction tasks of multimodal AI intelligent humans.

[0034] Example 2: This example is based on all the above examples. The multimodal task orchestration module and the multimodal task queue construction steps are specifically executed in the following manner.

[0035] When an AI-powered intelligent human terminal begins a user interaction, it generates a unique request number and sends the voice stream, camera image stream, input text, terminal location, current digital human action status, and user interaction priority to the edge computing intelligent server.

[0036] The multimodal task orchestration module establishes a task dependency graph based on the input data type. Speech recognition and image recognition tasks can be executed in parallel. The target tracking task is executed after consecutive video frames arrive. The multimodal semantic reasoning task is executed after the speech recognition results, image recognition results, and target tracking results meet the input completeness condition. The action generation task and speech synthesis task are executed in parallel after the multimodal semantic reasoning task is completed. The video rendering task is executed after the action generation results and speech synthesis time markers arrive.

[0037] The speech recognition task is divided into a first-segment recognition task and a complete recognition task. The first-segment recognition task enters the scheduling queue after receiving at least 320ms of speech data. The complete recognition task continuously receives subsequent speech data and updates the recognized text, so that the multimodal semantic reasoning task can receive the first segment of semantic information before the complete speech ends.

[0038] Image recognition tasks are performed based on keyframes uploaded by the AI ​​intelligent human terminal, while target tracking tasks are performed based on continuous video frames. The target tracking task acquisition frequency for the activity display terminal is set to 15 frames per second, the target tracking task acquisition frequency for the fixed shopping guide terminal is set to 8 frames per second, and the target tracking task acquisition frequency for the mobile tour guide terminal is set to 12 frames per second.

[0039] The initial recognition response time for standard shopping guide interaction is set to 180ms, the multimodal semantic reasoning response time is set to 450ms, the initial output time for speech synthesis is set to 220ms, the complete end-to-end response time is set to 900ms, and the video rendering frame rate must not be less than 25 frames per second.

[0040] The AI-powered human interaction for lost and found, assistance requests, and safe evacuation is set as a high-priority interaction. The initial recognition response time for high-priority interactions is set to 120ms, the multimodal semantic reasoning response time is set to 300ms, and the complete end-to-end response time is set to 600ms.

[0041] Each task records the expected amount of input data, allowed waiting time, allowed model accuracy loss, allowed cross-node transmission time, and the number of successor tasks. Multimodal semantic reasoning tasks with multiple successor tasks are set as critical chain tasks, and critical chain tasks have a higher impact on successor tasks in scheduling scores.

[0042] The task queue is established separately for edge computing nodes and model instances. Only one execution copy is retained for each task before it enters the model instance queue. When a task migrates across nodes, the original execution copy is locked first, and then the input state is transmitted and the target execution copy is established. The original execution copy is released after the target execution copy confirms receipt.

[0043] The speech recognition results, image recognition results, target tracking results, multimodal semantic reasoning results, action generation results, speech synthesis results, and video rendering results in the same request all carry a unified request number and task sequence number to avoid the intermediate results of different AI intelligent human terminals being incorrectly combined.

[0044] Through the above processing, the multimodal task queue construction steps can split and organize tasks according to the actual dependencies in the AI ​​intelligent human interaction process, solving the problem of allocating complete interaction requests that include voice, vision, semantics and rendering processing as a single whole task, resulting in excessively large task granularity and the inability to prioritize the scheduling of key sub-tasks.

[0045] Example 3: Based on all the above examples, the heterogeneous resource perception module, the heterogeneous resource status acquisition step, and the interactive load state occupancy calculation step are specifically executed in the following manner.

[0046] The heterogeneous resource awareness module reads the resource status of three edge computing nodes at a 100ms acquisition interval and uses 50 consecutive acquisition moments to form a 5s resource status window. The resource status window includes the utilization rate of each computing unit, the remaining amount of video memory, the remaining amount of running memory, the remaining network bandwidth, the length of the model instance queue, and the task arrival rate.

[0047] The CPU core utilization rate is determined based on the actual number of cores used by the model instance and the core running time. The GPU computing utilization rate is determined based on the computing core activity time. The neural network processor utilization rate is determined based on the inference engine activity time. The video memory usage includes the model weight usage, inference cache usage, batch processing input usage, and intermediate result usage waiting to be released.

[0048] The model instance queue length records not only the number of waiting tasks, but also the expected computational cost, expected GPU memory increment, and earliest response time of the waiting tasks, thus avoiding multiple short speech tasks being processed with the same queue length as a single large-scale semantic reasoning task.

[0049] The heterogeneous resource awareness module generates historical latency residuals based on the difference between the actual execution time of the most recent task and the corresponding predicted execution time, and generates historical memory residuals based on the difference between the actual peak memory usage of the most recent task and the corresponding predicted peak memory usage. The residual sequences are saved according to task type and model instance.

[0050] The interactive load state prediction module sets up prototypes for low-latency speech, high-throughput vision, memory-dominated semantic reasoning, frame rate-dominated rendering, and burst concurrency.

[0051] The prototype for a low-latency speech-dominated state is established based on the proportion of speech recognition and speech synthesis tasks, the average response time of speech tasks, and the length of the speech model instance queue. The prototype for a high-throughput vision-dominated state is established based on the proportion of image recognition and target tracking tasks, the video frame arrival rate, and the batch processing scale of the vision model. The prototype for a memory-dominated semantic reasoning state is established based on the proportion of multimodal semantic reasoning tasks, the length of the prompt data, and the remaining amount of graphics processor memory.

[0052] The prototype for the rendering frame rate-dominated state is established based on the number of active display terminals, the arrival rate of video rendering tasks, the current output frame rate, and the length of the graphics processor rendering queue. The prototype for the sudden congestion state is established based on the increase in the arrival rate of all tasks, the degree of synchronous growth of multiple model instance queues, the cross-node bandwidth usage, and the number of high-priority interactions.

[0053] The interactive load state occupancy calculation steps are as follows: calculate the interactive load state occupancy coefficient corresponding to the r-th interactive load state for the k-th resource status window according to Formula 1:

[0054] In Formula 1, This represents the interaction load occupancy coefficient corresponding to the r-th interaction load state in the k-th resource state window. r=1 indicates a low-latency speech-dominated state, r=2 indicates a high-throughput vision-dominated state, r=3 indicates a memory-dominated semantic inference state, r=4 indicates a rendering frame rate-dominated state, and r=5 indicates a burst concurrency congestion state. This represents the task resource input vector corresponding to the k-th resource status window. This represents the gate value output by the state gating unit for the r-th interactive load state. This represents the state deviation between the task resource input vector and the r-th interactive load state prototype. Indicates the state deviation suppression coefficient. This represents the state continuity coefficient between adjacent resource state windows. This represents the interaction load occupancy coefficient corresponding to the r-th interaction load state in the previous resource status window.

[0055] The sum of the occupancy coefficients of the five interactive load states is 1. The interactive load state prediction module does not force the current edge server load to be divided into a single state. When the commercial complex activity starts, it can simultaneously form the effective occupancy coefficients corresponding to the visual high throughput dominant state, the rendering frame rate dominant state, and the sudden concurrency congestion state.

[0056] When the occupancy coefficient of the low-latency dominant state for voice increases, the candidate scheduling evaluation module improves the role of first response time and voice stutter probability in scheduling score. When the occupancy coefficient of the high-throughput dominant state for vision increases, it improves the role of batch processing throughput and graphics processor queue length.

[0057] The extent to which the memory usage coefficient of semantic reasoning increases the peak memory usage and model instance switching cost; the extent to which the memory usage coefficient of rendering frame rate increases the video frame rate loss and rendering queue waiting time; and the extent to which the memory usage coefficient of sudden concurrency increases the cross-node transmission capacity and task failure probability.

[0058] Regarding parameter adjustments: The first step is to update the interactive load state prototype after the commercial complex completes one business day operation, based on the actual task ratio and resource usage results for each time period. The interactive load state prototype is updated during low task periods.

[0059] The second step is to increase the latency uncertainty of the corresponding model instance when the historical latency residuals in three consecutive resource status windows all exceed 15% of the historical execution time of the corresponding model instance, without directly increasing the concurrency limit of the model instance.

[0060] Third, when the maximum value of the five interactive load state occupancy coefficients is less than 0.45, the current resource state is determined as the interactive load transition state, thereby improving the conservatism of the candidate scheduling load prediction results.

[0061] Through the above processing, the interactive load state occupancy calculation step can use task ratio, remaining resource amount, queue status and historical prediction residual to express the continuous overlap and dynamic migration of multiple load states, solving the problems of sudden changes in scheduling weight caused by fixed resource threshold switching and the lag in load state identification in complex activity scenarios.

[0062] Example 4: This example is based on all the above examples. The candidate scheduling load prediction step is specifically performed in the following manner.

[0063] The candidate scheduling evaluation module reads the list of compatible computing units, the list of compatible model instances, and the allowable model accuracy loss for each task to be scheduled, and generates candidate scheduling actions based on the current resource status of the target edge computing node.

[0064] Candidate scheduling actions for speech recognition tasks include speech recognition instances of the local neural network processor, speech recognition instances of the local graphics processor, and speech recognition instances of the cross-node neural network processor. Candidate scheduling actions for image recognition and target tracking tasks include graphics processor visual model instances and neural network processor compact visual model instances.

[0065] Candidate scheduling actions for multimodal semantic reasoning tasks include complete semantic reasoning model instances of graphics processors, compact semantic reasoning model instances of graphics processors, and cross-node semantic reasoning model instances of graphics processors. Candidate scheduling actions for action generation tasks and speech synthesis tasks include central processing unit model instances, neural network processor model instances, and shared model instances of graphics processors.

[0066] The candidate scheduling actions for video rendering tasks include rendering instances of the local node's graphics processor, rendering instances of cross-node graphics processors, and compact rendering instances with reduced single-frame resolution. Video rendering tasks of the event display terminal must not be assigned to the central processing unit for execution.

[0067] The candidate scheduling load predictor includes a shared resource baseline prediction branch, five interactive load state residual branches, and a load fluctuation prediction branch. The shared resource baseline prediction branch predicts the basic execution latency, basic memory increment, and basic queue changes based on the current resource status and the characteristics of the tasks to be scheduled.

[0068] The five interactive load state residual branches correct the basic prediction results based on the voice low latency dominant state, the vision high throughput dominant state, the semantic reasoning memory dominant state, the rendering frame rate dominant state, and the sudden concurrency congestion state, respectively. The load fluctuation prediction branch generates the prediction interval based on the historical latency residual, the historical memory residual, and the change in task arrival rate.

[0069] The candidate scheduling load predictor outputs the median execution latency, the 90th percentile of execution latency, the median peak memory latency, the 90th percentile of peak memory latency, the 90th percentile of queue waiting time, the probability of task failure, the video frame rate loss, the probability of audio stuttering, the model accuracy loss, and the energy consumption increment for each candidate scheduling action.

[0070] When the model instance of this node lacks the corresponding model weight, the candidate scheduling load prediction result includes the model loading time and the memory usage of the model weight. When the target model instance is executing a batch processing task, the candidate scheduling load prediction result includes the batch processing end waiting time.

[0071] The execution delay of cross-node candidate scheduling actions includes input data serialization time, inter-node transmission time, target node queue waiting time, model execution time, and result return time. When the voice stream is transmitted across nodes, the transmission time is predicted separately for the first voice packet and subsequent voice packets.

[0072] If any of the following conditions are met: the 90th percentile of the execution delay of the task to be scheduled exceeds the task response time limit, the 90th percentile of the peak memory exceeds the remaining memory of the target computing unit, the target model instance is incompatible with the task input, or the remaining network bandwidth between nodes is lower than the task transmission requirements, the candidate scheduling evaluation module will mark the corresponding candidate scheduling action as an unexecutable action.

[0073] Through the above processing, the candidate scheduling load prediction step can predict the execution results and prediction range for different task types, computing units, model replicas and node transmission methods, solving the problem of selecting idle nodes only based on the current utilization rate without considering model loading, peak memory usage, batch processing wait time and cross-node transmission costs.

[0074] Example 5: This example is based on all the above examples. The rich feedback scheduling scoring step and the scheduling execution step are specifically executed in the following manner.

[0075] The candidate scheduling evaluation module only calculates the comprehensive scheduling score for candidate scheduling actions that are not marked as unexecutable actions. The lower the comprehensive scheduling score, the more suitable the candidate scheduling action is for the current task and the current interactive load state.

[0076] The rich feedback scheduling scoring step calculates the comprehensive scheduling score of task i to be scheduled in the k-th resource status window when candidate scheduling action a is adopted according to Formula 2:

[0077] In Formula 2, This represents the comprehensive scheduling score when the scheduled task i in the k-th resource status window adopts candidate scheduling action a. This represents the interaction load state occupancy coefficient for the r-th interaction load state. This represents the 90th percentile value of the execution latency corresponding to the candidate scheduling action. Indicates the target response time limit for the task to be scheduled.

[0078] In Formula 2, This represents the 90th percentile of the peak memory usage corresponding to the candidate scheduling action. This indicates the amount of remaining video memory in the computing unit corresponding to the candidate scheduling action. This represents the stability constant to prevent the denominator from being zero. This represents the 90th percentile of the queue waiting time corresponding to the candidate scheduling action. This represents the queue waiting control value for the corresponding model instance.

[0079] In Formula 2, Indicates the probability of task failure. Indicates video frame rate loss. Indicates the probability of voice interruption. This indicates the loss of model accuracy resulting from using a copy of the corresponding model. This represents the energy consumption increment corresponding to the candidate scheduling action. This represents the reference energy consumption corresponding to the task to be scheduled.

[0080] In Formula 2, This represents the set of successor tasks to task i to be scheduled. This represents the dependency hierarchy between the task to be scheduled i and its successor m. This represents the predicted delay increment that the candidate scheduling action will produce for the subsequent task m. This indicates the dependency level attenuation coefficient. Indicates the terminal priority of the task to be scheduled. These represent the score weights of each prediction item under the r-th interactive load state. The scoring weights represent the impact of subsequent tasks. This represents the scoring weight for terminal priority benefits.

[0081] The impact of subsequent tasks is calculated based on the task dependency graph. When the multimodal semantic reasoning task is assigned to a remote node, causing the action generation task, speech synthesis task, and video rendering task to increase the result transmission waiting time, the corresponding increase in the latency of subsequent tasks is included in the comprehensive scheduling score of the multimodal semantic reasoning task.

[0082] When the execution of the speech recognition task causes a delay in the first recognition result, the waiting time of the multimodal semantic reasoning task is inversely included in the impact of the subsequent tasks of the speech recognition task, enabling the scheduler to identify the impact of the preceding speech scheduling selection on the complete interaction chain.

[0083] The scheduling execution steps rank candidate scheduling actions from low to high according to the comprehensive scheduling score, select the candidate scheduling action with the highest ranking, and send the task number, input data address, model instance number, task response time limit, and prerequisite dependency completion mark to the target edge computing node.

[0084] When the occupancy coefficient of low-latency speech dominance increases, the scoring weight corresponding to initial response delay and speech stuttering increases. When the occupancy coefficient of memory dominance for semantic reasoning increases, the scoring weight corresponding to memory peak and the impact of subsequent tasks increases.

[0085] When the rendering frame rate-dominant state occupancy coefficient increases, the scoring weight corresponding to video frame rate loss increases. When the sudden concurrency state occupancy coefficient increases, the scoring weight corresponding to queue waiting time, task failure probability, and cross-node scheduling capability increases.

[0086] When all candidate scheduling actions are marked as non-executable, the scheduling execution module first switches ordinary tasks that allow for loss of model accuracy to compact model copies, then reduces the internal rendering resolution of the video rendering task by one level, and postpones low-priority non-interactive tasks until the current interactive request is completed.

[0087] When no executable candidate scheduling action is found for high-priority interactions, the scheduling execution module suspends the low-priority video rendering batch processing on the corresponding node, allocates the released graphics processor memory to the high-priority multimodal semantic reasoning task, and writes the intermediate state of the suspended task into the task migration receiving area.

[0088] Regarding parameter adjustments: The first step is to increase the role of incremental energy consumption in the overall dispatch score during normal business hours, and decrease the role of incremental energy consumption during peak activity periods while increasing its role in response delay and task failure probability.

[0089] The second step is to increase the scoring weight of video frame rate loss in the rendering frame rate-dominated state when the actual video frame rate is below 25 frames per second for three consecutive feedback windows.

[0090] Third, when the complete end-to-end response time of a high-priority interaction exceeds 600ms twice consecutively, the scoring weight of the impact of subsequent tasks and the benefits of terminal priority is increased.

[0091] Through the above processing, the rich feedback scheduling scoring step can dynamically adjust the degree of influence of latency, video memory, queue, precision, video, voice, energy consumption, and subsequent tasks according to different load states, thus solving the problem that a fixed scheduling target cannot simultaneously take into account voice response, semantic reasoning video memory, and video rendering frame rate.

[0092] Example 6: This example is based on all the above examples. The full-process feedback learning module, the full-process feedback update steps, and the scheduling execution and continuous service module are executed in the following manner.

[0093] After the task begins execution, the target edge computing node records the time when the task enters the queue, the time when execution begins, the time when the first output result is obtained, the time when execution is completed, the peak memory usage, the average utilization rate of the computing unit, the confidence level of the model output, the number of retries, and the amount of data transmitted across nodes.

[0094] Additional records for the speech recognition task include the first segment recognition time and the number of times the recognition result was corrected; additional records for the speech synthesis task include the first segment audio generation time and the number of times the audio buffer was underloaded; additional records for the video rendering task include the average frame rate, the lowest frame rate, and the number of times video frames were lost.

[0095] After a complete AI-powered human interaction request is completed, the full-process feedback learning module organizes the feedback of each subtask into a rich feedback scheduling trajectory according to the task dependency graph. The rich feedback scheduling trajectory not only records whether the final interaction timed out, but also records the impact of each preceding scheduling action on subsequent semantic reasoning, voice output and video rendering.

[0096] The full-process feedback learning module is equipped with a rich feedback trajectory corrector. The rich feedback trajectory corrector reads the actual scheduling trajectory, the historical prediction results of each candidate scheduling action, and the actual execution feedback of the task, and recalculates the candidate scheduling results based on the resource status accessed during task execution.

[0097] When the actual scheduling actions meet the target response time limit, resource constraints and model accuracy requirements, and when there are no voice interruptions, video frame drops and task retries in the complete interaction chain, the rich feedback trajectory corrector increases the allocation ratio of the actual scheduling actions in the feedback reference distribution throughout the process.

[0098] When actual scheduling actions lead to increased waiting time for subsequent tasks, peak video memory exceeding the safe occupancy ratio, decreased video frame rate, and underloaded audio buffer, the rich feedback trajectory corrector selects multiple actions from historical candidate scheduling actions that can reduce the total loss of the complete interaction chain, and generates a full-process feedback reference distribution based on the differences in the complete interaction loss corresponding to each action.

[0099] The full-process feedback learning module performs forward distribution correction on the scheduling strategy based on the actual resource status accessed by the current scheduling strategy using the full-process feedback reference distribution. It also distributes the feedback of delays, stutters, frame drops, and retries that occur in subsequent tasks back to the preceding scheduling actions that cause subsequent effects, according to the task dependency graph.

[0100] When a multimodal semantic reasoning task is assigned to a graphics processor with near-limited video memory, causing subsequent video rendering tasks to wait for video memory, the waiting feedback of the video rendering task is written into the preceding multimodal semantic reasoning scheduling action, causing the scheduling strategy to reduce the selection ratio of the corresponding scheduling action under the same resource conditions.

[0101] When a speech recognition task is assigned to a cross-node model instance, and the initial recognition time meets its own response time limit but delays the input of multimodal semantic reasoning, the delay feedback of the multimodal semantic reasoning task is written into the speech recognition scheduling action, so that the preceding task cannot be identified as a high-quality scheduling action based solely on its own execution time.

[0102] The full-process feedback learning module performs server-side policy training once every 5,000 complete rich feedback scheduling trajectories. The scheduling policy output layer is updated every 15 minutes based on the latest feedback. The load prediction backbone network is updated using complete feedback data after the end of each business day.

[0103] The updated scheduling strategy first enters the shadow scheduling state, which lasts for 10 minutes and only generates candidate scheduling results. It does not directly control task execution. The strategy replacement is completed when the 90th percentile response latency, task failure rate, and model accuracy loss of the shadow scheduling result are not higher than those of the current scheduling strategy.

[0104] The scheduling execution and continuous service module detects the running heartbeats of three edge computing nodes every 200ms. If a single edge computing node fails to return a heartbeat for three consecutive detection cycles, the corresponding node is identified as a disconnected node, and the allocation of new tasks to the disconnected node is stopped.

[0105] In the missing node, the speech recognition task migrates based on the completed speech segment location, the target tracking task migrates based on the nearest target location and tracking number, the multimodal semantic reasoning task migrates based on the completed input encoding state, and the video rendering task migrates based on the nearest complete keyframe and action time marker.

[0106] After the network connection is restored, the lost node first completes the model instance status check and resource status synchronization, and then enters the candidate scheduling action set. Model instances that have not completed the status check will not receive real-time interactive tasks.

[0107] Through the above processing, the whole-process feedback update step can improve the scheduling strategy by utilizing the first response, memory peak, voice stuttering, video frame drops, model confidence, and the impact of subsequent tasks in the task execution trajectory. This solves the problem that allocating rewards based solely on the final task success status cannot determine the actual contribution of preceding scheduling actions to the quality of subsequent interactions.

[0108] Example 7: This example is based on all the above examples and illustrates the actual operation of this method using a high-priority AI-powered human interaction during a weekend new product launch event in a commercial complex.

[0109] After the event started, 32 AI smart human terminals were online simultaneously. The event display terminal continuously performed high-definition video rendering, the fixed shopping guide terminal continuously received voice consultation, and the edge computing smart server detected that the total task arrival rate increased from 74 per second to 186 per second.

[0110] The interactive load state prediction module generates the following state occupancy coefficients: low latency speech occupancy (0.11), high throughput vision occupancy (0.18), memory-dominated semantic reasoning semantic reasoning occupancy (0.28), rendering frame rate occupancy (0.15), and sudden concurrency congestion occupancy (0.28).

[0111] The first graphics processor of the second edge computing node has a memory usage of 44.6GB and a remaining memory of 3.4GB. There are three waiting tasks in the multimodal semantic reasoning model instance queue of the second edge computing node. The remaining memory of the second graphics processor of the first edge computing node is 14.2GB, and the remaining memory of the first graphics processor of the third edge computing node is 21.7GB.

[0112] A user initiates a request for help through the 12th fixed shopping guide terminal. The AI ​​intelligent human terminal sets the corresponding request as a high-priority interaction. After the voice recognition task is completed, a multimodal semantic reasoning task is generated. The target response time of the multimodal semantic reasoning task is 300ms, and the complete end-to-end response time is 600ms.

[0113] The candidate scheduling evaluation module generates four candidate scheduling actions for the multimodal semantic reasoning task. The first candidate scheduling action assigns the task to the complete semantic reasoning model instance of the first graphics processor of the second edge computing node. The predicted execution latency is 90th percentile of 680ms and the predicted peak video memory is 7.2GB. Since the peak video memory exceeds the remaining video memory and the execution latency exceeds the target response time limit, the first candidate scheduling action is marked as an unexecutable action.

[0114] The second candidate scheduling action assigns the task to the complete semantic reasoning model instance of the second graphics processor of the first edge computing node. The predicted execution latency is 268ms at the 90th percentile, the predicted peak memory is 6.8GB, the cross-node input transmission time is 18ms, the model accuracy loss is 0, and the overall scheduling score is 0.43.

[0115] The third candidate scheduling action assigns the task to the compact semantic reasoning model instance of the first graphics processor on the third edge computing node. The predicted execution latency is 205ms at the 90th percentile, the predicted peak memory is 2.9GB, the cross-node input transmission time is 24ms, the model accuracy loss is 0.07, and the overall scheduling score is 0.49.

[0116] The fourth candidate scheduling action assigns the task to the compact semantic reasoning model instance of the central processor of the third edge computing node. The predicted execution latency is 820ms at the 90th percentile. Since the execution latency exceeds the target response time limit, the fourth candidate scheduling action is marked as an unexecutable action.

[0117] The scheduling execution steps select the second candidate scheduling action with a comprehensive scheduling score of 0.43, assign the multimodal semantic reasoning task to the complete semantic reasoning model instance of the second graphics processor of the first edge computing node, and at the same time migrate the two low-priority product animation rendering tasks in the first edge computing node to the third edge computing node.

[0118] The actual execution time of the multimodal semantic reasoning task was 241ms, the first output time of speech synthesis was 196ms, the complete end-to-end response time was 574ms, and the AI ​​intelligent human terminal maintained an output frame rate of 27 frames per second. No voice stuttering, video frame loss, or task retry occurred during the execution process.

[0119] The full-process feedback learning module writes the actual execution latency, peak memory usage, model output confidence, video output frame rate, voice output status, and subsequent task waiting time into the rich feedback scheduling trajectory, and improves the reference allocation ratio for allocating high-priority semantic reasoning tasks to the second graphics processor of the first edge computing node under the same sudden concurrency congestion conditions.

[0120] During the same activity period, a regular product consultation task is scheduled to a compact semantic reasoning model instance on the third edge computing node. The actual end-to-end response time is 638ms and the model output confidence meets the requirements of regular consultation. The full-process feedback learning module retains the reference allocation ratio of the compact model copy in regular tasks, so that the complete model copy prioritizes serving high-priority interactions.

[0121] After the event, the total task arrival rate dropped to 69 per second, the congestion occupancy rate dropped to 0.06, the memory dominance rate for semantic reasoning dropped to 0.14, the scheduling and execution module stopped delaying low-priority tasks, and restored the normal role of energy consumption increment in the comprehensive scheduling score.

[0122] Regarding parameter adjustments: The first step is to update the rendering frame rate-dominant state prototype based on the actual video rendering task arrival rate and graphics processor rendering load after the number of display terminals changes.

[0123] The second step involves re-collecting the execution latency, peak memory usage, and model accuracy loss of both the full and compact model copies after the multimodal semantic reasoning model has been upgraded, instead of directly using the prediction parameters from before the upgrade.

[0124] The third step is to collect the amount of input data and response time of the new AI intelligent human terminal after it is connected, and then add the corresponding terminal task to the formal scheduling queue in the manner of shadow task.

[0125] Through the above processing, this method can continuously identify server load status, predict the resource impact of heterogeneous candidate scheduling actions, and select and update scheduling strategies based on rich feedback throughout the entire process in actual deployment scenarios where multiple AI intelligent human terminals concurrently generate voice, vision, semantic and rendering tasks. This solves the problem that fixed threshold load balancing cannot simultaneously guarantee high-priority interactive response, model inference memory security and digital human video frame rate.

[0126] Example 8: Based on all the above examples, this example implements a lightweight deployment of the interactive load state prediction module, candidate scheduling evaluation module, and scheduling execution and continuous service module. The resource consumption, scheduling latency, and continuous service performance of the lightweight configuration are verified through multi-terminal concurrency, sudden task access, and edge computing node disconnection tests.

[0127] The lightweight configuration deploys the complete training network of the full-process feedback learning module on the factory server, and forms a lightweight scheduling inference model by combining the interactive load state prediction module, the candidate scheduling load predictor, and the scheduling policy output layer. This model is then deployed to the first edge computing node, the second edge computing node, and the third edge computing node, respectively. This enables each edge computing node to complete the interactive load state occupancy calculation, candidate scheduling load prediction, and target scheduling action selection without calling the remote training network.

[0128] The complete training network uses 128-dimensional shared resource state coding results and five independent interactive load state residual branches. The lightweight scheduling inference model compresses the dimension of the shared resource state coding to 64 dimensions and adjusts the five interactive load state residual branches into a shared prediction backbone and five low-rank state correction branches. Each low-rank state correction branch retains only the difference correction parameters generated by the corresponding interactive load state on execution latency, peak memory usage, queue waiting time, task failure probability, video frame rate loss, and voice stuttering probability.

[0129] The lightweight scheduling inference model retains task arrival rate, task type ratio, CPU utilization, GPU utilization, NPU utilization, remaining GPU memory, remaining RAM, estimated computational cost of model instance queue, remaining network bandwidth, historical latency residuals, historical GPU memory residuals, and number of high-priority interactions. It shares the prediction backbone by embedding the model instance identifier, edge computing node identifier, and task type identifier into the input vector, reducing parameter duplication caused by configuring independent prediction networks for different model instances.

[0130] The network weights and intermediate activation data of the lightweight scheduling inference model are stored using 8-bit integers. The calculation results of interactive load state occupancy, the 90th percentile of execution latency, the 90th percentile of peak memory usage, and the comprehensive scheduling score are output using 16-bit floating-point numbers. Before model quantization, calibration is performed using normal business, peak activity, and sudden concurrency data collected in the last three business days to prevent the high-load prediction results from being truncated due to the sole use of low-load data to determine the quantization range.

[0131] Under lightweight configuration, the candidate scheduling evaluation module first performs a pre-screening of candidate scheduling actions based on task input compatibility, model weight loading status, memory safety margin, and task response time limit. Then, it performs an initial sorting of candidate scheduling actions based on the difference between the current remaining resources and the expected resource requirements of the task to be scheduled. Only the top six candidate scheduling actions are input into the lightweight scheduling inference model, reducing the computational consumption caused by a large number of candidate scheduling actions that obviously do not meet the execution conditions participating in neural network inference.

[0132] The complete training network generates a full-process feedback reference distribution based on the rich feedback scheduling trajectory, and updates the shared prediction backbone and state correction branches after the end of each business day. The lightweight scheduling inference model only accepts model parameters verified through shadow scheduling. The model update package includes quantized weights, interactive load state prototypes, model instance resource benchmarks, and scheduling score parameters. The previous version of the lightweight scheduling inference model is retained during the update process, and the previous version is immediately restored if the current version fails to load.

[0133] This embodiment uses the three edge computing nodes and 32 AI intelligent human terminals from Embodiment 1 to establish a test environment. The test tasks consist of speech recognition, image recognition, target tracking, multimodal semantic reasoning, action generation, speech synthesis, and video rendering tasks. The proportions of the seven types of tasks are set to 28%, 12%, 10%, 18%, 8%, 12%, and 12%, respectively, with high-priority interactions accounting for 8% of all interaction requests.

[0134] The test process sequentially set a normal operating load of 60 tasks per second, an active load of 120 tasks per second, and a burst concurrent load of 180 tasks per second. Each load phase was run continuously for 30 minutes and repeated five times. The end-to-end response time limit for normal interaction was set to 900ms, the end-to-end response time limit for high-priority interaction was set to 600ms, and the video output frame rate control value was set to 25 frames per second.

[0135] The lightweight resource occupancy test used both the full scheduling inference model and the lightweight scheduling inference model to process the same 100,000 sets of resource status windows and candidate scheduling actions. The test results are shown in Table 1.

[0136] Table 1. Lightweight test results of the scheduling inference model:

[0137] In Table 1, the model file size of the lightweight scheduling inference model is reduced to 20.18% of that of the full scheduling inference model, the peak memory usage for inference is reduced to 29.44%, the average inference time for a single candidate scheduling batch is reduced to 30.15%, the execution latency and peak memory prediction error only increase to a limited extent, and the target action consistency rate remains at 94.7%, which can meet the local real-time scheduling requirements of edge computing nodes.

[0138] The dynamic load balancing test setup included a fixed threshold scheduling scheme, a full model scheduling scheme, and a lightweight model scheduling scheme. The fixed threshold scheduling scheme selected execution nodes based on GPU utilization, remaining video memory, and queue length. The full model scheduling scheme used an uncompressed interactive load state prediction and candidate scheduling load prediction network. The lightweight model scheduling scheme used the shared prediction backbone, low-rank state correction branch, and candidate scheduling action pre-screening processing of this embodiment. The test results under burst concurrent load are shown in Table 2.

[0139] Table 2. Scheduling test results under the condition of 180 tasks per second:

[0140] Table 2 shows that the lightweight model scheduling scheme reduces the average end-to-end response time by 300ms compared to the fixed threshold scheduling scheme, reduces the high-priority interaction timeout rate from 12.6% to 2.4%, and reduces the number of voice stutters per 1000 interactions from 37 to 9. Furthermore, no task failures caused by insufficient GPU memory occurred during the test. The average end-to-end response time of the lightweight model scheduling scheme differs from that of the full model scheduling scheme by 16ms, indicating that model compression does not significantly weaken the dynamic load balancing effect.

[0141] The node disconnection test was performed under an active load of 120 tasks per second. During the test, the network connection of the second edge computing node was disconnected and maintained for 30 seconds. After the scheduling execution and continuous service module did not receive a response from the second edge computing node for three consecutive 200ms heartbeat detection cycles, it determined that the node was disconnected, stopped assigning new tasks to the second edge computing node, and migrated the speech recognition task according to the completed speech segment position, the target tracking task according to the nearest target position, the multimodal semantic reasoning task according to the completed input encoding state, and the video rendering task according to the nearest complete keyframe.

[0142] During the node outage, there were 418 unfinished tasks. Of these, 402 tasks were successfully completed across nodes, 11 low-priority rendering tasks were postponed, and 5 tasks with locked input states were re-executed. The median task completion time was 184ms, and the 90th percentile was 326ms. No duplicate execution of the same task occurred on two edge computing nodes. After the network connection was restored, the second edge computing node was re-added to the candidate scheduling action set after model instance status checks and resource status synchronization.

[0143] The continuous test lasted for 72 hours. The lightweight scheduling inference model processed a total of 18.46 million candidate scheduling actions. The average CPU utilization of the scheduling inference process on the edge computing node was 6.8%, and the average memory usage was 148MB. There were no abnormal exits of the inference process or interruptions in model updates. During the model parameter update, task scheduling was continued by the previous version of the lightweight scheduling inference model, and the interactive services of the AI ​​intelligent human terminal were not interrupted.

[0144] Regarding parameter adjustments: The first step is to prioritize increasing the shared resource state encoding dimension and the low-rank state correction branch dimension when the target action consistency rate of the lightweight scheduling inference model is less than 92%, without directly canceling the pre-screening of candidate scheduling actions.

[0145] The second step is to recalibrate the 8-bit integer using the latest rich feedback scheduling trajectory when the mean relative error of the 90th percentile of the execution delay exceeds 10% for two consecutive test cycles, and update the historical delay residual benchmark of the corresponding model instance.

[0146] Third, when the average inference time of a single candidate scheduling batch exceeds 5ms, reduce the number of candidate scheduling actions entering the lightweight scheduling inference model, while retaining all executable candidate scheduling actions corresponding to high-priority interactions.

[0147] Through the above processing, this embodiment utilizes shared prediction backbone, low-rank state correction, integer model storage, and candidate scheduling action pre-screening to reduce the number of parameters, memory usage, and inference time of the scheduling model. The scheduling performance of the lightweight configuration is verified through multi-load phase, node disconnection, and continuous operation tests, which solves the problem that the complete scheduling model has large resource consumption and is difficult to deploy in real time on various edge computing nodes for a long time.

[0148] Example 9: Based on all the above examples, this example limits the range of parameter values ​​in Formula 1 and Formula 2, and compares and tests the effects of interaction load state occupancy calculation, dynamic scoring weight adjustment and full-process feedback update on the quality of multimodal AI intelligent human interaction by using a fixed threshold scheduling scheme, a fixed scoring weight scheduling scheme, a scheduling scheme without subsequent task feedback and the corresponding scheduling scheme of this method.

[0149] Before the interactive load state occupancy calculation, the task arrival rate, computing unit utilization, remaining video memory, expected computational load of the model instance queue, remaining network bandwidth, historical latency residual, and historical video memory residual in the task resource input vector are converted to a numerical range of zero to one. The interactive load state prototype and the task resource input vector use the same data scale, and the state deviation value ranges from zero to one to avoid the numerical magnitude of different resource indicators affecting the state gating results.

[0150] State deviation suppression coefficient in Formula 1 The value range is set to 0.40 to 1.40. When the state deviation suppression coefficient is less than 0.40, the state prototype deviation amount does not sufficiently limit the occupancy coefficient of the interactive load state, which easily allows mismatched interactive load states to obtain a high occupancy coefficient. When the state deviation suppression coefficient is greater than 1.40, the occupancy result of the interactive load state depends too much on the state prototype, making it difficult to identify the transitional load formed by the overlap of different interactive load states.

[0151] State continuity coefficient in Formula 1 The value range is set to 0.15 to 0.80. When the state continuation coefficient is close to 0.15, it can improve the load state response speed after a sudden task is connected. When the state continuation coefficient is close to 0.80, it can reduce the repeated changes in the interactive load state occupancy coefficient caused by short-term task fluctuations. In this embodiment, the state deviation suppression coefficient is used. Set it to 0.85 to adjust the state continuation coefficient. Set it to 0.45.

[0152] When the proportion of task types changes relatively smoothly during normal business hours in a commercial complex, the state continuity coefficient is set to 0.45 to 0.65. When weekend activities begin, people enter the complex in large numbers, and high-priority interactions occur in large numbers, the state continuity coefficient is set to 0.20 to 0.45. This allows the interaction load state prediction module to identify sudden concurrent congestion states in a timely manner while maintaining state continuity.

[0153] In Formula 2, the first to eighth scoring weights are all set to non-negative numbers. After each interactive load state, the eight scoring weights are normalized to make the sum of the eight scoring weights equal to 1. The scoring weights corresponding to execution latency range from 0.13 to 0.35, peak video memory usage range from 0.08 to 0.30, queue waiting time range from 0.08 to 0.25, task failure probability range from 0.10 to 0.30, video frame rate loss range from 0.03 to 0.30, voice stuttering probability range from 0.03 to 0.25, model accuracy loss range from 0.04 to 0.18, and energy consumption increment range from 0.02 to 0.12.

[0154] In this embodiment, the scoring weights in Formula 2 are set according to the five interactive load states, and the specific values ​​are shown in Table 3.

[0155] Table 3. Scoring weights for different interaction load states.

[0156] The weighting of the impact of subsequent tasks in Formula 2 The value range is set to 0.10 to 0.45, depending on the hierarchical attenuation coefficient. The value range is set to 0.35 to 0.85, which is the weight for the terminal priority revenue score. The value range is set to 0.05 to 0.30. In this embodiment, the weight of the subsequent task's impact on the score is determined. Setting it to 0.28 will reduce the dependency level attenuation coefficient. Set the weight of the terminal priority benefit score to 0.62. Set it to 0.18.

[0157] When the weight of the impact of subsequent tasks is less than 0.10, the delay impact of preceding tasks on action generation, speech synthesis and video rendering tasks is difficult to be included in the comprehensive scheduling score. When the weight of the impact of subsequent tasks is greater than 0.45, the scheduling results are prone to over-focusing on remote subsequent tasks and reducing the execution timeliness of current high-priority tasks.

[0158] When the dependency level decay coefficient is close to 0.35, the comprehensive scheduling score mainly considers the direct successor task. When the dependency level decay coefficient is close to 0.85, it can improve the retention of feedback from remote tasks in multi-layer task dependency chains. When the task dependency level does not exceed three levels, the dependency level decay coefficient is set to 0.50 to 0.70. When the task dependency level exceeds three levels, the dependency level decay coefficient is set to 0.65 to 0.85.

[0159] The terminal priority revenue score weight is adjusted based on the proportion of high-priority interactions. When the proportion of high-priority interactions to all interaction requests is less than 5%, it is set to 0.08 to 0.15; when the proportion of high-priority interactions reaches 5% but is less than 15%, it is set to 0.15 to 0.22; and when the proportion of high-priority interactions reaches 15%, it is set to 0.22 to 0.30.

[0160] The comparative test continued to use the three edge computing nodes and 32 AI intelligent human terminals in Example 8. The test task arrival rate was set to 180 per second. The proportion of the seven types of tasks, the proportion of high-priority interactions, the response time limit of ordinary interactions, the response time limit of high-priority interactions, and the video output frame rate control value remained unchanged. Each scheduling scheme was run continuously for 30 minutes and repeated five times.

[0161] The fixed threshold scheduling scheme determines the task execution node based on the graphics processor utilization, video memory usage, and model instance queue length. When any one of the following conditions is met: graphics processor utilization reaches 85%, video memory usage reaches 90%, and model instance queue length reaches 6, the new task is assigned to the edge computing node with the shorter queue length. The fixed threshold scheduling scheme does not calculate the interactive load state occupancy coefficient and does not predict the impact of subsequent tasks corresponding to candidate scheduling actions.

[0162] The fixed-score-weight scheduling scheme uses the same candidate scheduling actions and load prediction results as this method, but maintains the same score weight under different task structures and resource states. The score weights corresponding to execution latency, peak memory usage, queue waiting time, task failure probability, video frame rate loss, voice stuttering probability, model accuracy loss, and energy consumption increment are set to 0.20, 0.18, 0.15, 0.15, 0.10, 0.08, 0.08, and 0.06, respectively.

[0163] The scheduling scheme lacking feedback from subsequent tasks adopts an interactive load state occupancy coefficient and dynamic scoring weight, but sets the scoring weight of the impact of subsequent tasks to zero. After the task is completed, the scheduling strategy is updated only using the execution latency, peak video memory, and task completion status of the current task. Voice stuttering, video frame loss, and waiting results of subsequent tasks are not reverse-associated to the preceding scheduling actions.

[0164] The scheduling scheme corresponding to this method adopts the scoring weights in Table 3, and sets the state deviation suppression coefficient to 0.85, the state continuation coefficient to 0.45, the subsequent task impact scoring weight to 0.28, the dependency level decay coefficient to 0.62, and the terminal priority benefit scoring weight to 0.18. The test results are shown in Table 4.

[0165] Table 4. Comparison of test results for different scheduling schemes:

[0166] Compared to the fixed threshold scheduling scheme, the fixed score weight scheduling scheme can reduce significantly unreasonable task allocation by utilizing candidate scheduling load prediction results. However, since the same score weight is maintained under different interaction load states, there are still problems such as increased waiting time for voice tasks and timeouts for high-priority interactions when sudden concurrent congestion and semantic inference memory dominance occur simultaneously.

[0167] The scheduling scheme lacking feedback from subsequent tasks can dynamically adjust the scoring weights based on the interaction load state occupancy coefficient, but it cannot identify the delay propagation relationship between speech recognition tasks, multimodal semantic reasoning tasks, and video rendering tasks, causing the complete interaction chain to time out even after some preceding tasks have met their own response time limits.

[0168] Compared to the fixed threshold scheduling scheme, the scheduling scheme of this method reduces the average end-to-end response time by 300ms, reduces the high-priority interaction timeout rate by 10.2 percentage points, reduces the number of voice stutters by 75.68%, reduces the number of video frame drops by 75%, and does not cause any task failures due to insufficient GPU memory in five tests.

[0169] To verify the impact of the parameter values ​​in Formula 1 and Formula 2 on scheduling stability, this embodiment sets up a low continuation parameter group, an intermediate adaptation parameter group, and a high continuation parameter group. The test results of the three groups of parameters are shown in Table 5.

[0170] Table 5. Sensitivity test results for parameter combinations:

[0171] Low-duration parameter groups can respond quickly to changes in task structure, but the number of main state transitions in interactive load states is relatively large, causing frequent changes in scoring weights. High-duration parameter groups can reduce short-term state fluctuations, but there is a delay in state updates after sudden concurrent tasks arrive. Intermediate adaptation parameter groups achieve a good balance in terms of load state response speed, scheduling stability, and high-priority interaction protection.

[0172] Regarding parameter adjustments: The first step is to increase the state continuity coefficient when the number of main states of the interactive load state changes more than 30 times per hour and the task arrival rate does not change significantly. The increment for each adjustment is set to 0.05, and the state continuity coefficient shall not exceed 0.80.

[0173] The second step is to reduce the state congestion coefficient and increase the state deviation suppression coefficient when two consecutive resource status windows do not form the highest occupancy coefficient after a sudden concurrent task arrives. The adjustment amount is set to 0.05 and 0.10 respectively each time.

[0174] The third step is to increase the terminal priority benefit score weight and the subsequent task impact score weight when the high priority interaction timeout rate exceeds 3% for two consecutive test cycles, and at the same time, re-normalize the eight score weights.

[0175] Through the above processing, this embodiment supplements the range of formula parameters, the basis for parameter adjustment, and comparative test data without increasing hardware structure limitations or task execution timing diagrams. It uses test indicators to verify the actual technical effects of interactive load state occupancy calculation, dynamic scoring weight, and subsequent task feedback mechanism, thus solving the problems of formula parameters lacking implementation boundaries and scheduling improvement effects lacking comparative support.

Claims

1. A dynamic load balancing scheduling method for a multimodal AI intelligent human edge computing intelligent server, characterized in that, Includes the following steps: The steps for constructing a multimodal task queue are as follows: receiving interaction requests uploaded by multiple AI intelligent human terminals, splitting the interaction requests into multiple multimodal sub-tasks, establishing a task dependency graph between the multimodal sub-tasks, and recording the task type, input data volume, target response time limit, terminal priority, and prerequisite dependencies for each multimodal sub-task. The heterogeneous resource status acquisition step collects the resource utilization status, storage resource status, network resource status, model instance queue status, task arrival status, and historical prediction residuals of heterogeneous computing units in multiple edge computing nodes, and generates a task resource input vector. The interactive load state occupancy calculation step generates multiple interactive load state occupancy coefficients corresponding to the interactive load states based on the task resource input vector. The candidate scheduling load prediction step generates multiple candidate scheduling actions for the task to be scheduled. Based on the current resource status, the interaction load state occupancy coefficient, and the characteristics of the task to be scheduled, it generates the execution delay prediction result, resource occupancy prediction result, and interaction quality loss prediction result corresponding to each candidate scheduling action. The rich feedback scheduling scoring step adjusts the degree of influence of each prediction result in the scheduling score according to the interactive load state occupancy coefficient, generates the subsequent task impact of each candidate scheduling action according to the task dependency graph, and generates a comprehensive scheduling score in combination with terminal priority. The scheduling execution steps are as follows: candidate scheduling actions that do not meet resource constraints and target response time limits are eliminated; the target scheduling action with the lowest comprehensive scheduling score is selected from the remaining candidate scheduling actions; and the task to be scheduled is assigned to the edge computing node, heterogeneous computing unit and model instance corresponding to the target scheduling action. The entire process involves feedback and update steps, collecting multi-dimensional execution feedback during task execution, generating a rich feedback scheduling trajectory according to the task dependency graph, associating subsequent task feedback with corresponding preceding scheduling actions, and updating the interactive load state prediction parameters, candidate scheduling load prediction parameters, and scheduling strategy parameters based on the rich feedback scheduling trajectory.

2. The dynamic load balancing scheduling method according to claim 1, characterized in that, The multiple multimodal subtasks include speech recognition, image recognition, target tracking, multimodal semantic reasoning, action generation, speech synthesis, and video rendering. The task dependency graph places the speech recognition task, the image recognition task, and the target tracking task before the multimodal semantic reasoning task, places the action generation task and the speech synthesis task after the multimodal semantic reasoning task, and places the video rendering task after the action generation task and the speech synthesis task. The multimodal task queue construction step determines the scheduling order of each multimodal subtask based on the number of successor tasks, the completion status of prerequisite dependencies, and the target response time limit.

3. The dynamic load balancing scheduling method according to claim 2, characterized in that, The heterogeneous computing unit includes a central processing unit, a graphics processing unit, and a neural network processor; The heterogeneous resource status acquisition step collects the following data: CPU core utilization rate, GPU computing utilization rate, neural network processor utilization rate, remaining video memory, remaining running memory, remaining network bandwidth between nodes, model instance queue length, estimated computational load of model instance queue, number of concurrent model instances, task arrival rate, historical execution time, historical latency residual, and historical video memory residual. The queue status of the model instance includes the number of waiting tasks, the estimated computational load of the waiting tasks, the estimated increase in GPU memory, and the earliest response time.

4. The dynamic load balancing scheduling method according to claim 3, characterized in that, The multiple interactive load states include a voice low-latency dominant state, a visual high-throughput dominant state, a semantic reasoning video memory dominant state, a rendering frame rate dominant state, and a burst concurrency congestion state. The interactive load state occupancy calculation step establishes interactive load state prototypes corresponding to the multiple interactive load states. Based on the state deviation between the current task resource input vector and each interactive load state prototype, the state gating value output by the state gating unit, and the interactive load state occupancy coefficient corresponding to the previous resource state window, multiple interactive load state occupancy coefficients corresponding to the current resource state window are generated, and the sum of the multiple interactive load state occupancy coefficients is 1.

5. The dynamic load balancing scheduling method according to claim 4, characterized in that, The candidate scheduling actions include target edge computing nodes, target heterogeneous computing units, target model instances, model replica accuracy, and cross-node transmission methods; The candidate scheduling load prediction step generates the following values ​​through the shared resource baseline prediction branch, multiple interactive load state residual branches, and load fluctuation prediction branch: median execution latency, upper limit execution latency, median peak memory latency, upper limit peak memory latency, upper limit queue waiting time, task failure probability, video frame rate loss, voice stuttering probability, model accuracy loss, and energy consumption increment. When the target model instance has not loaded the corresponding model weights, the candidate scheduling load prediction step writes the model loading time and the memory usage of the model weights into the prediction result of the corresponding candidate scheduling action. When the candidate scheduling action includes cross-node transmission, the input data serialization time, the inter-node transmission time, and the result return time are written into the execution delay prediction result.

6. The dynamic load balancing scheduling method according to claim 5, characterized in that, The rich feedback scheduling scoring step weights execution latency, peak video memory, queue waiting time, task failure probability, video frame rate loss, voice stuttering probability, model accuracy loss, and energy consumption increment based on the occupancy coefficient of each interactive load state. The impact of subsequent tasks is generated based on the set of subsequent tasks of the task to be scheduled, the dependency hierarchy between the task to be scheduled and each subsequent task, and the predicted delay increment generated by the candidate scheduling action for each subsequent task. When all candidate scheduling actions fail to meet resource constraints and target response time limits, the scheduling execution steps sequentially perform model replica precision adjustment, video rendering resolution adjustment, low-priority task postponement, and cross-node collaborative scheduling, and prioritize the release of resources for the execution of critical chain tasks corresponding to high-priority interactions.

7. The dynamic load balancing scheduling method according to claim 6, characterized in that, The multidimensional execution feedback includes actual execution latency, first response time, peak memory usage, model output confidence, task timeout status, number of video frame losses, number of audio stutters, number of task retries, and user interaction interruption status. The full-process feedback update step combines the multi-dimensional execution feedback corresponding to multiple multimodal subtasks in the same interaction request into the rich feedback scheduling trajectory according to the task dependency graph. It generates the feedback impact quantity corresponding to the preceding scheduling action based on the delay, voice stuttering, video frame loss and task retry results generated by the subsequent task, and generates the full-process feedback reference distribution based on the actual scheduling action and historical candidate scheduling actions. The full-process feedback update step adjusts the selection probability of each candidate scheduling action based on the resource status actually accessed by the current scheduling strategy, according to the full-process feedback reference distribution, and replaces the current scheduling strategy after the updated scheduling strategy is verified by shadow scheduling.

8. A multimodal AI intelligent human edge computing intelligent server, characterized in that, It includes a multimodal task orchestration module, a heterogeneous resource awareness module, an interactive load state prediction module, a candidate schedule evaluation module, a full-process feedback learning module, and a schedule execution and continuous service module; The multimodal task orchestration module constructs multimodal subtasks and task dependency graphs; the heterogeneous resource perception module collects the heterogeneous resource status of multiple edge computing nodes; the interactive load state prediction module generates multiple interactive load state occupancy coefficients; the candidate scheduling evaluation module generates execution result predictions and comprehensive scheduling scores corresponding to candidate scheduling actions; the full-process feedback learning module updates scheduling strategy parameters based on rich feedback scheduling trajectories; and the scheduling execution and continuous service module performs task allocation, model replica switching, cross-node migration, and task continuation for disconnected nodes. The multimodal AI intelligent human edge computing intelligent server is configured to execute the dynamic load balancing scheduling method according to any one of claims 1 to 7.

Images