Dynamic load balancing method and system based on session granularity and bit mask mapping

Abstract

The invention discloses a dynamic load balancing method and system based on session granularity and bit mask mapping. The method comprises the following steps: S100, initializing a system; s200, thread management: adding a session created by an application to a task thread; s300, load monitoring: starting a timer at regular time, collecting load data of each thread, detecting whether a thread load exceeds a preset threshold value or not, if yes, triggering a balancing decision, and if not, directly triggering a session execution operation; s400, equilibrium decision making: searching for the most idle target migration task, and selecting a migration candidate session; s500, migration execution: removing from the source task, adding to the target task, and updating the bit mask state of the session; s600, session execution: each task thread executes the session allocated to the task thread according to the bitmap state in the main loop of the task thread, updates the execution time and the number of times, and regularly checks the use condition of the stack; and S700, the system exits. According to the invention, the problems of increased response delay, insufficient adaptability, unstable system and the like caused by resource competition during session concurrence can be solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electronic communication, and particularly relates to a dynamic load balancing method and system based on conversation granularity and bit mask mapping. BACKGROUND

[0002] Under the background of rapid development of artificial intelligence technology, more and more products are empowered by AI. For example, through the cooperation of one or more models such as a Speech-to-Text (STT) model, a Text-to-Speech (TTS) model, an Image Recognition (IR) model, an Image Generation (IG) model, a Video Generation (VG) model, and a Large Language Model (LLM), customized and super-natural human-computer interaction (such as voice dialogue and object recognition) is realized.

[0003] Among them, for small system embedded devices, it is obviously impossible to support local deployment of large models, and different model conversation services need to be provided through cloud servers. However, supporting multiple concurrent conversations is also very challenging for resource-constrained devices (the CPU frequency of these devices is usually lower than 240MHz, FLASH≤2M, and RAM≤256K), mainly in the following aspects:

[0004] Resource bottleneck: audio codec (such as Opus encoding format) occupies more than 50% CPU resources, encryption and decryption libraries and wireless protocol stacks consume 30%, and the remaining resources are difficult to support efficient concurrent multiple conversations.

[0005] Inefficient memory management: each conversation task is independently allocated a 8KB stack, and is frequently created / destroyed, resulting in a fragmentation rate of more than 40% and a decrease in system stability.

[0006] Lagging scheduling mechanism: the traditional scheme uses task-level scheduling and cannot be refined to the conversation granularity. If multiple conversations are directly run on a single thread, it will cause the response delay of some key conversations to fluctuate by more than 100ms.

[0007] The patent document with the publication number CN112256419B discloses an embedded system multi-task scheduling method based on timer clock control, and the core mechanism is as follows:

[0008] (1) Fixed time slice allocation mechanism: a fixed execution time slice (such as 5ms / task) is preset for each task, and the task switching is triggered by a hardware timer; the task priority is distinguished by the time slice size, and a higher priority task is allocated a larger time slice.

[0009] (2) Timer interrupt scheduling: The system clock interrupt frequency is 100Hz (10ms), and each interrupt triggers task switching logic; "time slice rotation + priority preemption" mixed scheduling strategy is adopted.

[0010] (3) Task state management: Maintain task ready queue, sort by priority, ensure high priority task execution first; manage task context through task control block (TCB), including register value, stack pointer, etc.

[0011] (4) Real-time guarantee: Introduce "emergency task" flag, allow critical tasks to immediately preempt the current executing task; ensure time slice accuracy through timer calibration mechanism.

[0012] Compared with the traditional method of relying entirely on the operating system scheduler, the above patent technology has the following advantages:

[0013] Good real-time performance: Fixed time slice scheduling ensures regularity of task switching;

[0014] Simple implementation: No dependence on complex operating system mechanisms, suitable for low-end MCU platforms;

[0015] Strong controllability: Developers can flexibly configure time slice size to adapt to different scene requirements.

[0016] However, the above patent technology also has obvious limitations, as follows:

[0017] Lack of dynamic adjustment capability: unable to dynamically adjust time slice or priority according to actual load of each task, which may lead to resource waste or critical task response delay;

[0018] High context switching overhead: Frequent interrupt handling and task switching cause additional CPU overhead, especially in multi-model concurrent scenarios.

[0019] In addition, the patent document with publication number CN119862038A discloses a kind of embedded device load balancing adjustment method, and its core technical scheme is introduced as follows:

[0020] (1) Load evaluation based on resource monitoring: Real-time collection of CPU usage, memory occupation, network bandwidth and other system resource indicators; Calculate task resource consumption coefficient (such as CPU consumption weight 0.6, memory consumption weight 0.4).

[0021] (2) Task migration mechanism: When detecting that task load exceeds threshold (such as CPU usage > 80%), trigger task migration; Preferentially migrate tasks with lower resource consumption coefficient to low-load processors (such as multi-core systems).

[0022] (3) Dynamic priority adjustment: dynamically adjust the priority according to the task response time, and the priority of the task whose response delay exceeds the threshold is improved; the task aging mechanism is introduced, and the priority of the task waiting for a long time is gradually improved.

[0023] (4) Resource isolation strategy: limit the resource usage of high-load tasks through resource quota to prevent resource exhaustion; assign a dedicated resource pool to critical tasks to ensure real-time performance.

[0024] Compared with the traditional static scheduling method, the above load balancing adjustment method has the following advantages: it can perform task migration and scheduling optimization according to system load changes, has certain dynamic adaptability; through load balancing, the risk of collapse caused by overloading of a single task is reduced, and the system stability is improved; it is suitable for task scheduling optimization in a multi-core processor environment, and enhances the multi-task collaboration capability.

[0025] However, the above load balancing adjustment method still has the following disadvantages:

[0026] Coarse scheduling granularity: only task level load evaluation and migration is performed, and it cannot be refined to the model session level, so multiple concurrent business sessions still need to create corresponding number of threads, resulting in large memory overhead;

[0027] Limited real-time performance: due to the use of periodic evaluation mechanism, the response speed is slow, and it is difficult to meet the low delay and high concurrency AI interaction requirements.

[0028] High algorithm complexity and poor adaptability: load evaluation and migration decision depends on complex calculation (such as task type classification algorithm), and the effect is not good on a single-core low-power CPU. SUMMARY

[0029] The present application provides a dynamic load balancing method and system based on session granularity and bit mask mapping, which aims to solve the problems of increasing response delay, insufficient dynamic adaptability of scheduling, and system instability caused by resource competition when supporting multi-model cloud session concurrency on resource-constrained embedded devices; The present application is implemented through the following technical solutions.

[0030] In a first aspect, the present application provides a dynamic load balancing method based on session granularity and bit mask mapping, comprising:

[0031] S100, system initialization: creating a specified number of task threads, creating a timer and a mutex for executing load monitoring;

[0032] S200, thread management: creating a session by an application, adding the session created by the application to the task thread, and specifying an initial task or assigning an initial task by the system;

[0033] S300, load monitoring: start a timer at a preset frequency, collect the load data of each thread, calculate the average load, detect whether the load of any thread exceeds a preset threshold, if yes, trigger the balancing decision, if no, directly trigger the session execution operation;

[0034] S400, balancing decision: find the target migration task with the least load, select the migration candidate session based on the load and importance;

[0035] S500, migration execution: remove from the source task, add to the target task, and update the bitmask state of the session;

[0036] S600, session execution: each task thread executes the session assigned to it in its main loop according to the bitmap state, and updates the execution time and frequency, and regularly checks the stack usage;

[0037] S700, system exit: stop the timer, destroy all task threads, and release system resources.

[0038] As a preferred technical solution, the preset threshold in step S300 is obtained by weighted operation of the average load of each thread collected.

[0039] As a preferred technical solution, the specific steps of load monitoring in step S300 include:

[0040] S310, start a timer at a preset frequency;

[0041] S320, judge whether the load balancing function is enabled, if yes, execute step S330, otherwise execute step S380;

[0042] S330, execute and complete thread traversal;

[0043] S340, judge whether the current number of threads is greater than 0, if yes, execute step S350, otherwise execute step S380;

[0044] S350, calculate the average load of each thread collected;

[0045] S360, execute thread traversal and judge whether it is completed, if yes, execute step S380, otherwise execute step S370;

[0046] S370, judge whether the current thread load is greater than the preset threshold, if yes, execute step S400 and execute step S360, otherwise directly execute step S360;

[0047] S380, end load monitoring.

[0048] As a preferred technical solution, in step S330, the execution thread is traversed while judging whether it is completed, yes, then step S340 is executed, otherwise, the following step S331 is executed:

[0049] S331, judging whether it is cold start task, yes, then the current thread is skipped and the traversal operation is continued, otherwise, the thread total load is increased and the thread number is increased, and then step S330 is returned.

[0050] As a preferred technical solution, in step S360, after the thread traversal is completed, the following step S361 is executed first:

[0051] S361, judging whether the average load of each thread collected is greater than a preset maximum load threshold, yes, then the preset frequency in step S310 is increased and step S380 is executed, otherwise, step S380 is directly executed.

[0052] As a preferred technical solution, in step S300, when collecting the load data of each thread, the weight is calculated and the thread load is evaluated in combination with multi-dimensional load indicators; after the session execution in step S600 is completed, the load indicators are updated and then step S700 is executed; the load indicators include at least one of the following elements: task execution time, execution times, stack usage, session delay.

[0053] As a preferred technical solution, in step S400, the specific steps of load monitoring include:

[0054] S410, judging whether the task reuse function has been enabled, yes, then step S420 is executed, otherwise, step S460 is executed;

[0055] S420, executing and completing thread traversal;

[0056] S430, judging whether the current thread id is valid, yes, then step S440 is executed, otherwise, step S460 is executed;

[0057] S440, executing and completing session traversal;

[0058] S450, selecting and judging whether the candidate session is valid, yes, then step S500 is executed and then step S460 is executed, otherwise, step S460 is directly executed;

[0059] S460, ending the balancing decision.

[0060] As a preferred technical solution, in step S420, the execution thread is traversed while judging whether it is completed, yes, then step S430 is executed, otherwise, the following steps S421, S422 and S423 are executed:

[0061] S421, judging whether a cold start task is executed, yes, skipping the current thread to continue the iteration operation, otherwise, executing step S422;

[0062] S422, judging whether the current thread load is less than the minimum load in the traversed thread load, yes, executing step S423 to continue the iteration operation, otherwise, skipping the current thread to continue the iteration operation;

[0063] S423, updating the minimum load in the traversed thread load as the current thread load, and performing the self-increment operation on the thread number, and continuing the iteration operation.

[0064] As a preferred technical solution, in step S440, the session traversal is executed while judging whether it is completed, yes, executing step S450, otherwise, executing the following steps S441, S442 and S443:

[0065] S441, judging whether the current session is a fixed session, yes, skipping the current session to continue the iteration operation, otherwise, executing step S442;

[0066] S442, calculating the migration score and judging whether the migration score is greater than the maximum session score in the traversed session, yes, executing step S443 to continue the iteration operation, otherwise, skipping the current session to continue the iteration operation;

[0067] S443, updating the maximum session score in the traversed session as the migration score of the current session, and performing the self-increment operation on the candidate session, and continuing the iteration operation.

[0068] In a second aspect, the application provides a dynamic load balancing system based on session granularity and bit mask mapping, comprising:

[0069] A system initialization module: used for creating a specified number of task threads, and used for creating a timer and a mutex for executing load monitoring;

[0070] A thread manager: used for adding the session created by the application to the task thread, and used for specifying the initial task or assigning the initial task by the system;

[0071] A load monitor: used for starting the timer at a regular time, collecting thread load data, calculating the average load, detecting whether the thread load exceeds a preset threshold, if yes, triggering the balancing decision maker, if no, directly triggering the session executor;

[0072] A balancing decision maker: used for finding the most idle target migration task, and selecting the migration candidate session based on the load and the importance;

[0073] A migration executor: used for removing from the source task, adding to the target task, and updating the bit mask state of the session;

[0074] Session executor: each task thread executes the session assigned to it according to the bitmap state in its main loop, and updates the execution time and number of times, and checks the stack usage periodically;

[0075] System exit module: used to stop the timer, destroy all task threads, and release system resources.

[0076] The application aims at the problems of resource competition leading to increased response delay, insufficient dynamic adaptability of traditional scheduling, and system instability caused by memory fragmentation in supporting multi-model cloud session concurrency for resource-limited embedded devices, and proposes a dynamic load balancing method and system based on session granularity and bitmask mapping, which effectively solves the bottleneck problems of high response delay and large memory overhead of key sessions in the multi-model session concurrency process. The scheme provides a lightweight, high-performance and highly scalable multi-model session concurrency scheduling solution for resource-limited embedded devices, and is suitable for various AI-enabled application scenarios such as smart home, industrial Internet of Things, and edge computing. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0078] Figure 1 is the main flowchart of the dynamic load balancing method based on session granularity and bitmask mapping provided by the embodiments of the application.

[0079] Figure 2 is the specific flowchart of the load monitoring link in the dynamic load balancing method based on session granularity and bitmask mapping provided by the embodiments of the application.

[0080] Figure 3 is the specific flowchart of the balancing decision link in the dynamic load balancing method based on session granularity and bitmask mapping provided by the embodiments of the application.

[0081] Figure 4 is the constituent block diagram of the dynamic load balancing system based on session granularity and bitmask mapping provided by the embodiments of the application. DETAILED DESCRIPTION

[0082] In order to make the technical solutions of the present application clearer and the technical advantages more obvious, the technical solutions of the present application will be described clearly and completely in combination with specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the disclosed scope of the present application.

[0083] In order to make the technical solutions of the present application clearer and the technical advantages more obvious, the technical solutions of the present application will be described clearly and completely in combination with specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the disclosed scope of the present application.

[0084] It should be noted that, in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or further include elements inherent in such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article, or device including the element, in addition, components, features, elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and the specific meaning thereof should be determined in combination with the explanation thereof in the specific embodiment or further in combination with the context in the specific embodiment.

[0085] It should be understood that, although each step in the flowchart in the embodiments of the present application is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless explicitly stated in this document, the execution of these steps has no strict sequence limitation, and they can be executed in other orders. Moreover, at least part of the steps in the figure can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order is not necessarily sequential, but can be executed in rotation or alternation with other steps or sub-steps or stages of other steps.

[0086] The application aims at the problems of resource competition leading to response delay increase, insufficient dynamic adaptability of traditional scheduling, and system instability caused by memory fragmentation when resource-limited embedded devices support multi-model cloud session concurrency, and proposes a dynamic load balancing method and system based on session granularity and bitmask mapping. By introducing a task allocation mechanism based on bitmask, lightweight metadata management, and multi-dimensional load perception scheduling algorithm, the bottleneck problems of high response delay of key sessions and large memory overhead in the multi-model session concurrency process are effectively solved. The application provides a lightweight, high-performance, and highly scalable multi-model session concurrency scheduling solution for resource-limited embedded devices, which is suitable for various AI-enabled application scenarios such as smart home, industrial Internet of Things, and edge computing.

[0087] In combination Figure 1 As shown in the basic embodiment of the application, a dynamic load balancing method based on session granularity and bitmask mapping is provided, which includes:

[0088] S100, system initialization: creating a specified number of task threads, creating a timer and a mutex lock for executing load monitoring;

[0089] S200, thread management: creating a session by an application, adding the session created by the application to a task thread, and specifying an initial task or assigning an initial task by the system;

[0090] S300, load monitoring: starting the timer at a regular time, collecting thread load data, calculating the average load, detecting whether the load of any thread exceeds a preset threshold, triggering balancing decision if yes, and directly triggering session execution operation if no;

[0091] S400, balancing decision: finding the most idle target migration task, and selecting migration candidate sessions based on load and importance;

[0092] S500, migration execution: removing from the source task, adding to the target task, and updating the bitmask state of the session;

[0093] S600, session execution: each task thread executes the session assigned to it according to the bitmap state in its main loop, updates the execution time and number, and regularly checks the stack usage;

[0094] S700, system exit: stopping the timer, destroying all task threads, and releasing system resources.

[0095] The preset threshold in step S300 is obtained by weighted operation of the average load of each thread collected, for example, the average load of each thread is denoted as avg load, the total load of each thread is denoted as total load, the number of threads collected is denoted as active tasks, avg load = total load / active tasks, and the preset threshold = avg load*150%, that is, the weighted coefficient is 150%.

[0096] In combination Figure 2 As shown in the figure, the specific steps of load monitoring in step S300 include:

[0097] S310, starting a timer at a preset frequency;

[0098] S320, judging whether the load balancing function has been enabled, yes to execute step S330, otherwise execute step S380;

[0099] S330, executing and completing thread traversal;

[0100] S340, judging whether the current number of threads is greater than 0, yes to execute step S350, otherwise execute step S380;

[0101] S350, calculating the average load of each thread collected;

[0102] S360, executing thread traversal and judging whether it is completed, yes to execute step S380, otherwise execute step S370;

[0103] S370, judging whether the current thread load is greater than the preset threshold, yes to execute step S400 and execute step S360, otherwise directly execute step S360;

[0104] S380, ending load monitoring.

[0105] Specifically, in step S330, while executing thread traversal, it is judged whether it is completed, yes to execute step S340, otherwise execute the following step S331:

[0106] S331, judging whether it is a cold start task, yes to skip the current thread and continue to execute the traversal operation, otherwise the total load of the thread is self-incremented, the thread number is self-incremented, and step S330 is returned.

[0107] In addition, in step S360, after judging that the thread traversal is completed, the following step S361 is executed first:

[0108] S361, judging whether the average load of each thread collected is greater than a preset maximum load threshold, if yes, executing step S380 after increasing the preset frequency in step S310, otherwise, directly executing step S380.

[0109] In the above dynamic load balancing method based on session granularity and bitmask mapping, in step S300 of collecting thread load data, the weight is calculated and the thread load is evaluated in combination with multi-dimensional load indicators; after the session execution in step S600 is completed, step S700 is executed after the load indicators are updated; the load indicators include at least one of the following elements: task execution time, execution times, stack usage, session delay.

[0110] In combination with Figure 3 As shown in the figure, the specific steps of load monitoring in step S400 include:

[0111] S410, judging whether the task reuse function has been enabled, if yes, executing step S420, otherwise, executing step S460;

[0112] S420, executing and completing thread traversal;

[0113] S430, judging whether the current thread id is valid, if yes, executing step S440, otherwise, executing step S460;

[0114] S440, executing and completing session traversal;

[0115] S450, selecting and judging whether the candidate session is valid, if yes, executing step S500 and then executing step S460, otherwise, directly executing step S460;

[0116] S460, ending the balancing decision.

[0117] In step S420, while executing thread traversal, it is judged whether it is completed, if yes, executing step S430, otherwise, executing the following steps S421, S422 and S423:

[0118] S421, judging whether it is a cold start task, if yes, skipping the current thread to continue the traversal operation, otherwise, executing step S422;

[0119] S422, judging whether the current thread load is less than the minimum load in the traversed thread load, if yes, executing step S423 and then continuing the traversal operation, otherwise, skipping the current thread to continue the traversal operation;

[0120] S423, updating the minimum load in the traversed thread load to the current thread load, and performing the self-increment operation on the thread number, and continuing the traversal operation.

[0121] Further, in step S440, it is judged whether the traversal of the session is completed while the traversal of the session is performed, if yes, step S450 is performed, otherwise, steps S441, S442 and S443 are performed:

[0122] S441, it is judged whether the current session is a fixed session, if yes, the traversal operation is continued by skipping the current session, otherwise, step S442 is performed;

[0123] S442, the migration score is calculated and it is judged whether the migration score is greater than the maximum session score in the traversed session, if yes, step S443 is performed and then the traversal operation is continued, otherwise, the traversal operation is continued by skipping the current session;

[0124] S443, the maximum session score in the traversed session is updated as the migration score of the current session, the candidate session is incremented, and the traversal operation is continued.

[0125] The key means and technical effects of the above-mentioned dynamic load balancing method based on session granularity and bit mask mapping are as follows:

[0126] (1) Migration strategy based on session granularity: Unlike task-level migration (such as the patent of CoreWeave), the present application migrates in units of sessions.

[0127] Lossless migration mechanism is adopted:

[0128] Atomic bitmap operation (os_atomic_bit_clear / set) ensures zero interruption during migration;

[0129] Cold start protection period (COLD_START_PERIOD) avoids mis-migration of new sessions;

[0130] Fixed session marking (pinned) guarantees the continuity of critical services.

[0131] (2) Bit mask-metadata coupling mechanism is adopted: sessions and threads are associated through, for example, 32-bit bit masks to realize O(1) complexity session positioning.

[0132] (3) Dynamic load balancing algorithm is adopted: based on multi-dimensional load indicators (delay, task stack), the migration priority is calculated to ensure optimal allocation of resources.

[0133] Multi-dimensional load model, for example:

[0134] Load = a * (ΣExecTime / Count) + b * (1 - StackRemain / StackSize) * K (K = 1000); wherein, ExecTime is execution time, Count is count value, StackRemain is remaining stack size, and StackSize is application stack size.

[0135] (4) Dynamic monitoring frequency adjustment: based on average thread load, dynamically adjust the timer trigger frequency when exceeding the threshold, double the monitoring frequency when the load is high.

[0136] In combination Figure 4 Another basic embodiment of the application provides a dynamic load balancing system based on session granularity and bit mask mapping, comprising:

[0137] System initialization module: for creating a specified number of task threads, for creating a timer and mutex for executing load monitoring;

[0138] Thread manager: for adding sessions created by the application to the task thread, and specifying the initial task or assigning the initial task by the system;

[0139] Load monitor: for starting the timer regularly, collecting thread load data, calculating the average load, detecting whether the thread load exceeds the preset threshold, and if so, triggering the balancing decision maker, and if not, directly triggering the session executor;

[0140] Balancing decision maker: for finding the most idle target migration task, and selecting migration candidate sessions based on load and importance;

[0141] Migration executor: for removing from the source task, adding to the target task, and updating the bit mask state of the session;

[0142] Session executor: each task thread executes the session assigned to it according to the bitmap state in its main loop, and updates the execution time and number of times, and regularly checks the stack usage;

[0143] System exit module: for stopping the timer, destroying all task threads, and releasing system resources.

[0144] This scheme mainly works through the cooperation of the four core modules of thread manager, load monitor, balancing decision maker, and migration executor, reduces the situation of thread starvation or overload, realizes thread load balancing, improves the processing capacity of threads for multiple session concurrency, reduces response delay, and the detailed architecture design is as follows.

[0145] I. Thread manager:

[0146] 1.1 Function description:

[0147] Interface provides: provides the interface API for application manager creation, release, thread registration, destruction, etc. Allow applications to allocate system resources (such as thread number, task stack size).

[0148] Metadata management: maintain the mapping relationship between session and thread, use bitmap to record the session set of each thread. For example, the session set of each thread is represented by a bitmap, and each bit of the bitmap corresponds to a session ID. 1 means that the session belongs to the current thread, and 0 means that it does not.

[0149] Session management: binding, unbinding, state information (such as session importance coefficient, whether the session is allowed to migrate) maintenance of session and thread.

[0150] 1.2, interaction mechanism:

[0151] With load monitor: Provide session task metadata (such as thread allocation information) for load monitor to collect.

[0152] With migration executor: coordinate the migration executor to complete the session migration and update the session bitmap state.

[0153] Two, load monitor:

[0154] 2.1, function description:

[0155] Load data collection: Collect real-time load indicators of each thread through a timer, such as execution time, call times, stack size, session delay, etc.

[0156] 2.2, load weight calculation:

[0157] Thread load = α * time load + β * stack load = α * (execution time / call times) + β * (1 - remaining stack size / allocated stack size) * 1000;

[0158] Session load = maximum execution time * (1 - importance coefficient).

[0159] 2.3, interaction mechanism:

[0160] With thread manager: Request the thread manager to provide the current session allocation state (bitmap) for associating load data with specific sessions.

[0161] With balancing decision maker: Transfer the collected load data (such as thread ID, load weight, resource consumption details) to the balancing decision maker as the basis for migration decision.

[0162] Three, balancing decision maker:

[0163] 3.1, function description:

[0164] Migration trigger condition: When the thread load weight exceeds the preset threshold (such as 150% of the average load), the migration strategy is triggered.

[0165] Migration target selection: Based on the weight ranking of thread and session load, select the thread with the lowest load and meet the non-cold start, and select the target session with the highest load and meet the non-fixed session constraint.

[0166] 3.2, Interaction mechanism:

[0167] With load monitor: Receive real-time load data from load monitor, trigger migration decision.

[0168] With migration executor: Send migration instruction, require to update session allocation.

[0169] Four, migration executor:

[0170] 4.1, function description:

[0171] Execute migration: Remove from source task, add to target task, update session bitmask state.

[0172] Update session metadata: Update migration counter.

[0173] 4.2, interaction mechanism:

[0174] With thread manager: Request thread manager to provide current session allocation state (bitmask) for updating state after migration.

[0175] With balancing decision maker: Feedback migration result (such as success / failure state) for balancing decision maker to optimize subsequent decision strategy.

[0176] Next, through the detailed data of specific examples, the processing process and beneficial effects of the present application are described:

[0177] Assume the system configuration is as follows:

[0178]

[0179] Session configuration is as follows:

[0180] Session ID Type Importance Fixed state S0 STT 0.8 No S1 TTS 0.3 No S2 LLM 0.9 Yes S3 IR 0.5 No

[0181] Initial allocation is as follows:

[0182] tasks[0]: S1, S2 (bitmap = 0b11)

[0183] tasks[1]: S3, S4 (bitmap = 0b1100)

[0184] Load monitoring data (before migration):

[0185]

[0186] Load analysis:

[0187] Average system load: (472.5 + 110) / 2 = 291.25;

[0188] Overload determination:

[0189] tasks[0] load 472.5 > 291.25 * 1.5 = 436.875 → trigger migration;

[0190] tasks[1] load 110 < 436.875 → normal;

[0191] Migration decision:

[0192] Target task: tasks[1] (lowest load);

[0193] Candidate session filtering:

[0194] S1: importance high (0.8), migration score low (300ms * (1 - 0.8) = 60).

[0195] S2: importance low (0.3), migration score high (300ms * (1 - 0.3) = 210), and non-fixed session, allow migration.

[0196] S3: fixed session, skip.

[0197] S4: already in target thread.

[0198] Perform migration:

[0199] Atomic bitmap update:

[0200] tasks[0].bitmap &= ~(1 << 1); / / 0b0011 → 0b0001 (remove S2);

[0201] tasks[1].bitmap |= (1 << 1); / / 0b1100 → 0b1110 (add S2).

[0202] Update session metadata:

[0203] S2's task_id = 1.

[0204] Load change (after migration):

[0205]

[0206]

[0207] Optimization effect comparison:

[0208]

[0209] Key conclusions:

[0210] (1) Load balancing takes effect:

[0211] The load of the overloaded thread (tasks[0]) is reduced by 53.4%;

[0212] The overall system load is reduced by 12%;

[0213] The maximum delay is reduced by 20%.

[0214] (2) Resource optimization:

[0215] The stack usage of the high-load thread is reduced from the dangerous value (87.5%) to the safe range (50%);

[0216] The overall system stack usage is more balanced.

[0217] (3) Strategy effectiveness:

[0218] The low-importance session (S2) is migrated preferentially;

[0219] The fixed session (S3) remains in place;

[0220] The comprehensive load model accurately identifies the overloaded thread.

[0221] Migration effect verification: After migration, the load of tasks[0] is 220, which is less than the system average load of 256.25 x 1.5 = 384.375, so the overloaded state is resolved and the system returns to balance. The overall system delay is reduced and the resource utilization is improved.

[0222] From the above, the beneficial effects brought by the technical scheme of the present application are as follows:

[0223]

[0224]

[0225] The specific technical effects are explained as follows:

[0226] (1) Response delay optimization:

[0227] Key session guarantee: Through fixed session marking (pinned) and dynamic priority adjustment, the response delay of key sessions such as STT is reduced from 120ms in the traditional scheme to 40ms, meeting the real-time interaction requirements (such as voice command response time <100ms).

[0228] Dynamic load balancing: Load evaluation and session migration every 50 ms, responding to load mutations in time, and narrowing the ordinary session delay fluctuation range from ±50 ms to ±20 ms.

[0229] (2) Memory efficiency improvement:

[0230] Metadata lightweight: Using 4-byte bitmap, 4-byte importance, and other 8-byte identifiers, session metadata is reduced from 200 bytes in traditional TCB to 16 bytes, with a 87.5% memory utilization improvement.

[0231] Stack dynamic allocation: Allocating session stacks on demand (average 4 KB / session), saving 50% memory space compared to traditional fixed allocation (8 KB / session), supporting more concurrent sessions (≥8 sessions per chip).

[0232] (3) Algorithm complexity and adaptability:

[0233] Lightweight algorithm: Bitmask operations (such as bitmap&(1<<i)) and linear scanning are used to implement session allocation and migration, with a computational complexity of only O(N), and can handle ≥1000 scheduling decisions per second on an ARM Cortex-M4 (168 MHz).

[0234] Cross-platform support: Through the hardware abstraction layer (HAL) and RTOS adaptation layer, the core logic can be ported to different architectures (such as from ARM to RISC-V) without modification, with an adaptation period shortened from 2 weeks to 1 day.

[0235] (4) Resource utilization:

[0236] Task reuse mode: Supports task pool reuse (such as TTS / STT sharing CPU task pool), reducing task creation and switching overhead, and CPU core utilization from 60% to 85%.

[0237] (5) Social and economic benefits:

[0238] Device cost reduction: Through resource optimization, lower-spec hardware can be used.

[0239] Development efficiency improvement: Standardized session management interface shortens application development cycle by 40%, supporting rapid iteration.

[0240] Scenario expansion: Enabling low-power devices to implement complex AI functions (such as multi-turn dialogue + image recognition), promoting the intelligent upgrade of smart home, industrial Internet of Things, and other fields.

[0241] The application realizes overall optimization of delay, memory, power consumption and portability in multi-model conversation scheduling of an embedded device through technical innovation, and provides a high-performance solution for a resource-constrained scene.

[0242] The above only discloses preferred embodiments of the application, and of course cannot limit the scope of the right of the application, so the equivalent changes made according to the claims of the application still belong to the scope covered by the application.

Claims

1. A dynamic load balancing method based on session granularity and bitmask mapping, characterized in that, Comprise: S100, system initialization: create a specified number of task threads, create a timer to execute load monitoring and a mutex; S200, thread management: application creates a session, and adds the application created session to the task thread, specifies the initial task or assigns the initial task by the system; S300, load monitoring: start the timer at a predetermined frequency, collect thread load data, detect whether the load of any thread exceeds a predetermined threshold, if so, trigger the balancing decision, if not, directly trigger the session execution operation; S400, balancing decision: find the most idle target migration task, select the migration candidate session based on the load and importance; S500, migration execution: remove from the source task, add to the target task, update the session's bitmask state; S600, session execution: each task thread executes the session assigned to it in its main loop according to the bitmap state, and updates the execution time and number of times, and periodically checks the stack usage; S700, system exit: stop the timer, destroy all task threads, and release system resources.

2. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 1, characterized in that, The predetermined threshold in step S300 is obtained by weighted operation of the average load of each thread collected.

3. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 1, wherein, The specific steps of load monitoring in step S300 include: S310, start the timer at a predetermined frequency; S320, determine whether the load balancing function has been enabled, if yes, execute step S330, otherwise execute step S380; S330, execute and complete thread traversal; S340, determine whether the current number of threads is greater than 0, if yes, execute step S350, otherwise execute step S380; S350, calculate the average load of each thread collected; S360, execute thread traversal and determine whether it is completed, if yes, execute step S380, otherwise execute step S370; S370, determine whether the current thread load is greater than the predetermined threshold, if yes, execute step S400 and execute step S360, otherwise directly execute step S360; S380, end load monitoring.

4. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 3, characterized in that, In step S330, while executing thread traversal, determine whether it is completed, if yes, execute step S340, otherwise execute the following step S331: S331, determine whether it is a cold start task, if yes, skip the current thread and continue to execute the traversal operation, otherwise, the thread total load is incremented and the thread number is incremented, and return to step S330.

5. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 3, wherein, In step S360, after determining that the thread traversal is completed, first execute the following step S361: S361, determine whether the average load of each thread collected is greater than a predetermined maximum load threshold, if yes, increase the predetermined frequency in step S310 and execute step S380, otherwise directly execute step S380.

6. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 1, characterized in that, In step S300, when collecting thread load data, the weight is calculated and the thread load is evaluated in combination with multi-dimensional load indicators; after the session execution in step S600 is completed, the load indicators are updated before executing step S700; the load indicators include at least one of the following elements: task execution time, execution times, stack usage, and session delay.

7. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 1, wherein, The specific steps of load monitoring in step S400 include: S410, judging whether the task multiplexing function has been enabled, yes, executing step S420, otherwise executing step S460; S420, executing and completing thread traversal; S430, judging whether the current thread id is valid, yes, executing step S440, otherwise executing step S460; S440, executing and completing session traversal; S450, selecting and judging whether the candidate session is valid, yes, executing step S500 and then executing step S460, otherwise directly executing step S460; S460, ending the balancing decision.

8. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 7, characterized in that, In step S420, while executing thread traversal, it is judged whether it is completed, yes, executing step S430, otherwise executing the following steps S421, S422 and S423: S421, judging whether it is cold start task, yes, skipping the current thread and continuing to execute the traversal operation, otherwise executing step S422; S422, judging whether the current thread load is less than the minimum load in the traversed thread load, yes, executing step S423 and then continuing to execute the traversal operation, otherwise skipping the current thread and then continuing to execute the traversal operation; S423, updating the minimum load in the traversed thread load as the current thread load, and performing self-increment operation on the thread number, and continuing to execute the traversal operation.

9. The method for dynamic load balancing based on session granularity and bitmask mapping according to claim 7, characterized in that, In step S440, while executing session traversal, it is judged whether it is completed, yes, executing step S450, otherwise executing the following steps S441, S442 and S443: S441, judging whether the current session is fixed session, yes, skipping the current session and continuing to execute the traversal operation, otherwise executing step S442; S442, calculating the migration score and judging whether the migration score is greater than the maximum session score in the traversed session, yes, executing step S443 and then continuing to execute the traversal operation, otherwise skipping the current session and then continuing to execute the traversal operation; S443, updating the maximum session score in the traversed session as the migration score of the current session, and performing self-increment operation on the candidate session, and continuing to execute the traversal operation.

10. A dynamic load balancing system based on session granularity and bitmask mapping, characterized in that, It comprises: System initialization module: used for creating a specified number of task threads, for creating a timer and a mutex for executing load monitoring; Thread manager: used for adding the session created by the application to the task thread, and specifying the initial task or assigning the initial task by the system; Load monitor: used for starting the timer regularly, collecting thread load data, detecting whether there is a thread load exceeding a preset threshold, if yes, triggering the balancing decision maker, if no, directly triggering the session executor; Balancing decision maker: used for finding the most idle target migration task, and selecting the migration candidate session based on the load and importance; Migration executor: used for removing from the source task, adding to the target task, and updating the bit mask state of the session; Session executor: each task thread executes the session assigned to it according to the bitmap state in its main loop, and updates the execution time and times, and regularly checks the stack usage; System exit module: used for stopping the timer, destroying all task threads, and releasing system resources.

Images