A RISC-V multi-core heterogeneous platform intelligent load balancing method and system

Abstract

The application provides an RISC-V multi-core heterogeneous platform intelligent load balancing method and system, and relates to the technical field of resource allocation and scheduling. Micro-architecture performance data of each processing core in the RISC-V multi-core heterogeneous platform is acquired to construct a state vector; the micro-architecture performance data comprises instruction cycle number, cache miss rate at each level and memory pause proportion; the state vector is input into a pre-trained deep Q network model to generate optimal action instructions, so that the load balancing of thread resources and core capacity is realized; the optimal action instructions comprise thread migration, thread exchange and core frequency adjustment; the pre-training of the deep Q network model is performed on a parallel computing program running on the RISC-V platform; the environment is randomly disturbed before the program runs; the action is executed through a random strategy, and state, action and reward data are collected; an offline experience dataset is constructed to perform pre-training. Dynamic, cooperative and adaptive optimization of parallel computing of the RISC-V multi-core heterogeneous platform is realized.

Description

Technical Field

[0001] This invention relates to the field of resource allocation and scheduling technology, and in particular to an intelligent load balancing method and system for a RISC-V multi-core heterogeneous platform. Background Technology

[0002] RISC-V multi-core heterogeneous platforms achieve a balance between computing power and energy efficiency by integrating computing power and instruction sets to expand different cores (such as scalar cores and vector cores). They have become a core hardware solution for high-performance computing, edge AI and other scenarios. The performance of their parallel computing is highly dependent on load balancing technology, that is, by reasonably scheduling tasks and resources, each core load is matched with its own capacity to avoid idleness or overload.

[0003] With the surge in computing power demands from scenarios such as HPC and AI training, the architectural complexity of RISC-V multi-core heterogeneous platforms continues to increase. Significant differences exist in instruction set compatibility, memory access latency, and computational efficiency among different cores, making traditional load scheduling methods based on static rules or experience insufficient to handle dynamically changing computing needs. Especially in parallel computing tasks, uneven load distribution among cores not only wastes hardware potential but can also exacerbate performance degradation due to resource contention, becoming a key bottleneck restricting the platform's computing power release.

[0004] Existing load balancing solutions have significant drawbacks: Firstly, scheduling methods that rely on manual configuration or fixed strategies cannot dynamically adapt to the customized characteristics and heterogeneous differences of the RISC-V architecture, resulting in poor versatility. Secondly, some dynamic scheduling algorithms lack accurate perception of microarchitecture performance data, making it difficult to capture real-time bottlenecks such as cache misses and memory pauses, leading to delayed or biased scheduling decisions. At the same time, traditional algorithms do not fully utilize multi-dimensional performance data for intelligent learning, failing to achieve adaptive optimization of load distribution, ultimately resulting in low parallel computing efficiency and untapped hardware potential. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes an intelligent load balancing method and system for RISC-V multi-core heterogeneous platforms. This method enables dynamic, collaborative, and adaptive optimization of parallel computing on RISC-V multi-core heterogeneous platforms, resolving performance and energy efficiency bottlenecks caused by static configuration and homogeneous scheduling.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides an intelligent load balancing method for a RISC-V multi-core heterogeneous platform, comprising:

[0008] The microarchitecture performance data of each processing core in the RISC-V multi-core heterogeneous platform is obtained and constructed into a state vector; the microarchitecture performance data includes instruction cycle count, cache miss rate at each level, and memory pause percentage.

[0009] The state vector is input into a pre-trained deep Q-network model to generate optimal action instructions, thereby achieving load balancing between thread resources and core capabilities. The optimal action instructions include thread migration, thread swapping, and core frequency adjustment.

[0010] The pre-training of the deep Q-network model involves running a parallel computing program on a RISC-V platform. Before the program runs, the environment is randomly perturbed, actions are executed through random policies, and state, action, and reward data are collected to construct an offline experience dataset for pre-training.

[0011] Secondly, the present invention provides an intelligent load balancing system for a RISC-V multi-core heterogeneous platform, comprising:

[0012] The state vector acquisition module is configured to acquire microarchitecture performance data of each processing core in the RISC-V multi-core heterogeneous platform and construct it into a state vector; the microarchitecture performance data includes instruction cycle count, cache miss rate at each level, and memory pause percentage.

[0013] The load balancing module is configured to input the state vector into a pre-trained deep Q-network model to generate optimal action instructions, thereby achieving load balancing between thread resources and core capabilities; the optimal action instructions include thread migration, thread swapping, and core frequency adjustment.

[0014] The pre-training of the deep Q-network model involves running a parallel computing program on a RISC-V platform. Before the program runs, the environment is randomly perturbed, actions are executed through random policies, and state, action, and reward data are collected to construct an offline experience dataset for pre-training.

[0015] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the intelligent load balancing method for a RISC-V multi-core heterogeneous platform described in the first aspect.

[0016] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the intelligent load balancing method for a RISC-V multi-core heterogeneous platform described in the first aspect.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] (1) This invention constructs a state vector by collecting microarchitecture performance data of each processing core of a RISC-V multi-core heterogeneous platform, and inputs it into a pre-trained deep Q-network model to generate optimal action instructions, thereby achieving load balancing. During its pre-training phase, the RISC-V platform environment is randomly perturbed, and diverse empirical data is collected, allowing the model to fully learn hardware characteristics. It can dynamically adapt to the differences in RISC-V heterogeneous cores, accurately capture microarchitecture performance bottlenecks and optimize them in real time, breaking through the limitations of the universality and timeliness of traditional manual or static strategies, significantly improving the platform's parallel computing efficiency, fully releasing the potential of multi-core heterogeneous hardware, and providing intelligent scheduling support for the deployment of RISC-V computing power in high-performance computing, edge AI, and other scenarios, combining architectural adaptability and performance optimization.

[0019] (2) Existing homogeneous scheduling strategies cannot identify the microarchitectural differences between heterogeneous cores such as "big cores" and "small cores" in the RISC-V platform, leading to a mismatch between tasks and core capabilities. This invention enables the DQN decision engine to accurately "profile" the current advantages of each core by real-time monitoring of underlying hardware metrics such as IPC and cache miss rate, and dynamically schedules tasks with different characteristics such as computationally intensive and memory-intensive tasks to the most suitable core. This fine-grained scheduling complements the shortcomings of traditional methods, fully unleashing the parallel computing potential of heterogeneous platforms, achieving higher GFLOPS performance scores in benchmark tests such as HPL, or significantly shortening runtime under the same computational load.

[0020] (3) Traditional static configuration schemes, which require "configuration once and use throughout the entire process," cannot cope with the changing load characteristics of parallel computing at different stages (such as initialization, main loop, and data aggregation). Once the initial configuration is poor or the operating conditions change, the performance will drop sharply. This invention establishes a continuous, real-time feedback optimization loop. No matter which stage the program is running at or how the load fluctuates, the system can perceive changes within a millisecond timescale and make optimal adjustments quickly. This powerful adaptive capability enables high-performance computing to break free from its fragile dependence on specific "golden parameters" and exhibit higher performance stability and robustness in diverse application scenarios.

[0021] (4) This invention uses deep reinforcement learning to automatically learn and solidify this complex, multi-dimensional, and nonlinear optimization knowledge into the DQN model. It transforms the complex manual tuning process into an automated "online learning and decision-making" process, which greatly shortens the optimization cycle and enables non-professional users to run their computing tasks efficiently on complex heterogeneous hardware, thus lowering the threshold for using high-performance computing.

[0022] (5) Existing optimization techniques are mostly single-dimensional optimizations, such as adjusting only thread allocation or focusing only on energy saving. This invention innovatively integrates multiple optimization dimensions, such as thread space scheduling, core energy consumption control, application algorithm granularity, and hardware resource isolation, into a global DQN decision framework. The AI ​​engine can weigh the pros and cons based on the overall state and perform multi-dimensional collaborative optimization, which can solve the problem that single-point optimization may cause new bottlenecks, thereby achieving the globally optimal system operating state.

[0023] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0025] Figure 1 This is a schematic diagram of the overall architecture of an intelligent load balancing method for a RISC-V multi-core heterogeneous platform provided in an embodiment of the present invention;

[0026] Figure 2 This is an overall flowchart of an intelligent load balancing method for a RISC-V multi-core heterogeneous platform provided in an embodiment of the present invention;

[0027] Figure 3 This is a flowchart of the training process for a deep Q-network model provided in an embodiment of the present invention;

[0028] Figure 4 A detailed flowchart of the real-time closed-loop optimization loop provided in the embodiments of the present invention. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] Explanation of technical terms

[0031] 1. Multi-core heterogeneous platform: An advanced computing architecture that integrates multiple different types of processing cores on a single chip (SoC, System on a Chip). These different types can manifest as one or more of the following differences: different instruction set architectures, the same instruction set architecture but different microarchitectures, or dedicated function hardware units. Each of these cores excels at handling different types of tasks, working together to achieve higher performance and energy efficiency than a single type of core.

[0032] 2. HPL: HPL (High Performance Linpack) is a benchmark for testing the floating-point performance of high-performance computing cluster systems. It evaluates the floating-point performance of high-performance computers by using Gaussian elimination to solve a system of dense linear algebraic equations of degree N in one variable. It has high reference value in the industry.

[0033] 3. Intelligent Load Balancing: This is an intelligent upgrade of load balancing. Unlike traditional load balancing, which simply distributes network traffic or computing tasks evenly across servers or resources, intelligent load balancing introduces real-time monitoring, predictive analysis, and adaptive adjustment, enabling dynamic optimization and allocation based on the current system status.

[0034] 4. GFLOPS: GFLOPS (Giga Floating-point Operations Per Second) is a core indicator for measuring the floating-point computing performance of a computer (or program). The higher the value, the faster the computing speed.

[0035] Example 1

[0036] like Figure 1 As shown, this embodiment discloses an intelligent load balancing method for a RISC-V multi-core heterogeneous platform, including the following steps:

[0037] S1: Obtain the microarchitecture performance data of each processing core in the RISC-V multi-core heterogeneous platform and construct it into a state vector; the microarchitecture performance data includes instruction cycle count, cache miss rate at each level, and memory pause percentage;

[0038] S2: Input the state vector into a pre-trained deep Q-network model to generate optimal action instructions, thereby achieving load balancing between thread resources and core capabilities; the optimal action instructions include thread migration, thread swapping, and core frequency adjustment.

[0039] The pre-training of the deep Q-network model involves running a parallel computing program on a RISC-V platform. Before the program runs, the environment is randomly perturbed, actions are executed through random policies, and state, action, and reward data are collected to construct an offline experience dataset for pre-training.

[0040] Next, combined Figure 2 This embodiment provides a detailed description of an intelligent load balancing method for a RISC-V multi-core heterogeneous platform.

[0041] In S1, the microarchitectural performance data of each CPU core of the parallel computing program running on the platform is collected in real time by accessing the hardware performance counter (PMU) of the RISC-V processor.

[0042] The collected multi-dimensional, continuous microarchitecture performance data is normalized and concatenated into a fixed-length one-dimensional state vector, which serves as a representation of the current operating state of the computing platform.

[0043] Microarchitecture performance data includes instruction cycles per second (IPC), cache miss rates at each level, and memory pause percentages. Specifically:

[0044] (1) Instructions Per Cycle (IPC) is calculated by collecting data. (CPU clock cycles) and The number of instructions completed (calculated) is used to characterize instruction execution efficiency. The calculation formula is as follows:

[0045] ;

[0046] IPC directly measures the computing efficiency of a CPU core. A high IPC indicates that the core is busy and efficient, while a low IPC indicates that the core may be idle, waiting, or paused.

[0047] (2) Caching system performance, by collecting L1-DCACHE-LOADS (L1 data cache load events), L1-DCACHE-LOAD-MISSES (L1 data cache load miss events), L2-CACHE-LOADS ( L2 data cache load events), L2-CACHE-LOAD-MISSES ( Events such as L2 data cache load miss events are mentioned. The specific event name depends on the RISC-V PMU implementation. L1 is the L1 cache closest to the CPU core, while L2 is the L2 cache, which is slower and has a larger capacity. They are different levels of the CPU's multi-level cache.

[0048] The cache miss rate for each level is the ratio of the number of cache access misses to the total number of accesses for that level. To calculate the L2 cache miss rate... The formula is as follows:

[0049] ;

[0050] Measuring data access efficiency is crucial for identifying whether the core and task data volumes are matched. A high cache miss rate indicates that the core is spending a significant amount of time fetching data from slower memory tiers, a typical performance bottleneck in program execution.

[0051] (3) Memory subsystem pressure: Collect raw data STALL_MEM_ANY ( Pipeline pauses caused by any memory access or similar events. The formula for calculating the percentage of memory pauses is:

[0052] .

[0053] The acquired microarchitecture performance data is constructed into a one-dimensional floating-point array-based state vector S (or S'), for example, in the following form:

[0054] ;

[0055] in, The number of instruction cycles for the k-th core. This represents the L2 cache miss rate of the k-th core. This represents the percentage of memory pauses for the k-th core, where N is the maximum number of the processor core.

[0056] In this embodiment, microarchitectural performance data such as instruction cycles per minute (IPC), cache miss rate, and memory pause percentage of each core in the RISC-V processor are collected and a state vector is constructed. This accurately characterizes the computational efficiency, data access efficiency, and memory pressure of each core. This allows the load balancing system to monitor the performance bottlenecks of heterogeneous cores in real time, providing a basis for decision-making in the deep Q-network model. This enables precise matching of thread and core capabilities, avoids resource mismatch, fully unleashes the parallel computing potential of the RISC-V multi-core heterogeneous platform, and improves overall system performance and resource utilization.

[0057] In S2, the state vector is input into a pre-trained deep Q-network model to generate the optimal action instruction. This instruction is then executed through the operating system interface to achieve load balancing of thread resources and core capabilities.

[0058] The goal of pre-training the Deep Q-Network model is to enable the DQN agent to fully learn the characteristics of the RISC-V multi-core heterogeneous platform and obtain a high-performance decision-making model without affecting the actual, critical HPL testing task. For example... Figure 3 As shown, specifically:

[0059] (a) Offline training

[0060] 1. Data acquisition and environmental exploration;

[0061] The goal is to collect a large amount of diverse "state-action-reward" experience data.

[0062] (1) State;

[0063] The HPL program was run multiple times on the RISC-V platform.

[0064] To enable the model to deeply understand hardware characteristics, a comprehensive empirical dataset must first be constructed.

[0065] Launch an automated control script that will execute thousands of short HPL benchmark tests in an unattended manner.

[0066] To ensure the broad spectrum of the collected data, the script introduces random perturbations into the computing environment before each HPL run. These include:

[0067] Application layer perturbation: Randomly select the computation block size (NB) of HPL from a preset set of valid options.

[0068] System-level perturbation: Randomly sets the initial frequency mode of the CPU core, such as switching between performance and powersave modes, and randomizes the initial binding relationship between HPL threads and CPU cores. Among them, "performance" is the performance mode, which makes the CPU run at the highest frequency to prioritize computing performance; "powersave" is the power saving mode, which makes the CPU run at a lower frequency to prioritize saving power consumption. These are common CPU frequency adjustment modes.

[0069] Thread-level perturbation: Generate a random CPU affinity mask independently for each HPL compute thread, binding it to a random single core, core group, or all cores to simulate diverse thread space layouts.

[0070] Specifically:

[0071] a. Application layer perturbation: randomized computation granularity;

[0072] Target structure: HPL program input configuration file HPL.dat.

[0073] Processing method: Predefine a candidate list of computation block sizes (NB) containing multiple reasonable values, such as [64,128,256].

[0074] Before each round of data collection begins, a NB value is randomly selected from this list.

[0075] The script automatically modifies the corresponding NB parameter line in the HPL.dat file.

[0076] This systematically changes the computation and memory access behavior of HPL applications, simulating various load characteristics from "memory intensive" to "computation intensive", enabling the model to learn the system response under different loads.

[0077] b. System-level disturbances: Randomizing hardware resource states;

[0078] This perturbation occurs in two independent dimensions:

[0079] Target structure: The scaling_governor file exposed by the Linux kernel through the / sysfs file system, which is used to control the frequency of each CPU core.

[0080] Processing method: Define a policy list containing "performance" (maximum performance) and "powersave" (lowest power consumption).

[0081] For each CPU core, a strategy is randomly selected from this list independently.

[0082] Write the selected strategy string into the scaling_governor file of the corresponding core.

[0083] This creates an asymmetric, heterogeneous initial state of computing power supply, enabling the model to learn how to optimize task allocation under various unbalanced computing power hardware conditions.

[0084] c. Thread-level perturbation: Randomize thread-core binding relationships;

[0085] Target structure: The CPU affinity mask maintained by the Linux kernel for each thread.

[0086] Processing method: Obtain the thread ID (TID) of all computation threads in the HPL program.

[0087] Generate a random CPU affinity mask independently for each thread. These masks can bind it to a random single core, a random group of cores, or allow it to run on all cores.

[0088] The generated mask is applied to the corresponding thread via the sched_setaffinity system call or the taskset tool.

[0089] This allows for the simulation of various thread space layouts, including initial scenarios such as load balancing, extreme imbalance, and mismatch between computation and core capabilities, enabling the model to deeply understand the direct impact of thread layout on the performance of the system microarchitecture.

[0090] In this embodiment, the random perturbation of the environment during pre-training can simulate the complex operating scenarios of the RISC-V multi-core heterogeneous platform from multiple dimensions, including the application layer, system layer, and thread layer. This comprehensive random perturbation can construct a broad and diverse empirical dataset, enabling the deep Q-network model to fully learn the characteristics of the RISC-V platform, ultimately achieving accurate and adaptive load balancing on the platform and significantly improving parallel computing efficiency and hardware resource utilization.

[0091] (2) Action;

[0092] During program execution, DQN decision-making is not employed; instead, random strategies or simple heuristics are used to execute actions (such as random thread migration and regular adjustment of problem size). An action A is randomly selected from a predefined set of actions covering multiple optimization dimensions. This set of actions is the same as that used in the online optimization phase, including:

[0093] MIGRATE_THREAD( ): Migrate thread i to core j.

[0094] SWAP_THREADS( ): Swap the core binding between thread i and thread j.

[0095] SET_FREQ( ): Set the frequency of core k to level l.

[0096] Specifically:

[0097] a.MIGRATE_THREAD( ): Thread migration;

[0098] Action meaning: This is a one-way resource allocation operation that forces the specified thread i to be scheduled to run on the target core j.

[0099] When to use this method: When the system detects that high-quality resources are idle. For example, a computationally intensive thread is working on a low-performance core, while a high-performance core is idle or under low load. This operation aims to directly match high-load tasks to the best available resource to quickly resolve obvious performance bottlenecks.

[0100] b.SWAP_THREADS( ): Thread swapping;

[0101] Meaning of the action: This is a two-way resource reallocation operation that allows thread i and thread j to swap the CPU cores they are currently on.

[0102] When to use this method: When the system detects that both types of resources are misallocated. The most typical scenario is that a computationally intensive thread occupies a weak core, while a support thread occupies a strong core. This operation corrects the two unreasonable allocations simultaneously through a single "swap," achieving a one-step optimization where "each resource is in its proper place."

[0103] c.SET_FREQ( ): Core frequency modulation;

[0104] Action meaning: This is an energy efficiency adjustment operation for the core hardware status, used to dynamically set the operating frequency of core k.

[0105] Timing for selection: When the system identifies that the current performance bottleneck is no longer the CPU's computing power itself. For example, microarchitectural data may reveal that a core is stalling due to waiting for a large amount of memory data. In this case, the system will proactively reduce the core's frequency to save power; conversely, when the data bottleneck is resolved, the frequency will be immediately increased to restore its full computing power.

[0106] In this embodiment, predefined instructions are generated through a random strategy, enabling the construction of rich and diverse experience samples for the DQN model during the offline training phase. Thread migration can quickly match high-load tasks with high-quality resources, thread swapping can simultaneously correct resource mismatches, and core frequency adjustment can dynamically adapt to performance bottleneck scenarios. This random exploration of the full-dimensional action space allows the model to fully learn the impact of different actions on the load balancing of the RISC-V multi-core heterogeneous platform, ultimately achieving accurate decision-making in the online phase, effectively improving the platform's parallel computing efficiency and resource utilization, and unleashing the hardware's potential.

[0107] (3) Reward;

[0108] During program execution, the environment monitoring module remains active, recording the state (microarchitecture data vector) before and after each action, as well as the reward (GFLOPS change) resulting from that action. After action A is executed, a short time window is waited, for example, a monitoring period of 5 seconds, before the environment monitoring module is invoked to obtain and construct the post-action state vector S'. Simultaneously, the performance changes within this time window are analyzed, and the immediate reward R for action A is calculated.

[0109] Calculate the reward R using the following formula:

[0110] ;

[0111] in, This indicates the final performance output of the previous round of HPL testing before the action was executed (unit: Gflops). This indicates the final performance output of the new round of HPL testing after the action is performed (unit: Gflops). This represents the action cost constant, with a value of 0.1 (unit: Gflops), used to suppress invalid operations; This is the highest GFLOPS value recorded in all historical tests to date. The brackets for Iverson indicate that the value is 1 if the condition is met, and 0 otherwise.

[0112] The actual effect of this reward function is as follows: when the performance improvement exceeds 0.1 GFLOPS from the previous step, the agent can obtain a positive reward; if the performance breaks the historical record, it can also obtain an additional significant positive reward of 10.0; and throughout the training process, the agent aims to maximize the cumulative reward and will continue to tend to refresh the historical best performance, thereby achieving a strict monotonically increasing GFLOPS value.

[0113] Finally, all the generated experience tuples (State, Action, Reward, Next_State) are stored in a large-scale offline experience dataset; Next_State is the next state.

[0114] 2. Offline pre-training of the model;

[0115] The goal is to use the collected data to thoroughly "pre-learn" the DQN neural network, enabling it to master a high-quality initial strategy.

[0116] Start the training program for the DQN model when computing resources are idle.

[0117] The program repeatedly draws mini-batch data randomly from the offline experience dataset.

[0118] Using this data, the network weights of the DQN model are trained through backpropagation until the loss function converges.

[0119] After training is complete, save the DQN model with optimized weights as a file for use in the next stage.

[0120] After multiple rounds of training, a well-trained deep Q-network model is obtained.

[0121] Then, in formal HPC tasks, the pre-trained model is applied to achieve closed-loop, intelligent performance and energy efficiency optimization.

[0122] (ii) Online optimization;

[0123] 1. System initialization and model loading;

[0124] The goal is to prepare the HPL test environment.

[0125] The optimization system of this embodiment is started when the HPL program is about to run.

[0126] The DQN decision engine loads the model weights pre-trained in the first stage from the file.

[0127] The environment monitoring module and action execution module complete initialization and establish a connection with the underlying interface of the operating system.

[0128] The HPL program starts running with a default configuration.

[0129] 2. Real-time closed-loop optimization of the loop;

[0130] The goal is to continuously and autonomously make dynamic adjustments to the system during HPL operation.

[0131] The system enters a high-frequency, never-ending loop until the HPL ends. Each loop strictly executes the following four steps (perception-decision-execution-learning), such as... Figure 4 As shown:

[0132] (1) Perception:

[0133] The environmental monitoring module collects current PMU hardware data from all core components.

[0134] The state vector construction module processes the raw data into a standardized current state vector. .

[0135] The real-time state vector S is input into the loaded DQN decision model. The model quickly evaluates the value of all available optimization actions and outputs the optimal action A that has the highest expected long-term benefit in the current scenario. The decision-making process is context-aware, for example:

[0136] If the model detects a mismatch between computation and core capabilities (e.g., computationally intensive threads running on low-performance cores), it will prioritize space scheduling actions such as thread migration or swapping.

[0137] If a memory access bottleneck is detected (for example, a core pauses extensively while waiting for memory), the model will prioritize power-saving actions such as reducing the frequency of that core to avoid unnecessary energy consumption.

[0138] If the model detects a mismatch between the task granularity and the core cache (for example, a spike in the missing rate of a certain core cache), it may decide to take collaborative actions at the algorithmic level to adjust the computational granularity of that task.

[0139] Among them, "perception" refers to the DQN decision engine's processing of the input state vector. The process of automating feature extraction and pattern recognition. This capability stems from the model's deep learning on a large amount of microarchitectural data during the first stage (offline training). The specific process is as follows:

[0140] a. Data Input: During the online optimization loop, the environmental monitoring module collects raw PMU data, which is then processed by the state vector construction module into a standardized high-dimensional state vector. This vector is a precise mathematical snapshot of the current microarchitectural state of all core components of the system (e.g., IPC, cache miss rate, memory pause percentage, etc.).

[0141] b. Pattern recognition: The data is directly fed into the neural network of the DQN decision engine. The network weights (pre-trained offline) automatically identify specific "patterns" present in the input vector. For example:

[0142] Pattern 1: "Computing power mismatch": The vector simultaneously exhibits the combined characteristics of "low-performance cores' IPC reaching their saturation limit" and "high-performance cores' IPC being far below their potential".

[0143] Pattern 2: "Memory access-dominated bottleneck": The vector shows a strong correlation between "extremely low IPC of a certain core" and "extremely high proportion of memory pause cycles of that core".

[0144] Pattern 3: "Cache capacity mismatch": The vector shows the characteristic of "an abnormal surge in the L2 cache miss rate of a certain core".

[0145] (2) Decision:

[0146] The DQN decision engine receives the state vector. .

[0147] The trained neural network performs a forward propagation to calculate the Q-value for all possible actions.

[0148] The system selects the optimal action with the highest Q-value based on the ε-greedy strategy (in the online phase, ε is usually set to a very small value, prioritizing the known optimal action with the highest current Q-value to ensure effectiveness, i.e., "utilizing" existing experience and only conducting random exploration with a very low probability). .

[0149] The DQN decision engine determines the optimal response strategy through a single forward propagation calculation based on the identified state patterns. Specific process:

[0150] Value assessment: Input state vector The computation is performed within a DQN neural network. The network's output layer generates a predicted Q-value for each predefined optimization action (such as MIGRATE_THREAD, SET_FREQ, etc.). This Q-value is the model's quantitative assessment of the total reward that can be obtained by performing this action in the current state.

[0151] Optimal decision: The system follows a greedy strategy, selecting the action with the highest Q-value among all actions as the optimal action for the current period. .

[0152] For example, when a “computing power mismatch” pattern is detected: the model predicts, based on its learning experience, that space scheduling actions such as MIGRATE_THREAD or SWAP_THREADS will bring the greatest performance improvement (i.e. the highest Q value), and therefore will prioritize such actions.

[0153] When the "memory access-dominated bottleneck" mode is detected: the model predicts that forcibly migrating threads will not solve the problem (low Q value), while executing the "SET_FREQ frequency reduction and energy saving" action can obtain energy efficiency rewards without sacrificing performance (higher Q value), so the energy saving action will be prioritized.

[0154] When the "cache capacity mismatch" pattern is detected: the model predicts a collaborative action to adjust the computational granularity of the task, which can fundamentally solve the problem compared to other actions, and therefore assigns it the highest Q value.

[0155] (3) Execution (Act):

[0156] The action execution module receives the optimal action. The instructions.

[0157] The module calls the corresponding operating system APIs (such as sched_setaffinity, cpufreq interface, etc.) to actually apply the optimization to the system.

[0158] (4) Learning:

[0159] System calculates and executes actions The immediate rewards that follow (For example, the increase in instantaneous GFLOPS).

[0160] The new state that the sensing system enters .

[0161] This entirely new experiential tuple ( Store it in the experience replay buffer.

[0162] Periodically (e.g., every 4 loops), a batch of experience is drawn from the buffer to fine-tune the DQN model online, enabling it to adapt to the specific patterns of the current task and continuously optimize itself.

[0163] 3. Task completion and model saving;

[0164] The goal is to complete the optimization task, output and save the optimization results.

[0165] After the HPL program finishes running, the optimization system's loop terminates, and the relevant experimental data (such as parameters in HPL.dat, DQN model weights, etc.) are saved.

[0166] This specific embodiment constructs a state vector by collecting microarchitecture performance data and generates optimal action instructions (thread migration, swapping, core frequency adjustment) using a pre-trained deep Q-network model, achieving dynamic intelligent load balancing. During the pre-training phase, a broad-spectrum empirical dataset is constructed through random perturbations at the application, system, and thread layers, enabling the model to fully learn hardware characteristics. This approach overcomes the limitations of traditional methods in adapting to heterogeneous architectures, accurately capturing microarchitecture bottlenecks and optimizing them in real time. It provides intelligent scheduling support for the RISC-V platform to fully unleash the potential of multi-core heterogeneous architectures in high-performance computing and other scenarios, combining architectural adaptability with performance optimization, effectively addressing the shortcomings of existing technologies in terms of dynamism, versatility, and accuracy.

[0167] Example 2

[0168] This embodiment provides an intelligent load balancing system for a RISC-V multi-core heterogeneous platform, including:

[0169] The state vector acquisition module is configured to acquire microarchitecture performance data of each processing core in the RISC-V multi-core heterogeneous platform and construct it into a state vector; the microarchitecture performance data includes instruction cycle count, cache miss rate at each level, and memory pause percentage.

[0170] The load balancing module is configured to input the state vector into a pre-trained deep Q-network model to generate optimal action instructions, thereby achieving load balancing between thread resources and core capabilities; the optimal action instructions include thread migration, thread swapping, and core frequency adjustment.

[0171] The pre-training of the deep Q-network model involves running a parallel computing program on a RISC-V platform. Before the program runs, the environment is randomly perturbed, actions are executed through random policies, and state, action, and reward data are collected to construct an offline experience dataset for pre-training.

[0172] Example 3

[0173] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in Embodiment 1 above.

[0174] Example 4

[0175] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in Embodiment 1 above.

[0176] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0177] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent load balancing on a RISC-V multi-core heterogeneous platform, characterized in that, include: The microarchitecture performance data of each processing core in the RISC-V multi-core heterogeneous platform is obtained and constructed into a state vector; the microarchitecture performance data includes instruction cycle count, cache miss rate at each level, and memory pause percentage. The state vector is input into a pre-trained deep Q-network model to generate optimal action instructions, thereby achieving load balancing between thread resources and core capabilities. The optimal action instructions include thread migration, thread swapping, and core frequency adjustment. Based on the deep Q-network model, it also includes a closed-loop processing process, specifically: during the online load balancing process, perception, decision-making, execution and learning are executed cyclically. The perception process involves collecting system microarchitecture performance data and constructing it into a current state vector, which is then used by the deep Q-network model to extract features and recognize patterns. The decision-making process involves the deep Q-network model calculating Q-values ​​for predefined actions and selecting the optimal action based on the identified patterns. Specifically, if a mismatch between computation and core capabilities is detected, thread migration or thread swapping actions are prioritized. If a memory access bottleneck is detected, actions to reduce the corresponding core frequency are prioritized. If a mismatch between task granularity and core cache is detected, actions to adjust the task computation granularity are prioritized. The execution involves performing the selected optimal action to adjust system resources. The learning process involves storing the new states and rewards generated after the actions are performed into an experience replay buffer, and periodically sampling to fine-tune the deep Q-network model online, thereby achieving continuous adaptive optimization of the model strategy. The pre-training of the deep Q-network model involves running a parallel computing program on a RISC-V platform. Before the program runs, the environment is randomly perturbed, actions are executed using a random policy, and state, action, and reward data are collected to construct an offline experience dataset for pre-training. The random perturbation includes: At the application layer, the computation block size of the parallel computing program is randomly selected from a preset set of valid parameters; At the system level, an initial operating frequency strategy is randomly set for each processing core from a preset set of frequency strategies; At the thread level, a random CPU affinity mask is generated independently for each computing thread to randomize the initial binding relationship between the program thread and the processing core, simulating diverse thread space layouts.

2. The intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in claim 1, characterized in that, The instruction cycle count is the ratio of the number of instructions to the number of CPU clock cycles required to execute that instruction. The cache miss rate for each level is the ratio of the number of cache access misses to the total number of accesses for the corresponding level. The memory pause percentage is the proportion of pipeline pause cycles caused by memory access to the total number of CPU clock cycles.

3. The intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in claim 1, characterized in that, The state vector is a one-dimensional floating-point array whose elements are composed of the instruction cycle count, cache miss rate, and memory pause percentage of all processing cores on the platform, concatenated in core order.

4. The intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in claim 1, characterized in that, The thread migration refers to the operation of forcibly scheduling a specified thread to run on a target core. The thread swapping refers to the swapping operation where two threads exchange the cores they are currently bound to. The core frequency adjustment refers to the operation of dynamically setting a specified core operating voltage and frequency.

5. The intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in claim 1, characterized in that, The pre-training process of the deep Q-network model is as follows: randomly sample batches of data from the offline experience dataset, and iteratively optimize the network parameters through the backpropagation algorithm until the loss function converges.

6. A RISC-V multi-core heterogeneous platform intelligent load balancing system, characterized in that, include: The state vector acquisition module is configured to acquire microarchitecture performance data of each processing core in the RISC-V multi-core heterogeneous platform and construct it into a state vector; the microarchitecture performance data includes instruction cycle count, cache miss rate at each level, and memory pause percentage. The load balancing module is configured to input the state vector into a pre-trained deep Q-network model to generate optimal action instructions, thereby achieving load balancing between thread resources and core capabilities; the optimal action instructions include thread migration, thread swapping, and core frequency adjustment. Based on the deep Q-network model, it also includes a closed-loop processing process, specifically: during the online load balancing process, perception, decision-making, execution and learning are executed cyclically. The perception process involves collecting system microarchitecture performance data and constructing it into a current state vector, which is then used by the deep Q-network model to extract features and recognize patterns. The decision-making process involves the deep Q-network model calculating Q-values ​​for predefined actions and selecting the optimal action based on the identified patterns. Specifically, if a mismatch between computation and core capabilities is detected, thread migration or thread swapping actions are prioritized. If a memory access bottleneck is detected, actions to reduce the corresponding core frequency are prioritized. If a mismatch between task granularity and core cache is detected, actions to adjust the task computation granularity are prioritized. The execution involves performing the selected optimal action to adjust system resources. The learning process involves storing the new states and rewards generated after the actions are performed into an experience replay buffer, and periodically sampling to fine-tune the deep Q-network model online, thereby achieving continuous adaptive optimization of the model strategy. The pre-training of the deep Q-network model involves running a parallel computing program on a RISC-V platform. Before the program runs, the environment is randomly perturbed, actions are executed using a random policy, and state, action, and reward data are collected to construct an offline experience dataset for pre-training. The random perturbation includes: At the application layer, the computation block size of the parallel computing program is randomly selected from a preset set of valid parameters; At the system level, an initial operating frequency strategy is randomly set for each processing core from a preset set of frequency strategies; At the thread level, a random CPU affinity mask is generated independently for each computing thread to randomize the initial binding relationship between the program thread and the processing core, simulating diverse thread space layouts.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in any one of claims 1-5.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the intelligent load balancing method for a RISC-V multi-core heterogeneous platform as described in any one of claims 1-5.

Images