A load-aware optimization method and optimization device

Abstract

This application provides a load-aware optimization method and apparatus. The method includes: sensing the expert computation load characteristics and inter-device communication load characteristics of the current training step, and combining them into a load state feature vector; inputting the load state feature vector into an online evaluation model, where the online evaluation model selects a target action from three candidate actions: maintaining the current configuration, performing sample reordering, and performing expert replication; and performing corresponding operations based on the selected target action: if the target action is to perform sample reordering, then sample reordering is performed to reduce cross-device communication; if the target action is to perform expert replication, then expert replication is created to distribute the computational load. This application simultaneously senses both computational and communication loads, adaptively and collaboratively scheduling reordering and replication, thereby improving resource utilization.

Description

Technical Field

[0001] This application relates to the field of large model training technology, and more specifically, to a load-aware optimization method and optimization device. Background Technology

[0002] Currently, hybrid expert models, through gating mechanisms, activate only a subset of experts in each forward propagation, significantly increasing the number of model parameters and expressive power while controlling computational overhead. This has become a core technical solution for realizing large-scale language models. However, hybrid expert models face severe challenges in distributed training: their sparse routing characteristics require full inter-device communication at each iteration to redistribute tokens from the current device to the corresponding expert's device, generating massive amounts of cross-device data transmission; simultaneously, gating decisions inherently suffer from uneven load distribution, with some popular experts carrying too many tokens, causing the computation time on these experts' devices to be significantly longer than on other devices, forming a computational long tail; while less popular experts remain idle, resulting in a large amount of idle computing resources.

[0003] To address the aforementioned issues, existing methods typically focus on a single dimension. One type of method, exemplified by sample rearrangement, adjusts the mapping between samples and devices, allowing more terms to remain locally for expert computation, thereby reducing cross-device communication. However, this method does not alter the distribution of expert computational load. When some experts become computational hotspots due to receiving too many terms, even with communication optimization, the devices hosting these experts remain computationally overloaded, and training speed is still slowed down by single-point computational bottlenecks. Another type of method, represented by expert duplication, creates multiple duplications for popular experts, distributing their computational load across different devices. However, continuous parameter synchronization between duplications introduces additional bandwidth overhead during periods of communication congestion, further exacerbating the network burden. Overall, existing methods cannot discern whether the current bottleneck is computational overload or communication congestion; they can only mechanically execute a single strategy, resulting in the system failing to fully utilize both communication and computational resources when both bottlenecks alternate. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a load-aware optimization method and optimization apparatus to overcome at least one of the above-mentioned defects.

[0005] In a first aspect, embodiments of this application provide a load-aware optimization method, the method comprising: sensing the expert computation load characteristics and inter-device communication load characteristics of the current training step, and combining the two into a load state feature vector; inputting the load state feature vector into an online evaluation model, wherein the online evaluation model selects a target action from three candidate actions: maintaining the current configuration, performing sample reordering, and performing expert replication; and performing corresponding operations based on the selected target action: if the target action is to perform sample reordering, then performing sample reordering to reduce cross-device communication volume; if the target action is to perform expert replication, then performing expert replication creation to distribute the computation load.

[0006] In one optional embodiment of this application, the expert computational load feature is obtained by: reading the number of tokens received by each expert in the previous training step to form an expert load vector; calculating the Gini coefficient of the expert load vector as a measure of overall skewness; calculating the ratio of the number of tokens received by the hottest expert to the total number of tokens received by all experts as a measure of the concentration of a single expert; and combining the Gini coefficient and the ratio of the number of tokens received by the hottest expert to form the expert computational load feature.

[0007] In an optional embodiment of this application, the inter-device communication load characteristics are obtained by: reading the lexical transmission records between physical devices and between each physical device and each logical computing unit in the previous training step to form a communication traffic matrix; calculating the ratio of the number of lexical units transmitted across devices to the total number of transmitted lexical units, as the cross-device communication proportion characteristic; calculating the sending load variation coefficient and receiving load variation coefficient of each device, as a measure of the unevenness of communication load distribution; and combining the cross-device communication proportion characteristic, the sending load variation coefficient, and the receiving load variation coefficient to form the inter-device communication load characteristics.

[0008] In one optional embodiment of this application, the online evaluation model is a linear evaluation model based on a confidence upper limit. This model maintains a linear parameter vector and a covariance matrix for each candidate action. The target action is selected by: calculating the inner product of the linear parameter vector and the load state feature vector as the expected return term; calculating the product of the exploration coefficient and the result of the operation based on the load state feature vector and the inverse matrix of the covariance matrix as the uncertainty term; adding the expected return term and the uncertainty term to obtain the comprehensive score of the candidate action; and selecting the candidate action with the highest comprehensive score as the target action.

[0009] In an optional embodiment of this application, the method further includes: recording the single-step time of each iteration within a fixed-step observation window after the target action is executed, and calculating the average single-step time within the observation window; comparing the average single-step time with the historical average single-step time before the target action is executed, and using the rate of change of time as a reward signal; and using the reward signal and the load state feature vector on which the trigger decision is based to incrementally update the linear parameter vector and covariance matrix corresponding to the target action.

[0010] In one alternative embodiment of this application, the sample rearrangement is performed by: extracting the communication traffic matrix from the previous training step; formalizing the problem of minimizing communication overhead into a combinatorial optimization problem that maximizes the proportion of local computation; and reconstructing the mapping relationship between physical devices and logical numbers using a greedy mapping algorithm to complete the sample rearrangement.

[0011] In one optional embodiment of this application, the greedy mapping algorithm includes: flattening the communication traffic matrix and sorting it in descending order of communication volume; traversing the communication volume in descending order, and mapping the target logical position with the largest communication volume directly to the physical device where the source data is located, provided that neither the source device nor the target logical position is occupied; and performing a safety net mapping logic on unallocated edge nodes to complete the reconstruction of the mapping relationship between physical devices and logical numbers.

[0012] In one optional embodiment of this application, expert replica creation is performed as follows: based on the communication traffic distribution after sample rearrangement, experts whose word reception exceeds a preset threshold are identified as popular experts; a global placement optimization model is constructed with the goal of maximizing the difference between communication savings and parameter synchronization costs, and the number and deployment location of replicas of the popular experts are determined; according to the determined number and deployment location, expert replicas are created at the determined deployment locations, thus completing the expert replica creation.

[0013] In one optional embodiment of this application, a decision reuse step is further included: in the early stage of training, load similarity analysis is performed on each layer of the hybrid expert model, the cosine similarity of expert load between adjacent layers is calculated, and the model depth is divided into shallow region, intermediate homogeneous region and deep region according to the similarity change law; in subsequent training, independent decision-making is maintained only for the first layer of the shallow region and the last layer of the deep region, and for each layer in the intermediate homogeneous region, a decision is triggered only once in the first layer of the region and its result is reused to all layers in the region, thus completing the decision reuse.

[0014] Secondly, embodiments of this application also provide a load-aware optimization device, comprising: a load state feature vector combination module, used to sense the expert computation load characteristics and inter-device communication load characteristics of the current training step, and combine the two into a load state feature vector; a target action selection module, used to input the load state feature vector into an online evaluation model, and the online evaluation model selects a target action from three candidate actions: maintaining the current configuration, performing sample rearrangement, and performing expert replication; and an optimization execution module, used to perform corresponding operations according to the selected target action: if the target action is to perform sample rearrangement, then perform sample rearrangement to reduce cross-device communication volume; if the target action is to perform expert replication, then perform expert replication creation to distribute the computation load.

[0015] The load-aware optimization method and apparatus provided in this application perceive the expert computation load characteristics and inter-device communication load characteristics of the current training step, and combine them into a load state feature vector. This load state feature vector is input into an online evaluation model, which selects a target action from three candidate actions: maintaining the current configuration, performing sample reordering, and performing expert replication. Based on the selected target action, the corresponding operation is performed: if the target action is sample reordering, sample reordering is performed to reduce cross-device communication; if the target action is expert replication, expert replication is created to distribute the computational load. This application simultaneously perceives both computational and communication loads, adaptively and collaboratively scheduling reordering and replication, thereby improving resource utilization and training throughput.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the load-aware optimization method provided in this application embodiment; Figure 2 This is a flowchart illustrating the process of obtaining expert-calculated load characteristics provided in an embodiment of this application; Figure 3 A flowchart illustrating the characteristics of inter-device communication load provided in an embodiment of this application; Figure 4A flowchart of the target selection action provided in the embodiments of this application; Figure 5 A flowchart of the execution sample rearrangement provided in the embodiments of this application; Figure 6 This is a schematic diagram of the internal structure of the Hybrid Expert Model (MoE) module provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of the load-aware optimization device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0020] First, the applicable scenarios for this application will be introduced. This application can be applied to the field of large model training technology.

[0021] Research has revealed that existing optimization methods for training hybrid expert models generally suffer from the following drawbacks: for example, FasterMoE creates expert replicas but introduces additional synchronization overhead during the communication-dominated phase, while NetMoE performs sample rearrangement but cannot alleviate the computational long tail caused by extremely popular experts; moreover, the load distribution changes dynamically in time and space, with frequent switching between computational overload and communication congestion.

[0022] Existing methods lack the ability to perceive the current bottleneck type and cannot distinguish whether the main cause of system performance limitation is computational load skew or communication congestion. They can only mechanically execute a single, pre-set strategy. Methods centered on sample rearrangement adjust the communication topology without changing the computational distribution. However, they fail when faced with long tails of computation caused by popular experts because the computational bottleneck cannot be alleviated. Methods centered on expert replicas introduce additional parameter synchronization overhead when communication is already congested, which actually increases the network burden. Due to the lack of an adaptive decision-making mechanism, the two optimization methods operate independently and cannot effectively cooperate when the bottleneck type dynamically changes, resulting in persistently low utilization of communication and computing resources.

[0023] Based on this, the embodiments of this application provide a load-aware optimization method and device. By simultaneously sensing the dual-dimensional characteristics of expert computing load and inter-device communication load, the bottleneck type of the current training step is accurately identified. The target action is adaptively selected from three candidate actions—maintaining the current configuration, performing sample rearrangement, and performing expert copy—using an online evaluation model. This achieves synergistic linkage between communication optimization and computing optimization, thereby solving the technical problem of low utilization of communication and computing resources caused by the single optimization dimension and lack of adaptive collaborative intervention mechanism in existing methods.

[0024] Please see Figure 1 , Figure 1 A flowchart illustrating the load-aware optimization method provided in this application embodiment. Figure 1 As shown in the embodiments of this application, the load-aware optimization method includes: S101. Perceive the expert computation load characteristics and inter-device communication load characteristics of the current training step, and combine the two into a load state feature vector.

[0025] The system directly reads the historical records automatically saved after the previous training step, extracting two types of raw data. The first type is a statistical summary of the number of terms received by each expert, organized as an expert load vector. Each element of the vector corresponds to the number of terms an expert participated in the computation of during that training step. The second type is a record of term transmission between physical devices and between physical devices and logical computing units, organized as a communication traffic matrix. Rows in the matrix correspond to sending devices, columns correspond to receiving devices, and the matrix element values ​​are the number of terms sent from the sender to the receiver. Both types of data are statistical information naturally generated during the distributed training process, requiring no additional global synchronization.

[0026] First, define the system size parameter. Let the number of samples in a single iteration be... The total number of experts is The total number of computing devices is The sample is routed to via a gating function. One expert (satisfied) Define the routing matrix. If the sample Distributed to experts but Otherwise, it is 0. Let the expert... Mapped to physical devices ,sample The initial position is Based on this, device-level computing load Cross-device communication Represented as: (1) in, Indicates the first The total number of tokens received by all experts mapped to the device on the platform, where N is the number of samples (total number of tokens) in a single iteration. It is a non-negative integer.

[0027] (2) Where C represents the initial position of the sample device in a single iteration. With the target expert's equipment The total number of cross-device token transmissions resulting from different factors. This metric directly reflects the global communication pressure, and is related to the device-wide computing load represented by equation (1). Together, these constitute the two core basic indicators for characterizing the bottlenecks in system operation. E represents the total number of experts, P represents the total number of computing devices, and K represents the number of experts routed to for each sample (Top-K). For each element in the routing matrix, it indicates whether sample i is distributed to an expert. (1 indicates yes, 0 indicates no) ( (for experts) The physical device number where it is located. ( ) as a sample The physical device number where the initial location is located. For the first The computing load on the device (the total number of tokens received by all experts on that device).

[0028] This method monitors the state through two dimensions: computation and communication. In the first... In this iteration, the statistical expert load vector and communication traffic matrix .in, For the first The number of tokens calculated by the experts Send to device i The number of tokens. Considering the strong temporal locality of load distribution between adjacent training iterations, the system adopts the th Step This serves as a representation of current decision-making, in order to reduce monitoring overhead.

[0029] After obtaining the above raw data, the expert computation load characteristics and inter-device communication load characteristics are calculated separately. The expert computation load characteristics consist of two indicators: the Gini coefficient and the percentage of the most active expert load. The Gini coefficient is obtained by averaging the absolute values ​​of the differences between each pair of elements in the expert load vector and dividing by the product of twice the total number of experts and the average load. The percentage of the most active expert load is obtained by taking the maximum value in the expert load vector and dividing it by the sum of all elements. The inter-device communication load characteristics consist of three indicators: the percentage of cross-device communication, the coefficient of variation of transmitting load, and the coefficient of variation of receiving load. The percentage of cross-device communication is obtained by dividing the sum of the off-diagonal elements in the communication traffic matrix by the sum of all elements. The coefficient of variation of transmitting load is obtained by the ratio of the standard deviation of the sum of elements in each row to the mean. The coefficient of variation of receiving load is obtained by the ratio of the standard deviation of the sum of elements in each column to the mean.

[0030] Specifically, the mathematical expressions for the above indicators are as follows: (1) Expert calculation of load characteristics: Based on expert load vector Calculate the Gini coefficient To quantify the overall imbalance in the distribution of expert workloads: (3) in, The total number of experts; For the average load of all experts, satisfy ; and They represent the first The and the first The number of tokens for each expert ; The Gini coefficient has a range of values ​​of 100. , The closer to 1, the more concentrated the load; the closer to 0, the more balanced the load.

[0031] Simultaneously calculate the percentage of the hottest expert's load. : (4) Among them, molecules This represents the maximum number of tokens among all experts; the denominator is the sum of the token counts of all experts. The range of values ​​is The larger the value, the higher the concentration of a single expert.

[0032] (2) Characteristics of communication behavior: Based on communication traffic matrix First, calculate the proportion of cross-device communication. : (5) Where the numerator is the sum of the token transfers between all different devices, and the denominator is the total token transfers (including local); P is the total number of computing devices; Fij represents the number of tokens sent from device i to device j; The range of values ​​is The larger the value, the more the communication is mainly based on cross-device transmission.

[0033] Secondly, define the first The transmission load of each device With received load : (6) in, Indicates equipment The total number of tokens sent to all devices (including itself); Indicates equipment The total number of tokens received from all devices (including itself); both are non-negative integers. .

[0034] Further calculate the coefficient of variation of the transmit load and the receive load: (7) in, and These are the average values ​​of the transmitted and received loads for all devices, respectively. , ; and These are the standard deviations of the transmit load and the receive load, respectively. and This is a dimensionless value; the larger the value, the more unevenly the communication load is distributed among devices.

[0035] The aforementioned expert-calculated load characteristics and communication behavior characteristics are combined to form a low-dimensional feature vector. This feature vector serves as input for subsequent decision-making modules. It can accurately characterize the main bottleneck type of the current training step.

[0036] All the indicators calculated above are combined into a low-dimensional load state feature vector, with each indicator serving as a dimension of the vector, and directly passed to the subsequent online evaluation model.

[0037] Specifically, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the process of obtaining expert-calculated load characteristics, provided as an embodiment of this application. Figure 2 As shown, the expert-calculated load characteristics are obtained in the following way: S201. Read the number of tokens received by each expert in the previous training step to form the expert load vector.

[0038] During each training step, the gating network routes each term in the current batch to one or more expert subnetworks for computation. After each training step is completed, the number of terms each expert actually participated in computation is naturally stored as historical data. S201 directly reads the number of terms received by each expert from these historical records and organizes it into an expert load vector. The dimension of the vector equals the total number of experts, and the value of each element in the vector is the number of terms processed by the corresponding expert in that training step. The raw data of this vector comes entirely from the statistical information generated during the training process itself, requiring no intrusive modifications to the training process.

[0039] S202. Calculate the Gini coefficient of the expert load vector as a measure of the overall skewness.

[0040] After obtaining the expert load vector, S202 calculates the Gini coefficient of this vector to quantify the overall imbalance in load distribution among experts. The Gini coefficient is calculated by averaging the absolute values ​​of the differences between each pair of expert loads, then dividing by twice the product of the total number of experts and the average expert load. The Gini coefficient ranges from zero to one; a value closer to one indicates a higher concentration of load among a few experts, while a value closer to zero indicates a more even distribution of load among experts. This indicator characterizes the skewness of computational load from a global perspective, effectively distinguishing between "slight skewness" and "severe skewness," providing a basis for subsequent judgments on whether computational optimization needs to be triggered.

[0041] S203. Calculate the ratio of the number of tokens received by the hottest expert to the total number of tokens received by all experts, as a measure of the concentration of a single expert.

[0042] Unlike the Gini coefficient, which focuses on the overall distribution, S203 focuses on the individual expert with the heaviest workload. Specifically, it extracts the maximum value from the expert load vector—the number of tokens received by the most active expert—and divides this value by the sum of the number of tokens received by all experts to obtain the proportion of that most active expert in the total computational load. This ratio ranges from one (divided by the total number of experts) to one; a higher value indicates a higher concentration of computation by a single expert, potentially leading to a situation where one expert bears the vast majority of the computational load in extreme cases. This metric directly reflects the risk of a long tail of computation caused by a single expert.

[0043] S204. Combine the Gini coefficient and the percentage of tokens received by the hottest expert to form the expert computation load feature.

[0044] S204 combines the two metrics calculated by S202 and S203 to form the expert computation load feature. This feature includes information on both the global skewness and the concentration of individual hotspots, which complement each other and can comprehensively characterize the computational load state of the current training step.

[0045] For further details, please refer to Figure 3 , Figure 3 This is a flowchart illustrating the characteristics of inter-device communication load provided in an embodiment of this application. Figure 3 As shown, the inter-device communication load characteristics are obtained in the following way: S301. Read the word transmission records between physical devices and between physical devices and logical computing units in the previous training step to form a communication traffic matrix.

[0046] In each iteration of the distributed hybrid expert model, after being routed through a gated network, tokens need to be transmitted between different physical devices to move from the device where the token is currently located to the device where the target logical location is located. S301 reads the token transmission records retained after the previous training step, counts the number of tokens transmitted between any two physical devices, and organizes this count into a communication traffic matrix. The rows of the matrix correspond to the sending devices, and the columns correspond to the receiving devices. The value of each element in the matrix represents the number of tokens sent from the sender to the receiver. This matrix completely records the scale and distribution of all cross-device communication behaviors in a single iteration.

[0047] S302. Calculate the ratio of the number of lexical units transmitted across devices to the total number of lexical units transmitted, and use this as a characteristic of the proportion of cross-device communication.

[0048] Based on the communication traffic matrix, S302 calculates the total number of tokens transmitted between all different devices (i.e., the off-diagonal elements of the matrix), divides it by the total number of tokens transmitted across all elements of the matrix (including local transmissions within devices), and obtains the cross-device communication ratio. This ratio ranges from zero to one; a higher value indicates that communication is primarily cross-device transmission, with a lower proportion of local direct transmission. A high ratio indicates that a large number of tokens need to be transmitted via network links, resulting in significant communication pressure.

[0049] S303. Calculate the coefficient of variation of the transmitting load and the coefficient of variation of the receiving load of each device as a measure of the unevenness of the communication load distribution.

[0050] The transmit and receive loads of each device are statistically analyzed from the communication traffic matrix. A device's transmit load is the total number of bytes it sends to all devices (including itself), which is the sum of the elements in that row of the matrix. A device's receive load is the total number of bytes it receives from all devices (including itself), which is the sum of the elements in that column of the matrix. After obtaining the transmit and receive load sequences for all devices, the coefficient of variation (COP) is calculated for both, which is the ratio of the standard deviation to the mean. A higher COP indicates a more uneven distribution of communication load among devices, suggesting that some devices are bearing an excessively heavy transmit and receive load and are prone to becoming communication bottlenecks.

[0051] S304. Combine the cross-device communication proportion characteristics, the transmit load variation coefficient, and the receive load variation coefficient into the inter-device communication load characteristics.

[0052] The three metrics obtained—cross-device communication ratio, transmit load coefficient of variation, and receive load coefficient of variation—are combined to form the inter-device communication load characteristics. These three metrics describe communication behavior from the perspectives of communication localization, transmit-side load balancing, and receive-side load balancing, respectively. Their combination provides a relatively comprehensive reflection of the current training step's load status on the communication side.

[0053] S102. Input the load state feature vector into the online evaluation model, and the online evaluation model selects a target action from three candidate actions: maintain the current configuration, perform sample rearrangement, and perform expert copy.

[0054] After obtaining the load state feature vector, the vector is input into an online evaluation model. This online evaluation model maintains a set of parameters for each of the three candidate actions: maintaining the current configuration, performing sample rearrangement, and executing expert replicas. Based on these parameters, the model scores and compares the applicability of each candidate action under the current load state, and selects one target action as the scheduling instruction for this decision.

[0055] Here, the online evaluation model is a linear evaluation model based on the confidence upper limit, which maintains a linear parameter vector and a covariance matrix for each candidate action.

[0056] The system provides each candidate action Maintain a 3D linear parameter vector and Covariance matrix ,in For feature vectors The dimension. Action scores are calculated using a linear model based on upper confidence limits: (8) in, This indicates that the action $a$ is in the current context. The expected returns; This indicates the uncertainty of action selection and reflects the confidence level of the model in estimating the payoff of that action; To explore coefficients and control the trade-off between exploring new strategies and utilizing existing experience, the empirical value in this embodiment is... ; Covariance matrix The inverse matrix.

[0057] The system calculates all candidate actions at each decision observation point. The system selects the action with the highest score as the scheduling instruction for the current step and issues it for execution. In the early stages of training or when the load pattern changes abruptly, the uncertainty is large, and the system tends to explore different actions; as data accumulates, the expected reward factor dominates the score, and the system quickly converges to the optimal strategy.

[0058] For further details, please refer to Figure 4 , Figure 4 This is a flowchart illustrating the target selection action provided in an embodiment of this application. Figure 4 As shown, the target action is selected in the following way: S401. Calculate the inner product of the linear parameter vector and the load state characteristic vector, and use it as the expected revenue term.

[0059] The online evaluation model maintains an independent set of parameters for each of the three candidate actions: maintaining the current configuration, performing sample rearrangement, and performing expert replication. These parameters include a linear parameter vector and a covariance matrix. The linear parameter vector is a real-number vector with the same dimension as the load state feature vector. Its initial value can be randomly initialized or set to zero, and it is gradually adjusted through online updates as the training process progresses.

[0060] Once the load state feature vector output by S101 arrives, for each candidate action, the corresponding linear parameter vector is extracted and a dot product operation is performed with the load state feature vector. The dot product operation means multiplying the elements at corresponding positions of the two vectors and summing them to obtain a scalar value, which is the expected benefit term. The expected benefit term reflects the model's estimate, based on historical experience, of the potential performance improvement that the action might bring under the current load state; a larger value indicates that the model believes the action is more suitable for the current scenario.

[0061] S402. Calculate the product of the exploration coefficient and the inverse matrix based on the load state eigenvector and covariance matrix, and use it as the uncertainty term.

[0062] Along with the linear parameter vector, each candidate action also maintains a covariance matrix, a square matrix with the same dimensions as the feature vector. This matrix records the accumulated observations of the relationships between different feature dimensions in past decisions for that action. S402 first calculates the result of the operation between the load state feature vector and the inverse of the covariance matrix. The specific form of this operation is: first, multiply the transpose of the load state feature vector by the inverse of the covariance matrix, then multiply by the load state feature vector itself, and finally take the square root. The result characterizes the model's sufficient understanding of the current load state under that action—if the current load state differs significantly from the states that triggered the action historically, the result will be larger, meaning the model lacks confidence in the action's effectiveness under that state. Multiplying this result by a preset exploration coefficient forms the uncertainty term. The exploration coefficient is a positive real number, empirically set to a constant between one and two, used to control the weighting of exploring unknown strategies and utilizing existing experience.

[0063] S403. Add the expected return item and the uncertainty item to obtain the comprehensive score of the candidate action.

[0064] The expected return and uncertainty terms are summed to obtain the comprehensive score of the candidate action under the current load condition.

[0065] The overall score comprises two parts: the expected return component, which favors actions with good historical performance based on existing experience, and the uncertainty component, which favors actions with fewer attempts and unclear effects. The combination of these two components allows the model to converge quickly to a known efficient strategy when faced with recurring load patterns. However, when the load pattern changes abruptly or when historical data is insufficient in the early stages of training, the uncertainty component carries a larger weight, prompting the model to try different actions to accumulate more comprehensive experience.

[0066] S404. Select the candidate action with the highest comprehensive score as the target action.

[0067] The combined scores of all three candidate actions are compared, and the one with the highest score is directly selected as the target action and sent to the distributed execution engine. This selection method ensures that each decision is the optimal choice under the current model state. The entire evaluation and selection process involves only lightweight vector and matrix operations, and the decision latency is much lower than the time consumed by a single training iteration.

[0068] The sequential execution of S401 to S404 allows the online evaluation model to dynamically and adaptively choose between maintaining the current configuration, performing sample rearrangement, and executing expert replicas based on real-time load conditions. The expected return-driven model uses historical experience to quickly match known load patterns, while the uncertainty-driven model continues to explore potentially better actions when load conditions change abruptly or experience is insufficient. The combination of these two approaches, along with a comprehensive score, selects the optimal action. This ensures that the scheduling strategy can quickly respond to recurring typical bottlenecks and adapt to the non-stationary evolution of load distribution during training, avoiding decision lag caused by static strategies.

[0069] S103. Based on the selected target action, perform the corresponding operation: if the target action is to perform sample rearrangement, then perform sample rearrangement to reduce cross-device communication; if the target action is to perform expert replication, then perform expert replication creation to distribute the computational load.

[0070] After the online evaluation model selects the target action, S103 translates the decision instruction into an actual physical execution operation. If the target action is to perform sample rearrangement, the communication traffic matrix from the previous training step is extracted. With maximizing the local computation ratio after rearrangement as the optimization objective, a greedy mapping algorithm is used to reconstruct the mapping relationship between physical devices and logical numbers. This algorithm flattens the communication traffic matrix and arranges it in descending order of communication volume. It then sequentially maps the target logical location with the largest communication volume to the physical device where the source data resides, ensuring that neither the source device nor the target logical location is repeatedly occupied. For logical locations that are still unassigned after the traversal is complete, a fallback mapping is performed.

[0071] After rearrangement, a large number of tokens that originally needed to be transmitted across devices are now accessed from the device's local memory, fundamentally reducing cross-node communication traffic. If the target action is to create expert replicas, based on the fact that the sample rearrangement has already concentrated popular traffic on a few physical devices, experts whose token reception exceeds a preset threshold are identified as popular experts. A global placement optimization model is constructed, aiming to maximize the difference between communication savings and parameter synchronization costs, to determine the optimal number of replicas and deployment locations for popular experts, and replica creation is completed accordingly. This method covers global computing hotspots with a very small replica scale, mitigating the slow node effect while keeping the additional bandwidth overhead introduced by parameter synchronization between replicas at a low level.

[0072] The preset threshold is a dynamic threshold, and its specific value is determined based on the statistical distribution of the number of words received by each expert in the current training step. In one embodiment, the preset threshold is set to α times the average number of words received by all experts, where α ranges from 1.5 to 3.0, and a value of 2.0 is recommended. In another embodiment, the number of words received by each expert is arranged in descending order, and the top-ranked experts are selected. E / K The top-ranked experts are designated as popular experts, with a preset threshold being the ranking. E / K The number of tokens received by each expert, where E is the total number of experts and K is the number of experts routed to per sample.

[0073] For popular experts who are judged to have abnormal load The system iterates through its potential number of replicas. Combined with physical location, to determine an optimal set of device deployments. , so that the objective function Reaching the maximum value: (9) in, Representative equipment Send to experts on this hot topic The demand for tokens when devices Included in the deployment set At that time, all tokens on the device can directly hit the local copy, thus saving the cross-node transmission bandwidth that would otherwise be required; Represents the total number of copies; The parameter synchronization overhead coefficient for a unit replica is the communication resources consumed in one replica parameter synchronization, equivalent to the overhead of transmitting a certain number of tokens. It is determined by the number of model parameters and the cluster bandwidth. This term serves as a penalty factor that increases linearly with the number of replicas, and is used to suppress disordered replication when marginal returns decrease.

[0074] The solution to this optimization problem directly benefits from the sample rearrangement results of the above steps. Since the rearrangement has already been sent to experts... Traffic Traffic is highly clustered on a very small number of ranks, exhibiting a significant long-tail sparsity in its distribution. When solving the maximization problem, the algorithm only needs to select the top few devices with the highest traffic (i.e., the smaller ones). This can cover the vast majority of demand, quickly saturating the benefits on the left; while the costs on the right increase linearly with the number of replicas. Therefore, the system can achieve [benefits] with a very small number of replicas. The peak value is used to determine the optimal number of replicas. And its physical location.

[0075] The two optimization actions can be performed independently or in tandem: first, the high traffic destined for popular experts is gathered to at least a few physical nodes through rearrangement, and then a small number of replicas are created in a local area to absorb the computing hotspot, thereby simultaneously reducing bottlenecks in both communication and computing.

[0076] The load-aware optimization method provided in this application combines two indicators of expert computing load (Gini coefficient and the proportion of the most active experts) with three indicators of inter-device communication load (cross-device communication proportion, transmission coefficient of variation, and reception coefficient of variation) into a unified load state feature vector. This allows the system to simultaneously grasp the degree of computing skew and communication congestion in a single perception, accurately distinguishing the current bottleneck type. Based on this, an online evaluation model based on confidence upper limits is used to select actions by comprehensively considering expected benefits and uncertainties, avoiding the shortcomings of static strategies that cannot adapt to dynamic load changes. By reconstructing device logical mappings through greedy mapping to reduce cross-device communication volume, and by using a global placement optimization model to absorb computing hotspots with a small number of replicas in the rearranged aggregated traffic distribution, communication optimization and computing optimization can work together rather than operate independently.

[0077] The sequential execution of the above three steps can reduce the idle time of computing power when bottlenecks occur alternately, reduce the cross-device network bandwidth occupation, and maintain a high throughput in the distributed training process without changing the model structure or adding additional global synchronization overhead.

[0078] For further details, please refer to Figure 5 , Figure 5 This is a flowchart illustrating the execution sample rearrangement provided in an embodiment of this application. Figure 5 As shown, sample rearrangement is performed in the following manner: S501. Extract the communication traffic matrix from the previous training step.

[0079] After each distributed training iteration is completed, the actual number of tokens transmitted between the computing devices is automatically saved as a communication log. The S501 directly reads the communication log generated in the previous training step, extracts the number of tokens transmitted between any two physical devices, and organizes them into a communication traffic matrix.

[0080] The row indices of the communication traffic matrix correspond to the sending physical devices, and the column indices correspond to the receiving physical devices. The element value in the i-th row and j-th column of the matrix represents the total number of tokens sent from device i to device j. This matrix completely records the source, destination, and scale information of all cross-device communications in a single iteration, and the extraction of the matrix only involves reading and organizing existing log data, without introducing any additional network communication or synchronization operations. Due to the strong temporal locality of communication patterns between adjacent training iterations, the communication traffic matrix of the previous training step can serve as a reliable basis for communication optimization decisions in the current step.

[0081] S502. The problem of minimizing communication overhead is formalized into a combinatorial optimization problem that maximizes the proportion of local computation.

[0082] In the hybrid expert model, each term is routed through a gated network and then sent to the device where the target logical location is located to complete the computation. If the device where the term is located is different from the device where the target logical location is located, an inter-device communication occurs.

[0083] The core idea is to reduce the amount of communication that needs to cross physical devices by rearranging the logical numbers of each physical device in the fully interconnected communication group, while keeping the physical deployment locations of experts and the gating routing decisions completely unchanged. This allows more terms to be computed locally on their source devices. Since the total routing demand generated by the gating network in a single iteration is constant, reducing cross-device communication is equivalent to increasing the proportion of local computation, i.e., maximizing the sum of the diagonal elements of the rearranged communication traffic matrix. The diagonal elements represent the number of terms sent from a device to the same device, i.e., the amount of data that does not need to leave the device. Thus, the problem of minimizing communication overhead is formalized as a combinatorial optimization problem: among all possible permutations of device-to-logical number, find the permutation that maximizes the sum of the diagonal elements of the communication traffic matrix.

[0084] Suppose the system contains P training devices, and the rearrangement mapping from device to logistic Rank is given by the permutation vector. It means that among them This represents the logical Rank number newly assigned to the physical device originally numbered i after rearrangement. The optimization objective is formalized as: (10) in, The set of all possible permutations; The communication load intensity from physical device i to logical Rankj. In this mapping Below, the logical Rank number obtained by device i is So all the data that originally needed to be sent to Logical Rank The token's source device happens to be i itself, therefore This represents the amount of token communication that can be computed locally on device i after rearrangement. The objective function sums over i, quantifying the total amount of communication that can be localized for a given permutation. The larger the value, the more token expert computations can be completed on the source device, and the lower the cross-device communication cost.

[0085] To achieve low-overhead online scheduling, this method proposes an ultra-fast greedy mapping algorithm based on global traffic descending order. This algorithm first assigns the inter-device communication load matrix... The data is flattened and sorted globally in descending order based on communication volume. Then, the system traverses the data in descending order of traffic volume, prioritizing the mapping of the target expert with the highest traffic volume directly to the physical device where the source data resides, provided that no row or column conflicts occur (i.e., neither the source device nor the target logical location is occupied). For unallocated edge nodes, a backup mapping logic is executed. The time complexity of this algorithm is O(log n). Compared to traditional global solution methods, it can complete the reordering decision of large-scale devices in the tens of milliseconds, meeting the real-time requirements of online scheduling.

[0086] S503. The greedy mapping algorithm is used to reconstruct the mapping relationship between physical devices and logical numbers to complete the sample rearrangement.

[0087] For the combinatorial optimization problem of maximizing the proportion of local computation, if the global maximum weight matching algorithm is used to solve it, the computational time complexity is high in large-scale clusters, which makes it difficult to meet the real-time requirements of online scheduling.

[0088] This step employs a greedy mapping algorithm based on global traffic descending order. The algorithm first flattens the communication traffic matrix extracted by S501 into a list. Each record in the list contains the source device number, the target logical number, and the corresponding communication volume. Then, the list is sorted in descending order of communication volume. The algorithm iterates through each record in descending order, checking during the traversal whether the source device and target logical number in the current record are both unoccupied. If both are unoccupied, the source device of the current record is mapped to the target logical number, and both are marked as occupied. If either the source device or the target logical number is already occupied, the current record is skipped.

[0089] After the traversal is complete, for edge devices that have not yet been assigned a logical number, a backup mapping is performed using the remaining logical numbers. The time complexity of this greedy mapping algorithm is controlled to be on the order of the square of the number of devices multiplied by the logarithm of the number of devices, and it can complete the reordering decision for a large number of device groups in the tens of milliseconds. After the reordering is completed, the proportion of diagonal elements in the communication traffic matrix increases, and a large number of tokens that originally needed to be transmitted across devices through network links are transformed into accesses to the device's local memory, thereby reducing cross-node communication traffic at the source.

[0090] Here, the greedy mapping algorithm includes: Flatten the communication traffic matrix and sort it globally in descending order according to the amount of communication traffic.

[0091] Each element of the communication traffic matrix contains three pieces of information: the source device number, the target logical number, and the amount of token transmission between them. The flattening operation extracts all elements from the matrix one by one, transforming them into a list of triples, where each record consists of the source device, the target logical number, and the communication amount. The flattened list is then sorted in descending order of communication amount, placing the source device and target logical number pair with the highest communication amount at the front of the list. This sorting operation aims to ensure that the algorithm prioritizes processing the pairs with the highest communication amounts, as mapping high-communication-amount pairs to local computation yields the greatest communication savings.

[0092] Traverse the data in descending order of communication volume, and ensure that neither the source device nor the target logical location is occupied, directly map the target logical location with the largest communication volume to the physical device where the source data is located.

[0093] In this step, during the traversal, the algorithm maintains a source device occupancy record and a target logical location occupancy record. Initially, all source devices and all target logical locations are marked as unoccupied. For each record encountered, the algorithm checks whether the source device in the record has been assigned a logical number and whether the target logical location in the record has been occupied by another source device. Only when both the source device and the target logical location are unoccupied will the algorithm assign the target logical location to the source device and mark both as occupied. If either one is occupied, the current record is skipped, and the process continues to the next record. This mechanism ensures that each source device ultimately corresponds to only one logical number, and each logical number is assigned to only one source device, avoiding mapping conflicts, while prioritizing the successful mapping of the pair with the highest communication volume.

[0094] Perform a backup mapping logic on unassigned edge nodes to reconstruct the mapping relationship between physical devices and logical numbers.

[0095] After traversing all records in the flattened list, some source devices may not have matched any idle target logical positions due to low communication volume, or some target logical positions may not be occupied by any source device. The backup mapping logic arbitrarily pairs the remaining unassigned source devices with the remaining unassigned target logical positions, ensuring that all physical devices obtain a unique logical number, and the reconstruction of the mapping relationship completely covers all devices.

[0096] Furthermore, it also includes: Within a fixed-step observation window after the target action is executed, record the single-step time for each iteration and calculate the average single-step time within the observation window.

[0097] The fixed step size observation window refers to a pre-set number of consecutive iterations. The length W of the observation window is a dynamically adjustable parameter, and its setting needs to strike a balance between capturing the stable effects of action intervention and rapidly responding to load changes. The preferred value range for W is 10 to 100 training steps.

[0098] In one specific embodiment, the value can be dynamically set according to the total number of training steps T, for example, set as: W=min(50, T / 10 In another embodiment, the number of training steps can be fixed at 30 or 50. After the target action is executed, the time consumed per step is continuously recorded in each training iteration covered by the window. After the window ends, the arithmetic mean of all recorded single-step times within the window is taken to obtain the average single-step time within the window. This average value reflects the actual impact of the target action on training efficiency within the window interval.

[0099] The average single-step time is compared with the historical average single-step time before the target action is performed, and the rate of change in time is used as a reward signal.

[0100] The historical average single-step time refers to the average single-step time of all iterations within a historical window prior to executing the current target action. The system calculates the relative rate of change between the two averages, for example, subtracting the window average from the historical average and then dividing by the historical average. A positive ratio indicates faster training speed and improved efficiency, while a negative ratio indicates slower training speed and decreased efficiency. This rate of change, after being truncated or scaled, is directly used as the reward signal for the current step's decision, quantifying the actual improvement or deterioration in training throughput after the action is executed.

[0101] By using the reward signal and the load state feature vector on which the trigger decision is based, the linear parameter vector and covariance matrix corresponding to the target action are incrementally updated.

[0102] Utilize the effective rewards obtained The context feature vector that corresponds exactly to when the action is triggered Incremental updates are performed on the decision model parameters for each action: (11) in, For action The covariance matrix, whose cumulative update process can reflect the coverage of the feature space, thus ensuring the prediction stability of the model in different contexts. For feature vectors Its outer product with itself is used to supplement the information accumulation in the feature space; For action The cumulative reward-weighted feature vector, whose update is directly related to the effective reward; This is the effective reward scalar after reshaping and truncation, used to quantify the actual benefit after the action is performed; For the updated covariance matrix The inverse matrix is ​​the core computational factor for updating the linear parameter vector; The updated linear parameter vector, whose update directly corrects the expected return of actions, enabling the strategy to quickly adapt to dynamic changes in load; symbol This indicates an assignment operation, which assigns the result of the calculation on the right side of the equation to the variable on the left side.

[0103] Specifically, the update operation is as follows: First, the covariance matrix of the target action is multiplied by the outer product of the load state feature vector and itself, thus accumulating and updating the covariance matrix to continuously expand the information coverage of the feature space. Then, the cumulative reward-weighted feature vector is multiplied by the product of the reward signal and the load state feature vector, completing the weighted accumulation. Finally, the updated linear parameter vector is obtained by multiplying the inverse of the updated covariance matrix by the updated cumulative reward-weighted feature vector. This update process involves only algebraic operations on matrices and vectors, with minimal computational cost, and will not cause any perceptible delay to the main training process.

[0104] The above steps collect real training time data through a fixed step-size window and convert it into a reward signal. This reward signal then drives the incremental update of the decision model's parameters, enabling the online evaluation model to adjust its estimation of the relationship between load state and action reward based on the actual effect after each action execution. The model continuously updates online as the training progresses, eliminating the need for an offline pre-training phase. This gradually improves the accuracy of identifying the optimal action under different load states, adapts to the non-stationary evolution of load distribution during training, and avoids decision lag and mismatch caused by static strategies.

[0105] Furthermore, this application also includes a decision-making reuse step: In the early stages of training, load similarity analysis is performed on each layer of the hybrid expert model. The cosine similarity of expert loads between adjacent layers is calculated. Based on the similarity variation pattern, the model depth is divided into shallow region, intermediate homogeneous region and deep region.

[0106] Here, the hybrid expert model is composed of multiple stacked transformers, with some layers replacing the standard feedforward network with feedforward expert modules. Different network depths exhibit varying sensitivities to input feature selection, resulting in different degrees of skewness in the expert load of each layer. In the initial iterations after training begins, the system records the number of tokens received by each expert layer by layer, forming the expert load vector for each layer.

[0107] For any two adjacent network layers, the expert load vectors of the upper and lower layers are taken, and the cosine similarity between them is calculated. The cosine similarity is the inner product of the two vectors divided by the product of their magnitudes. The closer the value is to one, the more similar the load distribution patterns of the two layers are; the closer it is to zero, the greater the distribution difference. The system traverses all adjacent layer pairs to obtain a set of cosine similarity values, and then observes the trend of similarity changes along the network depth direction: when the cosine similarity between adjacent layers is consistently higher than a high threshold, it indicates that these layers are in a highly homogeneous load distribution segment; when the similarity drops significantly at the boundary of a certain layer, it is marked as a region boundary. Based on these boundaries, the model depth is divided into shallow regions, intermediate homogeneous regions, and deep regions. The shallow and deep regions each contain one to three layers, and the intermediate homogeneous region contains the remaining layers.

[0108] In subsequent training, independent decision-making is maintained only for the first layer of the shallow region and the last layer of the deep region. For each layer in the middle homogeneous region, a decision is triggered only once in the first layer of the region and its result is reused in all layers of the region to complete the decision reuse.

[0109] After dividing the region, for both shallow and deep regions, each layer independently executes a complete perception and decision-making process in each iteration, that is, each layer calculates its own load state feature vector and inputs it into the online evaluation model to select the target action corresponding to its layer.

[0110] For intermediate homogeneous regions, since the load distribution of each layer within the region is highly similar, the decision-making process of the online evaluation model is triggered only once at the first layer of that region. The target action corresponding to the first layer is selected, and then this target action is directly applied to all subsequent layers within the region. Subsequent layers do not repeat the decision-making process. This reuse mechanism allows all layers within the intermediate homogeneous region, except for the first layer, to skip the computation stage of the online evaluation model, compressing the number of decision-making operations from layer-by-layer execution to only once per region.

[0111] The aforementioned decision reuse step identifies network segments with homogeneous load distributions through inter-layer load similarity analysis in the early stages of training, and shares the same decision results among multiple layers within these homogeneous segments during subsequent training. This step directly reduces the number of times the online evaluation model needs to be run in each iteration, reducing the total number of online evaluation model calls from being proportional to the total number of network layers to being only related to the number of the first layer in shallow, deep, and intermediate homogeneous regions. This reduces the cumulative computational overhead of the scheduling process without changing the model structure and routing decisions.

[0112] For further details, please refer to Figure 6 , Figure 6 This is a schematic diagram of the internal structure of the Hybrid Expert Model (MoE) module provided in an embodiment of this application. Figure 6As shown in the image, the Chinese meanings of each English symbol are as follows: Input hidden state u t (InputHidden) is the latent vector of the previous layer's output at the current time step; a router, or gated network, is used to calculate the probability or weight of each routing expert being selected; Top Kr indicates that the router selects the top Kr routing experts with the highest scores to participate in the calculation of the current term; the routing experts are numbered 1, 2, 3, 4, ..., Nr There are Nr independent expert subnetworks, which are dynamically selected and activated by the router; there are Ns shared experts, numbered from 1 to Ns, which are always active and used to capture features common to all inputs; the output hidden state ht′ is obtained by fusing the output of the shared experts with the output of the selected routing experts.

[0113] The overall working principle of this graph is as follows: In the forward computation of the MoE module, the input hidden state ut is first sent to the router. The router calculates a score for each routed expert, and then selects the top Kr experts (Top Experts) based on their scores. The selected routing expert and all shared experts independently compute the input ut in parallel. Finally, the outputs of the shared experts are weighted and fused with the outputs of the selected routing experts, or merged in other ways, to obtain the final output hidden state ht′. This structure combines sparse activation (only a few routing experts are triggered) with shared feature extraction, improving the model's expressive power while controlling computational overhead. The MoE training adaptive co-optimization method involved in this application is applied to hybrid expert models composed of such modules.

[0114] Compared with existing methods centered on sample rearrangement and expert replicas, the load-aware optimization method and device provided in this application simultaneously sense the dual-dimensional characteristics of expert computing load and inter-device communication load, combining them into a load state feature vector. An online evaluation model adaptively selects the target action from three candidate actions—maintaining the current configuration, performing sample rearrangement, and performing expert replicas—based on the load state feature vector. Furthermore, during sample rearrangement, a greedy mapping algorithm reconstructs the mapping relationship between physical devices and logical numbers to reduce cross-device communication volume. During expert replica creation, a small number of replicas are used to absorb computing hotspots in the rearranged traffic distribution. This solves the problems of existing methods having a single optimization dimension and lacking an adaptive collaborative intervention mechanism. It achieves collaborative optimization of communication and computing resources in complex scenarios where computing overload and communication congestion alternate, reducing cross-device network bandwidth usage, minimizing idle computing time during bottleneck switching, and maintaining high throughput in the distributed training process.

[0115] Furthermore, in the embodiments of this application, iterative execution and training result output are also included.

[0116] Throughout the training process, two core monitoring metrics are continuously recorded after each iteration step: single-step iteration time and training loss value. The training loss value characterizes the model's fit to the training data, and its decreasing trend with the number of iterations directly reflects the convergence quality of the model training. After training terminates, two sets of results are output. First, the parameters of the completed hybrid expert model. To verify the impact of this method on training quality, the curve of the loss value recorded during training as a function of iteration steps is compared with the curve of the baseline model's loss value trained under the same conditions without this optimization method. If the decreasing trends of the two loss curves are basically overlapping at the same number of iterations, and their final convergence levels are comparable, it proves that the scheduling optimization introduced by this method does not impair the model's training accuracy. Second, the time statistics of the entire training process, using the average single-step time or total training time as comparison metrics, quantifies the acceleration effect of this method compared to the baseline method. These two outputs together verify the technical effectiveness of this method in improving training efficiency while maintaining model accuracy.

[0117] experiment: This experiment uses the domestic Ascend NPU computing platform for performance evaluation. The physical test cluster consists of one compute node, with each node equipped with eight Ascend 910B AI accelerator cards, each with 64GB of dedicated video memory. For topology interconnection, high-bandwidth full interconnection is achieved between compute cards within a node via the HCCS high-speed bus, while cross-machine distributed communication topology is constructed using a 100Gbps RoCE RDMA network between nodes. The specific software configuration of the experimental environment is shown in Table 1. Table 1

[0118] The test datasets used four corpora: Wikitext-103, C4-200M, Meta Math QA, and Slim Pajama-chunked. By leveraging their differences in semantic distribution, text length, and routing stability, the adaptive scheduling capability of the proposed method was comprehensively tested. The model structure used was based on the DeepSeekV3-MoE design, with adjustments made to some parameter sizes and expert configurations according to experimental requirements. The hidden layer dimension was 4096, the number of layers was 4, the number of multi-head attention heads was 16, the total number of MoE experts was 64, the routing Top-K value was 8, and the number of experts shared per computation card was 1. Table 2 lists the main hyperparameter settings of the DeepSeekV3 model in this experiment.

[0119] Table 2

[0120] To verify the performance improvement effect of the load-aware MoE training adaptive collaborative optimization method during training, this experiment conducts comparative tests, selecting NetMoE as the baseline. Its core idea is to adjust the mapping relationship between samples and computing nodes before All-to-All communication through a dynamic sample placement strategy, transforming cross-node communication into intra-node communication, thereby reducing communication overhead. Under a fixed sequence length, the end-to-end single-step training time of NetMoE and the proposed method on different datasets is tested. These time data are averages calculated after 500 consecutive training steps. The comparison results for a sequence length of 4096 are shown in Table 3, and the comparison results for a sequence length of 8192 are shown in Table 4.

[0121] Table 3

[0122] Table 4

[0123] Compared to NetMoE, the method in this application achieves different degrees of speedup at sequence lengths of 4096 and 8192. At a sequence length of 4096, the end-to-end training speedup of the method in this application is approximately 3.3% to 8.4% higher than that of NetMoE; at a sequence length of 8192, the end-to-end training speedup is approximately 7.0% to 16.4%.

[0124] Table 5 records the end-to-end single-step time for three independent tests on the Wikitext-103 dataset with a sequence length of 8192. The results show that the proposed method can stably speed up the process by approximately 16% in each run, further validating the stability of the proposed method.

[0125] Table 5

[0126] To verify that this method improves training throughput without compromising model accuracy, the experiment simultaneously extracted the complete 600 iterations of the feedforward language model loss log data from the Wikitext-103 dataset (sequence length 8192). Given the large number of steps throughout the entire lifecycle, Table 6 uses equidistant sampling to present the actual loss values ​​of representative nodes in the training logs for time-series comparison. The comparison results show that the single-step loss values ​​of this application and the baseline method are in good agreement, and the extremely small fluctuations between them are entirely within the normal floating-point rounding error range of distributed hardware, proving the model's convergence accuracy.

[0127] Table 6

[0128] The optimization scheme proposed in this application is based on a load-aware adaptive collaborative scheduling and sample rearrangement, along with an expert replica linkage strategy. This effectively alleviates the problems of single-dimensional static optimization strategies failing to adapt to spatiotemporal dynamic load changes and insufficient utilization of communication and computing resources due to scheduling decision lags. Under training scenarios with different semantic distribution characteristics and sequence lengths, this optimization method maintains stable acceleration performance, especially on datasets with high topic concentration and a high risk of extreme routing skew. It can take into account different data distribution characteristics and long sequence load fluctuations, ensuring stable acceleration of the MoE model training process and significantly improving overall training efficiency.

[0129] Based on the same inventive concept, this application also provides a load-aware optimization device corresponding to the load-aware optimization method. Since the principle of the device in this application is similar to the load-aware optimization method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0130] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of the load-aware optimization device provided in an embodiment of this application. Figure 7 As shown, the load-aware optimization device 600 includes: The load state feature vector combination module 601 is used to sense the expert calculation load features and inter-device communication load features of the current training step, and combine the two into a load state feature vector. The target action selection module 602 is used to input the load state feature vector into the online evaluation model, and the online evaluation model selects a target action from three candidate actions: maintain the current configuration, perform sample rearrangement, and perform expert copy. The optimization execution module 603 is used to perform corresponding operations based on the selected target action: if the target action is to perform sample rearrangement, then the sample rearrangement is performed to reduce cross-device communication; if the target action is to perform expert replication, then expert replication creation is performed to distribute the computing load.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between apparatuses or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0135] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A load-aware optimization method, characterized in that, include: The expert computation load characteristics and inter-device communication load characteristics of the current training step are perceived and combined into a load state feature vector. The load state feature vector is input into the online evaluation model, which then selects a target action from three candidate actions: maintaining the current configuration, performing sample rearrangement, and performing expert replication. Based on the selected target action, perform the corresponding operation: if the target action is to perform sample rearrangement, then perform sample rearrangement to reduce cross-device communication; if the target action is to perform expert replication, then perform expert replication creation to distribute the computational load.

2. The method according to claim 1, characterized in that, The expert-calculated load characteristics are obtained through the following method: Read the number of tokens received by each expert in the previous training step to form the expert load vector; Calculate the Gini coefficient of the expert load vector as a measure of the overall skewness; The ratio of the number of tokens received by the most popular expert to the total number of tokens received by all experts is calculated as a measure of the concentration of a single expert. The Gini coefficient and the percentage of tokens received by the hottest expert are combined to form the expert computational load feature.

3. The method according to claim 1, characterized in that, The inter-device communication load characteristics are obtained in the following way: Read the word transmission records between physical devices and between physical devices and logical computing units in the previous training step to form a communication traffic matrix; The ratio of the number of tokens transmitted across devices to the total number of tokens transmitted is calculated as a characteristic of cross-device communication proportion. Calculate the coefficient of variation of the transmit load and the coefficient of variation of the receive load for each device as a measure of the unevenness of the communication load distribution; The cross-device communication proportion feature, the transmit load variation coefficient, and the receive load variation coefficient are combined to form the inter-device communication load feature.

4. The method according to claim 1, characterized in that, The online evaluation model is a linear evaluation model based on the upper confidence limit. This model maintains a linear parameter vector and a covariance matrix for each candidate action. The target action is selected in the following manner: Calculate the inner product of the linear parameter vector and the load state feature vector, and use it as the expected revenue term; The product of the exploration coefficient and the result of the operation based on the load state feature vector and the inverse of the covariance matrix is ​​calculated as the uncertainty term; The expected return term is added to the uncertainty term to obtain the comprehensive score of the candidate action; The candidate action with the highest overall score is selected as the target action.

5. The method according to claim 4, characterized in that, Also includes: Within a fixed-step observation window after the target action is executed, the single-step time of each iteration is recorded, and the average single-step time within the observation window is calculated. The average single-step time is compared with the historical average single-step time before the target action is performed, and the rate of change in time is used as a reward signal. Using the reward signal and the load state feature vector on which the trigger decision is based, the linear parameter vector and covariance matrix corresponding to the target action are incrementally updated.

6. The method according to claim 1, characterized in that, The sample rearrangement is performed in the following manner: Extract the communication traffic matrix from the previous training step; The problem of minimizing communication overhead is formalized as a combinatorial optimization problem that maximizes the proportion of local computation. A greedy mapping algorithm is used to reconstruct the mapping relationship between physical devices and logical numbers to complete the sample rearrangement.

7. The method according to claim 6, characterized in that, The greedy mapping algorithm includes: Flatten the communication traffic matrix and sort it globally in descending order according to the amount of communication traffic; Traverse the data in descending order of communication volume, and map the target logical location with the largest communication volume directly to the physical device where the source data is located, provided that neither the source device nor the target logical location is occupied. Perform a backup mapping logic on unassigned edge nodes to reconstruct the mapping relationship between physical devices and logical numbers.

8. The method according to claim 1, characterized in that, Perform expert copy creation as follows: Based on the communication traffic distribution after sample rearrangement, experts whose word element reception exceeds a preset threshold are identified as popular experts. A global placement optimization model is constructed with the goal of maximizing the difference between communication savings and parameter synchronization costs to determine the number of replicas and deployment locations of the popular experts. Based on the determined number of replicas and deployment locations, create expert replicas at the determined deployment locations to complete the creation of expert replicas.

9. The method according to claim 1, characterized in that, It also includes decision reuse steps: In the early stage of training, load similarity analysis is performed on each layer of the hybrid expert model, the cosine similarity of expert load between adjacent layers is calculated, and the model depth is divided into shallow region, intermediate homogeneous region and deep region according to the similarity change law. In subsequent training, independent decision-making is maintained only for the first layer of the shallow region and the last layer of the deep region. For each layer in the middle homogeneous region, a decision is triggered only once in the first layer of the region and its result is reused in all layers of the region to complete the decision reuse.

10. A load-aware optimization device, characterized in that, include: The load state feature vector combination module is used to perceive the expert computation load characteristics and inter-device communication load characteristics of the current training step, and combine the two into a load state feature vector. The target action selection module is used to input the load state feature vector into the online evaluation model, and the online evaluation model selects a target action from three candidate actions: maintain the current configuration, perform sample rearrangement, and perform expert copy. The optimization execution module is used to perform corresponding operations based on the selected target action: if the target action is to perform sample rearrangement, then sample rearrangement is performed to reduce cross-device communication; if the target action is to perform expert replication, then expert replication creation is performed to distribute the computational load.

Images