Adaptive weight language model adjustment method based on environment feedback

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a self-adaptive weight language model adjustment method based on environment feedback, which comprises the following steps: firstly, establishing topological mapping of expert nodes and communication channel groups; secondly, channel-level aggregation and capacity normalization are carried out on the collected flow, and a dominant congestion distribution vector representing a network bottleneck structure is extracted by using an iterative algorithm; then, calculating the projection proportion of the current load on the dominant congestion distribution vector to determine the locking strength; and finally, generating a suppression coefficient and a smoothing parameter based on the locking strength, which are respectively used for adjusting expert selection bias and output scoring distribution of the router. Through cross-layer feedback and two-dimensional nonlinear control, intrinsic mode locking of network congestion is effectively broken, and communication delay of distributed reasoning is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an adaptive weighted language model adjustment method based on environmental feedback. Background Technology

[0002] In recent years, large-scale language models based on the Mixture of Experts (MoE) architecture have become the mainstream choice in the field of natural language processing because they can increase model capacity while maintaining low inference computation. In the MoE architecture, each layer of the model contains multiple expert networks working in parallel. For each input token, the routing network (Router / Gate) distributes it to a few of the most relevant experts for processing based on its semantic attributes.

[0003] In real-world industrial deployments, the sheer number of parameters in a MoE model typically prevents it from being loaded onto a single computing device. Instead, it requires distributed deployment across multiple computing nodes (such as GPU servers). This inevitably leads to a significant amount of cross-device communication during inference. Particularly in Expert Parallelism mode, each layer of MoE computation requires two all-to-all collective communication operations: dispatch and combine. This means that communication efficiency directly impacts the end-to-end inference latency of the entire system.

[0004] In current technologies, MoE's routing strategies primarily focus on load balancing at the model level. For example, to prevent a few experts from becoming computational bottlenecks by handling too many tokens, existing routing algorithms typically introduce auxiliary loss functions or set capacity limits, attempting to force tokens to be evenly distributed among logical experts. However, this load balancing strategy based solely on logical expert load has significant limitations in real-world deployment environments.

[0005] The main problem is that existing routing mechanisms lack awareness of the topology of the underlying interconnect network. Modern data centers typically employ a layered interconnect architecture (such as Rail-Optimized networks or Clos networks), where the interconnect bandwidth within a device is much higher than the interconnect bandwidth across devices or racks, and cross-node communication is often limited to a few critical paths (such as Spine switch ports or Rail channels). This constitutes a cross-layer cutset constraint.

[0006] In practical reasoning, the input data of language models naturally exhibits a long-tail distribution, causing routing requests to tend to converge towards a few popular experts. When this logical convergence trend happens to align with the direction of certain critical paths (cut sets) with limited bandwidth in the network, it can trigger severe network congestion. Specifically, this manifests as a large number of data flows simultaneously competing for the same set of link resources, forming a many-to-one congestion pattern (incast).

[0007] However, existing expert-count-based load balancing algorithms cannot identify this level of congestion. The system might redirect traffic from a busy expert to an idle one, but if these two experts happen to be on the same congested rail, this redirection not only fails to alleviate communication latency but may also exacerbate congestion due to the introduced scheduling overhead. Once this congestion pattern forms, it is often repeatedly amplified by the synchronization barrier mechanism in the MoE architecture, resulting in persistently high tail latency and severely impacting the real-time performance and stability of online inference services. Summary of the Invention

[0008] This invention provides an adaptive weighted language model adjustment method based on environmental feedback, which solves the technical problems mentioned in the background art.

[0009] This invention provides an adaptive weighted language model adjustment method based on environmental feedback, comprising: Configure a topology mapping table between logical expert nodes in the hybrid expert model and the communication channel group to which their computing devices belong; The collected inter-device communication traffic data is aggregated based on the communication channel groups, and combined with the channel bandwidth capacity to generate a dominant congestion distribution vector characterizing the distribution status of key network bottlenecks. The congestion pattern matching degree is determined based on the similarity between the current traffic load distribution and the dominant congestion distribution vector. Based on the congestion pattern matching degree, a congestion suppression coefficient and a distribution smoothing parameter are generated. The congestion suppression coefficient is used to adjust the expert selection bias parameter of the router according to the dominant congestion distribution vector, and the distribution smoothing parameter is used to non-linearly scale the route score value to generate an updated route probability distribution that avoids key network bottlenecks.

[0010] The beneficial effects of this invention include: Existing technologies typically rely solely on expert load counts within the model for load balancing, neglecting the actual network capacity. This invention establishes a mapping between logical expert nodes and communication channel groups, reducing the granularity of congestion awareness from logical experts to cut sets. This enables the system to identify hidden bottlenecks where experts themselves are not busy, but their network paths are congested, thereby preventing traffic from converging towards these bottleneck areas and significantly reducing communication tail latency caused by network topology limitations.

[0011] Unlike traditional simple feedback based on thresholds (such as bandwidth utilization), this invention introduces spectral analysis to extract the dominant congestion distribution vector (eigenmode) from the traffic matrix. This approach can identify the shape matching degree between the traffic distribution and the network's vulnerable structure, thus detecting mode locking trends in advance before congestion fully erupts. This deep structural awareness capability makes routing adjustments predictable, avoiding the network oscillations caused by the lag in adjustments in traditional methods.

[0012] This invention designs a control mechanism that coordinates the congestion suppression coefficient and the distribution smoothing parameter. On one hand, the suppression coefficient is used to apply a targeted potential energy penalty to experts located on congested channels, achieving targeted elimination; on the other hand, the smoothing parameter (temperature) is used to dynamically adjust the routing entropy, flexibly diffusing traffic to non-congested areas. This approach can quickly break congestion lockout while preventing the model from losing its specialized capabilities due to excessive intervention, ensuring a balance between inference throughput and model accuracy. Attached Figure Description

[0013] Figure 1 This is a flowchart of an adaptive weighted language model adjustment method based on environmental feedback according to the present invention; Figure 2 This is an architecture diagram of the adaptive weighted language model of the present invention. Detailed Implementation

[0014] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0015] like Figure 1 As shown, an adaptive weighted language model adjustment method based on environmental feedback includes: Configure a topology mapping table between logical expert nodes in the hybrid expert model and the communication channel group to which their computing devices belong; The collected inter-device communication traffic data is aggregated based on the communication channel groups, and combined with the channel bandwidth capacity to generate a dominant congestion distribution vector characterizing the distribution status of key network bottlenecks. The congestion pattern matching degree is determined based on the similarity between the current traffic load distribution and the dominant congestion distribution vector. Based on the congestion pattern matching degree, a congestion suppression coefficient and a distribution smoothing parameter are generated. The congestion suppression coefficient is used to adjust the expert selection bias parameter of the router according to the dominant congestion distribution vector, and the distribution smoothing parameter is used to non-linearly scale the route score value to generate an updated route probability distribution that avoids key network bottlenecks.

[0016] In a preferred embodiment, configuring a topology mapping table between logical expert nodes in the hybrid expert model and the communication channel groups to which their respective computing devices belong includes: Construct a device channel association matrix Its element definition is as follows: ; in, This indicates the total number of computing devices. Indicates the total number of packets in the communication channel; Constructing an expert channel association matrix Its element definition is as follows: ; in, This represents the total number of logic expert nodes. For indicator functions, Represents logical expert nodes An index of the communication channel group to which the residing computing device belongs.

[0017] It should be noted that the first step is to configure the topology mapping table between the logical expert nodes and the communication channel groups to which the computing devices belong in the hybrid expert model. In the distributed inference deployment of the hybrid expert model, different computing devices will correspond to different communication channel groups, and logical expert nodes need to reside on the computing devices. In order to associate the expert's routed traffic with the resource status of the communication channel groups later, it is necessary to first establish the mapping relationship between the computing devices and the communication channel groups, as well as between the logical expert nodes and the communication channel groups.

[0018] Specifically, the first step is to determine the total number of computing devices. , The number of computing devices participating in hybrid expert model inference; then determine the total number of communication channel packets. , This represents the number of communication channel groups divided within the deployment environment. Based on this, a device channel association matrix is ​​constructed. ,in This is the index of the computing device, with a value ranging from 1 to... , This is the index of the communication channel group, and its value ranges from 1 to... Elements in the matrix The rule for determining the value is: when the computing device The corresponding communication channel group is hour, When computing devices The corresponding communication channel group is not hour, This rule can be used to determine The values ​​of all elements in the dataset are used to establish the correspondence between computing devices and communication channel groups.

[0019] Next, determine the total number of logic expert nodes. , Let be the number of logical expert nodes in the hybrid expert model; for each logical expert node... , The value range is 1 to , For logic expert nodes The index of the communication channel packets corresponding to the residing computing device. The value of is determined by the communication channel group to which the computing device belongs. Based on this, an expert channel correlation matrix is ​​constructed. The elements in the matrix The value is determined by Sure, The rule for determining the value is: when The value of is related to the communication channel packet index. When the values ​​are the same, ,at this time ;when The value of is related to the communication channel packet index. When the values ​​are inconsistent, ,at this time This rule can be used to determine The values ​​of all elements in the table are used to establish the correspondence between logical expert nodes and communication channel groups.

[0020] Communication channel grouping is a description specific to the network deployment scenario of MoE distributed inference: In actual deployments, network interconnection topologies (such as leaf-spine architecture) typically contain multiple independent communication links, which are then divided into... A discrete, indexable group (e.g.) At this time, the four packets correspond to four independent spine links, and each packet corresponds to a communication channel with quantifiable resource status. This abstraction is not a general network layering concept, but rather a way to reduce the dimensionality of high-dimensional inter-device communication traffic by projecting it onto a more dimensional plane. By grouping dimensions and then performing specific modeling for congestion modal analysis, complex network congestion problems can be quantified through group-level traffic aggregation.

[0021] It is a three-layer mapping logic that connects model layer experts with layer communication resources: for logical expert nodes First, determine the computing device where it resides (such as an expert). Residing in the device Then determine the communication channel group to which the device belongs (e.g., device). Belongs to group ),final That is, the index of the group (e.g.) This logic is not a simple resource allocation relationship, but an intermediate rule that directly associates the expert nodes within the MoE model with the communication channel groups in the deployment, so that subsequent routing adjustments to experts can directly correspond to the resource status of specific communication channels.

[0022] The device channel association matrix is ​​defined as follows: ,in To calculate the number of devices, For the number of communication channel packets, element The rule for determining the value is: if the computing device Belongs to communication channel packets ,but Otherwise For example, when , When devices 1-4 belong to group 1 and devices 5-8 belong to group 2, The first four rows and first column contain 1s, the last four rows and second column contain 1s, and the rest are 0s. This matrix represents the device-level communication traffic matrix. ( )pass Projection as channel group-level flow matrix ( ).

[0023] The expert channel correlation matrix is ​​defined as follows: ,in For the number of logic expert nodes, elements ( For the characteristic function, when It takes the value 1 when it is active, and 0 otherwise. For example, when... , And experts 1-8 Experts 9-16 hour, The first 8 rows and first column contain 1, and the last 8 rows and second column contain 1. Existing technologies typically only establish an expert-to-device mapping, while the purpose of constructing this matrix is ​​to directly bind experts to communication channel groups. Subsequent adjustments to the expert's routing bias parameters can be made through... It is directly related to the congestion potential of the corresponding channel group, realizing direct feedback of model weight adjustment on network congestion.

[0024] It should be noted that existing MoE inference optimization methods are mostly based on adjusting routes according to device computational load. However, constructing a mapping between devices, experts, and communication channel packets directly correlates the expert's routed traffic with the resource status (bandwidth, congestion) of the communication channel packets. Since the performance bottleneck of MoE inference often comes from all-to-all communication rather than computation, this mapping can aggregate device-level traffic to the channel packet dimension, analyze its congestion mode, and then transform the congestion status into expert route weight adjustment signals. Ultimately, this achieves a closed loop of network environment state → model weight optimization, specifically addressing the problem of uncontrolled weight adjustment caused by communication congestion.

[0025] In a preferred embodiment, the collected inter-device communication traffic data is aggregated based on the communication channel groups, and combined with the channel bandwidth capacity to generate a dominant congestion distribution vector characterizing the distribution status of key network bottlenecks, including: Calculate the channel-level aggregated traffic matrix : ; in, For the first The layered hybrid expert model represents the device-level communication traffic matrix in the current communication cycle. This is the device channel association matrix; Construct a capacity-normalized flow matrix : ; in, For channel bandwidth capacity vector The capacity diagonal matrix formed; Extract the dominant congestion distribution vector through iterative calculation. : Set initial vector ,implement Next iteration update: ; in, Denotes the Euclidean norm of a vector, taking the vector after the iteration. As the dominant congestion distribution vector .

[0026] It should be noted that device-level communication traffic data only corresponds to the communication status of a single computing device, while the actual communication bottleneck occurs at the communication channel group dimension. Therefore, device-level traffic needs to be mapped to the communication channel group dimension to aggregate the traffic status at this dimension. At the same time, different communication channel groups have different bandwidth capacities, and directly using aggregated traffic cannot reflect the actual load level. Therefore, normalization processing is required in conjunction with bandwidth capacity. The dominant congestion distribution corresponds to the main eigenvector of the normalized traffic matrix. This vector can be stably extracted through power iteration to represent the distribution status of key bottlenecks.

[0027] Specifically, first determine , This is the layer index of the hybrid expert model, and its value range is the number of layers contained in the hybrid expert model; For the first The layered hybrid expert model represents the device-level communication traffic matrix within the current communication cycle, where the elements of the matrix correspond to the communication traffic data between different computing devices. Based on the previously configured device channel association matrix, calculate the channel-level aggregated traffic matrix. Its calculation formula is This formula can be used to map device-level traffic data to the communication channel group dimension, thereby obtaining the aggregated traffic relationship between different communication channel groups.

[0028] Next, determine the channel bandwidth capacity vector. , The elements correspond to the bandwidth capacity of each communication channel packet; based on Construct a capacity diagonal matrix The diagonal elements of this matrix are The elements, with off-diagonal elements being 0; then through Construct a capacity-normalized flow matrix Its calculation formula is This eliminates the impact of differences in bandwidth capacity between different communication channels on traffic status representation, enabling the matrix to reflect the relative actual load of each channel group.

[0029] Next, extract the dominant congestion distribution vector. First, set the initial vector. , The preferred value is a unit vector with all elements equal, for example... (in It is a vector of all 1s. (Total number of communication channel groups); then determine the number of iterations. , The preferred value is 10 because the exponentiation iteration converges to the principal eigenvector of the matrix in about 10 iterations; then execute... The second iteration update, the... The formula for calculating the next iteration is: ,in The Euclidean norm of a vector is used to normalize the vector after iteration; (Complete) After the iteration, take the vector after the iteration ends. As the dominant congestion distribution vector This characterizes the distribution of key bottlenecks in the network.

[0030] The formula for calculating the channel-level aggregated traffic matrix is ​​as follows: ,in It is a device channel association matrix. It is a device-level communication traffic matrix. Through this specific projection transformation, high-dimensional inter-device traffic (such as...) is transformed into a more comprehensive model. Each device corresponds to Dimensionality reduction from traffic matrix to communication channel group dimension (e.g.) Each group corresponds to (Matrix), which aggregates the traffic status between different groups. For example, when devices 1-4 belong to group 1 and devices 5-8 belong to group 2, for 0-1 matrix, Traffic between devices 1-4 will be aggregated into internal traffic of group 1, and traffic between devices 1-4 and 5-8 will be aggregated into traffic from group 1 to group 2. Traffic distribution that directly corresponds to the bottleneck dimension is a unique method for modeling network congestion.

[0031] The formula for calculating the capacity-normalized flow matrix is ​​as follows: ,in It is a channel bandwidth capacity vector The corresponding diagonal matrix. By employing this symmetric normalization form (similar to the normalization of the Thule-Laplace matrix), it is to make... This makes it a symmetric matrix, thus supporting subsequent spectral analysis (eigenvector extraction). For example, if... (Unit: GB / s), then Through this matrix After symmetrical scaling, The elements can reflect the relative relationship between traffic and bandwidth, rather than the absolute value of traffic. This is an innovative way to transform network congestion into a spectral analysis problem.

[0032] The dominant congestion distribution vector is calculated iteratively from... The vectors extracted are not existing network metrics (such as latency and packet loss rate), but rather congestion shape metrics. Specifically, the dominant congestion distribution vector corresponds to... The principal feature vector, whose element distribution characterizes the spatial distribution of key bottlenecks in the network. For example, if (correspond If the first communication channel group is the main congestion bottleneck of the current network, then the vector quantifies the shape characteristics of which groups are congested and is the core reference variable for achieving environmental feedback adaptation.

[0033] This scheme applies the power iteration algorithm to the capacity-normalized flow matrix. To extract the dominant congestion distribution vector. The core is through... The iterative process gradually converges from the initial vector to... The dominant eigenvectors are used to obtain the dominant congestion distribution vector. Combining power iteration with the network traffic matrix to extract the shape of the congestion distribution is a unique approach to introducing spectral analysis into network congestion modeling.

[0034] Initial vector optimization ,in It is a vector of all 1s, for example when hour, This initial value was chosen to unbiasedly detect the congestion status of all communication channel packets, avoiding initial bias towards any particular packet; the optimal number of iterations was also selected. This value is derived from engineering experience; the exponentiation iteration converges in about 10 iterations. The principal feature vector ensures the accuracy of the results without increasing the computational overhead due to excessive iterations; this is a specific parameter constraint for the actual implementation.

[0035] It's important to note that existing technologies typically analyze congestion directly based on absolute traffic values. However, by combining this with the bandwidth capacity of communication channel packets, the relative relationship between traffic and bandwidth is analyzed to reflect the actual load level. For example, if packet 1 has a traffic of 100 and a bandwidth of 100, while packet 2 has a traffic of 150 and a bandwidth of 200, packet 2 appears to have a higher absolute traffic. However, after normalization, the load of packet 1 (100 / 100=1) is higher than that of packet 2 (150 / 200=0.75), meaning packet 1 actually has a greater risk of congestion. Eliminating bandwidth differences between different packets through normalization and focusing on the relative distribution of load is a unique approach to addressing network congestion.

[0036] In a preferred embodiment, determining the congestion pattern matching degree based on the similarity between the traffic load distribution of the current communication cycle and the dominant congestion distribution vector includes: Calculate the normalized channel load vector : ; in, For channel-level aggregated traffic matrix, packets from communication channels arrive Traffic, Grouping communication channels Channel bandwidth capacity, The duration of the observation window; Calculate the normalized dominant congestion distribution vector : ; in, For the first The dominant congestion distribution vector of the layer, It is a vector of all 1s. This indicates the absolute value operation; Calculate congestion pattern matching degree : First calculate the... Single-layer congestion pattern matching degree : ; Then take the average across all layers: ; in, This represents the total number of layers in the hybrid expert model.

[0037] It should be noted that the current traffic load distribution needs to be quantized into a vector with the same dimension as the dominant congestion distribution vector in order to measure the similarity between the two. At the same time, the dominant congestion distribution vector needs to be normalized to eliminate the difference in magnitude. Then, the matching degree of a single layer is calculated through the projection relationship between the vectors. Finally, the matching degree of all layers is averaged to obtain a scalar that reflects the matching status of the congestion pattern of the entire model.

[0038] Specifically, the normalized channel load vector is first calculated. ,in For the layer index of the hybrid expert model, For the first Layer-level channel-level aggregated traffic matrix Grouping from the communication channel in this matrix To communication channel packets Traffic, Grouping communication channels Channel bandwidth capacity, For the duration of the observation window, The preferred value is 10 seconds, which is a commonly used length of traffic observation time in inference scenarios. medium elements The calculation formula is: This formula quantifies the load of different communication channel groups into a unified metric by using the ratio of traffic to bandwidth capacity and observation duration.

[0039] Next, the normalized dominant congestion distribution vector is calculated. ,in For the first The dominant congestion distribution vector of the layer, It is a vector of all 1s. This indicates the absolute value operation, which is used to eliminate the difference in signs between elements in a vector. Based on these parameters, The calculation formula is: This formula normalizes the dominant congestion distribution vector by taking the inner product of the transpose of the all-1 vector and the absolute value vector, so that the sum of the vector elements is 1, thus transforming it into a distribution form.

[0040] Then calculate the congestion pattern matching degree. First calculate the first Single-layer congestion pattern matching degree Its calculation formula is ,in The dot product of the normalized channel load vector and the normalized dominant congestion distribution vector reflects the degree of similarity between the two. The sum of all elements of the normalized channel load vector is used to normalize the similarity; then determine... The total number of layers in the hybrid expert model is defined as the actual number of layers included in the model. The average congestion pattern matching degree across all layers is calculated using the following formula: This formula obtains a scalar that reflects the current congestion pattern matching status of the entire hybrid expert model by averaging the matching degree of each layer.

[0041] Normalized channel load vector The calculation formula is: ,in It is a channel-level aggregated traffic matrix All communication channel packets To group The total flow It is a communication channel packet bandwidth capacity, This refers to the observation window duration. While the equation load = traffic / capacity is a fundamental logic in communications, this is based on a channel-level aggregated traffic matrix. The calculations are performed and explicitly defined as vectors for channel groups. By combining traffic, bandwidth, and observation time, the load of different groups is transformed into a unified metric for the proportion of bandwidth usage per unit time. For example, when... If group 1 , , ,but Group 2 , , ,but ,final This directly quantifies the actual load level of each group.

[0042] Observation window duration Defined as the duration of traffic observation, with a preferred value of 10 seconds. While time windows are a common method for traffic monitoring, the 10-second value is a specific choice for online inference scenarios of the MoE model: the decoding iteration of MoE inference is usually executed at the millisecond level. A 10-second window can cover a sufficient number of communication cycles to ensure that the collected traffic data is statistically representative, while preventing subsequent weight adjustments from lagging behind changes in network congestion due to an excessively long window. It is a specific parameter recommendation that balances statistical validity and real-time adjustment.

[0043] Normalized dominant congestion distribution vector The calculation formula is ,in It is the dominant congestion distribution vector. This indicates the absolute value operation. It is an all-1 vector. This vector transforms the dominant congestion distribution vector into a distribution form where the sum of its elements is 1, eliminating the influence of vector magnitude, so as to calculate the matching degree with the normalized channel load vector. For example, if After taking the absolute value, we get , ,but This is transformed into the congestion tendency distribution of each communication channel group.

[0044] Single-layer congestion pattern matching degree The calculation formula is ,in It is the dot product of the normalized channel load vector and the normalized dominant congestion distribution vector. It is the sum of all elements of the normalized channel load vector. Single-layer congestion pattern matching is not a simple cosine similarity, but a weighted projection ratio that quantifies the proportion of the current actual traffic load concentrated on the dominant congested communication channel packets. For example, when... , At that time, the molecule is The denominator is ,but This indicates that 74% of the load in this layer is concentrated on the dominant congested packets, which intuitively reflects the extent to which current traffic has fallen into a congestion trap.

[0045] Congestion pattern matching degree The calculation formula is ,in It is the total number of layers in the hybrid expert model. It is the first Single-layer congestion pattern matching degree. The congestion pattern matching degree is an aggregation strategy designed for the multi-layered structure of the MoE model: each layer of the MoE model includes two all-to-all communications: dispatch and restore. The congestion states of each layer are independent. By averaging the single-layer matching degrees of all layers, the congestion locking states of multiple layers can be aggregated into a global control scalar, reflecting the congestion pattern matching status of the entire model. For example, when the total number of layers in the MoE model is... Each floor When the average value is 0.6, This scalar will serve as the core basis for subsequently generating the congestion suppression coefficient and distribution smoothing parameters.

[0046] In a preferred embodiment, generating a congestion suppression coefficient and a distribution smoothing parameter based on the congestion pattern matching degree includes: Calculate the congestion suppression coefficient : ; in, The preset maximum suppression intensity, To adjust the slope, The congestion pattern matching degree, Set a preset matching threshold; Calculate the distribution smoothing parameter : ; in, For the minimum smoothing parameter, For the maximum smoothing parameter, is the growth rate constant.

[0047] It should be noted that the higher the congestion pattern matching degree, the more closely the current traffic matches the dominant congestion distribution. It is necessary to strengthen the suppression of experts corresponding to congested channels and improve the smoothness of the routing distribution. The sigmoid function can achieve a smooth change in the suppression intensity, and the exponential growth function can make the smoothing parameter monotonically approach the saturation value, avoiding parameter mutations that affect the stability of the model.

[0048] Specifically, the congestion suppression coefficient is first calculated. ,in The preset maximum suppression strength is preferably 0.8. This value is chosen based on the commonly used effective range of expert bias adjustment in hybrid expert model inference. To adjust the slope, its preferred value is 5, which allows the subsequent function to exhibit a suitable rate of change near the congestion pattern matching threshold; The matching degree of the previously determined congestion pattern; The preset matching threshold is preferably set to 0.3, a value derived from empirical observations showing that congestion patterns begin to become significant in actual deployments. The formula for calculating the congestion suppression coefficient is as follows: Through this formula, when Less than hour, It will remain at a lower value, when Exceed hour, Will follow The rise is smooth and approaches .

[0049] Next, calculate the distribution smoothing parameter. ,in The minimum smoothing parameter is preferably set to 1.0, which is the base temperature value for the route distribution in the hybrid expert model. The maximum smoothing parameter is preferably 3.0. The reason for this value is that if the smoothing parameter exceeds 3, the route distribution will be too scattered, which will affect the inference performance of the model. The growth rate constant is preferably 2, which allows for... Follow The increase in exhibits a stable growth trend. Based on these parameters, the formula for calculating the distribution smoothing parameter is: Through this formula, Will from At the beginning, with The increase is monotonically increasing and gradually approaches This enables gradual adjustments to ensure smooth routing distribution.

[0050] The formula for calculating the congestion suppression coefficient is as follows: ,in For maximum suppression intensity, To adjust the slope, For congestion pattern matching degree, A preset matching threshold is set. The Sigmoid function input is associated with the congestion matching degree. The output correlation is the suppression coefficient of MoE routing, which belongs to unique control logic: when hour, Maintaining it at a lower value results in weaker suppression of expert bias; when hour, Follow The rise is smooth and approaches To avoid mutations in suppression strength. For example, when hour, This enables smooth control that provides stronger suppression as congestion levels increase.

[0051] Maximum suppression strength The preferred value is 0.8. This value is based on the effective range setting of expert bias adjustment in deep learning model inference: if An excessively high value (e.g., exceeding 1.0) may cause the expert bias assigned to a high-congestion channel to be excessively reduced, resulting in a complete loss of the probability of being selected; if the value is too low (e.g., below 0.5), it will not effectively reduce the traffic load on high-congestion channels. A value of 0.8 is an empirical engineering value that balances the suppression effect with the expert participation, ensuring that while reducing congestion, experts still maintain a reasonable probability of being selected.

[0052] Adjusting the slope With threshold , The preferred value is 5. The preferred value is 0.3. It is an engineering criterion for judging severe congestion, that is, when the congestion pattern matches... When the value exceeds 0.3, it is considered that the current traffic is significantly concentrated in the dominant congestion group; Used to control the steepness of the Sigmoid function, making exist When the surrounding area changes, It can show a clear growth trend. For example, when hour, , Will from time Rapid rise enhances the responsiveness to inhibition.

[0053] Distribution smoothing parameter The calculation formula is ,in For the minimum smoothing parameter, For the maximum smoothing parameter, The growth rate constant is This represents the congestion pattern matching degree. This design, based on an exponential saturation growth function, maps the congestion state to the temperature parameter of Softmax: Follow The increase from Monotonic approximation This increases the entropy of the route distribution, thereby breaking the low-rank lock-in of traffic. For example, when When it approaches 1, near This reduces the differences in routing probabilities among different experts, and distributes traffic more evenly across communication channel packets.

[0054] Minimum / Maximum Smoothing Parameters , The preferred value is 1.0. The preferred value is 3.0. Wherein This is the standard temperature configuration for the Softmax function, where route distribution is entirely determined by expert scores; [Settings...] To balance inference performance and accuracy: if If the value exceeds 3.0, the routing distribution will be too uniform, causing the experts in the MoE model to lose their specialization and the inference accuracy to decrease. The value of 3.0 is the result of a trade-off between distributing traffic to reduce congestion and maintaining the specialization of experts.

[0055] growth rate constant The preferred value is 2. This parameter controls the distribution smoothing parameter. Growth rate: Too small will lead to Follow The growth is too slow, making it impossible to distribute traffic in a timely manner under high congestion conditions; Too large will lead to exist It approaches quickly when it is small This excessively disperses traffic during periods of low congestion. When the value is 2, about That will achieve This ensures a rapid response during periods of high congestion while avoiding excessive adjustments during periods of low congestion.

[0056] It should be noted that if existing technologies use a step function (such as...) hour Take directly This can lead to sudden changes in route suppression strength, causing oscillations in route distribution and affecting the stability of MoE inference. This scheme implements the congestion suppression coefficient using the Sigmoid function. The smoothing of the distribution parameters is achieved through an exponential function. The gradual increase in congestion allows the model to smoothly transition between non-congested and congested states, ensuring the effectiveness of congestion adjustment while avoiding inference fluctuations caused by routing abrupt changes.

[0057] In a preferred embodiment, adjusting the router's expert selection bias parameters based on the dominant congestion distribution vector using the congestion suppression coefficient includes: Define the congestion potential vector of communication channel packets. : ; in, This refers to the normalized dominant congestion distribution vector. Computation for logical expert nodes Experts choose the bias parameter increment : ; in, The congestion suppression coefficient is... For logic expert nodes Index of the communication channel group to which it belongs. For the congestion potential vector at index The component values ​​at that location.

[0058] It should be noted that the normalized dominant congestion distribution vector can already reflect the congestion tendency of each communication channel group. Using it directly as the congestion potential vector can make the potential components correspond to the congestion degree of the channel group. The expert selection bias parameter of the router affects the probability of the expert being selected. By combining the congestion suppression coefficient with the congestion potential component of the channel group to which the expert belongs, the bias of the expert belonging to the high congestion potential channel can be reduced, thereby reducing the probability of these experts being selected and reducing the traffic of the corresponding channel.

[0059] Specifically, first determine , This is the layer index for the hybrid expert model, and its value ranges from the number of layers contained in the model; For the first The normalized dominant congestion distribution vector of the layer is used to define the congestion potential vector of communication channel packets. Its calculation formula is , The elements correspond to the congestion potential of each communication channel group. The larger the element value, the stronger the congestion tendency of the corresponding channel group.

[0060] Next, determine , This is the index for the logical expert node, and its value ranges from 1 to... ( (The total number of logical expert nodes in the hybrid expert model). This is the congestion suppression coefficient; For logic expert nodes The index of the communication channel group to which it belongs, which is generated by the logical expert node. The communication channel group corresponding to the computing device where it resides is determined; The congestion potential vector In the index The component values ​​at each location correspond to the logical expert nodes. The congestion potential of the associated channel group. Based on these parameters, the congestion potential for logical expert nodes is calculated. Experts choose the bias parameter increment Its calculation formula is In this formula, the negative sign means that the expert corresponding to the channel group with higher congestion potential has a smaller increment of bias parameter, which reduces the probability that these experts will be selected by the router and reduces the traffic load of the corresponding communication channel group.

[0061] Congestion potential vector The calculation formula is ,in It is the normalized dominant congestion distribution vector. The congestion potential vector directly equates the abstract congestion distribution in network science to a potential field and applies it to the adjustment of neural network weights. Unlike the traditional scalar feedback where congestion equals delay, It is vector feedback, and each element This corresponds to the congestion potential of a packet in a communication channel; a larger value indicates a stronger congestion tendency for that packet. For example, when... When (grouping 3 communication channels), if ,but This indicates that the first channel group has the highest congestion potential, and subsequent weight adjustments will specifically reduce the probability of experts belonging to this group being selected.

[0062] Experts select bias parameter increments The calculation formula is ,in This refers to the previously generated congestion suppression coefficient. For logic expert nodes The corresponding communication channel packet index. The congestion potential vector In the index The component value at that point. Specifically, the negative sign has the effect of making the increment of the bias parameter of the expert belonging to the high-congestion potential channel group negative, that is, the bias parameter is reduced, thereby reducing the probability that these experts will be selected by the router. For example, when , , hour, ,expert The bias parameter will be reduced by 0.56, its weight in the route scoring will be reduced, and the probability of it being selected will decrease, thereby reducing the traffic load on the corresponding channel group.

[0063] Mapping Index Logic Through logical expert nodes Find the congestion potential vector using the corresponding communication channel packet index. The corresponding components Specifically, the first step is to identify the experts. The resident computing device is then identified through a device channel association matrix to determine the communication channel group to which it belongs, ultimately yielding... The value of this index allows us to extract the congestion potential component corresponding to that expert. For example, the expert... Residing in computing devices ,equipment Determine the communication channel group by using the device channel association matrix. ,but And then from Extract As a calculation The parameters ensure that the weight adjustment is aligned with the congested channel.

[0064] It should be noted that, through the congestion potential vector Quantify the congestion tendency of packets in each communication channel, and combine it with the congestion suppression coefficient. ,pass The calculation increments the expert selection bias parameter, reducing the bias parameter of experts belonging to high-congestion potential channel packets. This reduction directly lowers the weight of these experts in the router's scoring, thus decreasing their probability of being selected by the token. As the probability of experts corresponding to high-congestion channel packets decreases, the communication traffic received by that channel packet deterministically decreases, gradually releasing congestion eigenmode locking. This process does not directly intervene in network transmission but indirectly regulates network traffic distribution by adjusting model routing weights, ensuring both the continuity of model inference and precise optimization targeting the root causes of congestion.

[0065] In a preferred embodiment, the route score is non-linearly scaled using the distribution smoothing parameter to generate an updated route probability distribution that avoids critical network bottlenecks, including: Calculate the scaled route score : ; in, For the current input token, For routers targeting logical expert nodes The output is the raw value of the route score. The bias parameter increment is selected for the aforementioned expert. The distribution smoothing parameter; Generate updated route probability distribution : ; in, This represents the total number of logic expert nodes. It is an exponential function.

[0066] It should be noted that the original routing score output by the router has been combined with the congestion suppression bias increment and non-linearly scaled by the distribution smoothing parameter. This can adjust the relative difference between the scores of different experts, thereby changing the concentration of the route distribution. Then, exponential normalization is used to transform the scaled score into a probability distribution, so that the route selection conforms to the probability constraints while avoiding key network bottlenecks.

[0067] Specifically, first determine , The input token currently used for inference in the hybrid expert model; This is the layer index for the hybrid expert model, and its value ranges from the number of layers contained in the model; This is the index for the logical expert node, with a value range of 1 to... ( (Total number of logical expert nodes). For routers targeting logical expert nodes The output is the raw routing score, which reflects the router's rating of the expert. The initial degree of preference; Increment the previously calculated expert selection bias parameter; Using the previously generated distribution smoothing parameters, calculate the scaled route score. Its calculation formula is This formula is passed The results of the original scores and bias increments are scaled. The larger the value, the smaller the score difference between different experts, and the smoother the route distribution.

[0068] Then, an updated routing probability distribution is generated. ,in This represents the total number of logic expert nodes. This is an exponential function, its function is to map the scaled score values ​​to positive values. Based on these parameters, the calculation formula is: This formula sums and normalizes the index scores of all experts, ensuring that the sum of the routing probabilities corresponding to all experts is 1, thus satisfying the probability constraints of routing selection. This refers to the probability distribution of updated routes to avoid critical network bottlenecks.

[0069] Scaled route score The calculation formula is ,in It is a router for logical expert nodes. The output is the raw value of the route score. It is the increment of the expert-selected bias parameter calculated in the previous steps. It is a distribution smoothing parameter. Although adding bias and dividing by temperature exist individually in deep learning, this scheme combines the two and introduces a parameter based on network congestion potential. and based on congestion locking strength This forms a dynamic correction logic. For example, if experts... of , (Belongs to the high congestion channel group). (The value taken when the congestion lockout strength is high), then , both through This lowered the scoring basis for experts corresponding to high-congestion channels, and also through... It narrows the scoring gap between different experts, and its dual function guides traffic away from bottleneck channels.

[0070] Distribution smoothing parameter Its core principle is that the temperature coefficient affects the distribution entropy, and the distribution smoothing parameter is applied to avoid the network's key bottlenecks. The magnitude of the value directly determines the relative difference in route scores: The larger the value, the smaller the difference in scores among different experts after scaling, the smoother the routing distribution, and the more difficult it is for traffic to be concentrated in a few channel groups; The smaller the value, the more significant the score difference and the more concentrated the routing distribution. For example, if two experts give revised scores of 3 and 1 respectively, when... When scaled, the values ​​remain 3 and 1, showing a significant difference; when When the scaling factor is 1 and 0.33, the difference is greatly reduced. The traffic that was originally attracted by high-scoring experts will be partially distributed to low-scoring experts. These low-scoring experts may belong to different communication channel groups, thereby avoiding key bottlenecks.

[0071] Experts select bias parameter increments As the core input variable for calculating the scaled route score, The core function of the expert-selected bias parameter increment is to transform the network's congestion potential into a correction term for the model's routing score; its value is determined by the congestion suppression coefficient. Congestion potential components of the expert's channel group A joint decision, and always a non-positive value, ensures that the expert scores assigned to high-congestion potential channel groups are lowered. For example, when , hour, This value will be directly added to the expert's... Based on the original scores, a revised scoring basis was formed, which will be used for subsequent... The non-linear scaling provides an initial input with congestion feedback.

[0072] It should be noted that existing technologies typically aim for load balancing, that is, to distribute the load across computing devices or experts to a more even level, while also striving to avoid bottlenecks, that is, to actively divert traffic away from critical congested channels in the network. The composite correction is then transformed into an updated route probability distribution via the Softmax function. Ultimately, this goal will be achieved: Targeted measures to reduce the probability of experts being selected for high-congestion channels. Further dispersing traffic distribution, the two working together enable routing selection to reduce traffic flowing to critical bottleneck channels while adhering to probabilistic constraints (the sum of probabilities for all experts equals 1). For example, if a communication channel packet is a major bottleneck, its corresponding experts will... The score was lowered due to a negative result, and then... After scaling, the routing probability of these experts is greatly reduced, and more traffic will flow to experts belonging to non-bottleneck channel groups, thereby relieving congestion locking and improving overall inference performance.

[0073] like Figure 2 As shown, Figure 2 The deployment and adaptive congestion control process of the Hybrid Expert Model (MoE) are demonstrated: Input tokens are first passed to the MoE router, which then distributes them to logical expert nodes deployed on different computing devices. Communication between these nodes is accomplished through a switching network containing multiple communication channel packets. Simultaneously, the traffic acquisition module collects the communication traffic between devices and passes it to the congestion mode analysis module, which analyzes and obtains the dominant congestion distribution and matching degree. Subsequently, the adaptive control module generates updated routing probabilities based on this information through operations such as bias adjustment, congestion suppression, and scoring smoothing, and feeds them back to the router. This optimizes the distribution of communication traffic and allows route selection to proactively avoid key network bottlenecks.

[0074] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. An adaptive weighted language model adjustment method based on environmental feedback, characterized in that, include: Configure a topology mapping table between logical expert nodes in the hybrid expert model and the communication channel groups to which their computing devices belong; The collected inter-device communication traffic data is aggregated based on the communication channel groups, and combined with the channel bandwidth capacity to generate a dominant congestion distribution vector characterizing the distribution status of key network bottlenecks. The congestion pattern matching degree is determined based on the similarity between the current traffic load distribution and the dominant congestion distribution vector. Based on the congestion pattern matching degree, a congestion suppression coefficient and a distribution smoothing parameter are generated. The congestion suppression coefficient is used to adjust the expert selection bias parameter of the router according to the dominant congestion distribution vector, and the distribution smoothing parameter is used to non-linearly scale the route score value to generate an updated route probability distribution that avoids key network bottlenecks.

2. The adaptive weighted language model adjustment method based on environmental feedback according to claim 1, characterized in that, Configure a topology mapping table between logical expert nodes in the hybrid expert model and the communication channel groups to which their computing devices belong, including: Construct a device channel association matrix that indicates the affiliation between each computing device and each communication channel group, wherein if a specific computing device belongs to a specific communication channel group, it is marked as a valid value at the corresponding position in the device channel association matrix; otherwise, it is marked as an invalid value. In addition, an expert channel association matrix is ​​constructed to indicate the correspondence between each logical expert node and each communication channel group, wherein if the computing device where a particular logical expert node resides belongs to a particular communication channel group, then the corresponding position in the expert channel association matrix is ​​marked as a valid value.

3. The adaptive weighted language model adjustment method based on environmental feedback according to claim 1, characterized in that, The collected inter-device communication traffic data is aggregated based on the communication channel groups, and combined with the channel bandwidth capacity to generate a dominant congestion distribution vector characterizing the distribution status of key network bottlenecks, including: The collected device-level communication traffic matrix is ​​projected and transformed using the device channel correlation matrix to generate a channel-level aggregated traffic matrix that reflects the traffic transmission relationship between communication channel groups. Construct a capacity diagonal matrix based on the channel bandwidth capacity, and use the inverse square root matrix of the capacity diagonal matrix to perform bilateral scaling on the channel-level aggregated traffic matrix to generate a capacity-normalized traffic matrix. The capacity-normalized flow matrix is ​​feature-extracted using a power-law iteration algorithm. After a predetermined number of iterative multiplications and vector length normalization operations, the converged iterative vector is used as the dominant congestion distribution vector.

4. The adaptive weighted language model adjustment method based on environmental feedback according to claim 1, characterized in that, The congestion pattern matching degree is determined based on the similarity between the current traffic load distribution and the dominant congestion distribution vector, including: Calculate the total input traffic of each communication channel packet in the current communication cycle, and divide it by the product of the observation window duration and the channel bandwidth capacity to generate a normalized channel load vector; The absolute value of the dominant congestion distribution vector is normalized to generate a normalized dominant congestion distribution vector whose sum of elements is a unit value. Calculate the inner product of the normalized channel load vector and the normalized dominant congestion distribution vector, and divide it by the sum of all elements of the normalized channel load vector to obtain the single-layer congestion pattern matching degree. The congestion pattern matching degree is obtained by arithmetically averaging the single-layer congestion pattern matching degrees of all layers in the hybrid expert model.

5. The adaptive weighted language model adjustment method based on environmental feedback according to claim 1, characterized in that, Based on the congestion pattern matching degree, a congestion suppression coefficient and distribution smoothing parameters are generated, including: The congestion suppression coefficient is obtained by nonlinearly mapping the difference between the congestion pattern matching degree and the preset matching degree threshold using an S-shaped function, and multiplying the mapping result by the preset maximum suppression strength. The saturation growth ratio based on the congestion pattern matching degree is calculated using the exponential decay function. This saturation growth ratio is then applied to the difference range between the minimum smoothing parameter and the maximum smoothing parameter, and superimposed on the minimum smoothing parameter to obtain the distribution smoothing parameter.

6. The adaptive weighted language model adjustment method based on environmental feedback according to claim 1, characterized in that, Adjusting the expert selection bias parameters of the router based on the dominant congestion distribution vector using the congestion suppression coefficient includes: The normalized dominant congestion distribution vector is used as the potential energy vector characterizing the packet congestion potential of each communication channel. Determine the specific communication channel group to which the computing device where each logical expert node resides belongs; Extract the component values ​​corresponding to the specific communication channel group from the potential energy vector; Calculate the product of the negative value of the congestion suppression coefficient and the component value, and use this product as the increment value of the expert selection bias parameter for the logical expert node.

7. The adaptive weighted language model adjustment method based on environmental feedback according to claim 1, characterized in that, The route score is non-linearly scaled using the aforementioned distribution smoothing parameter to generate an updated route probability distribution that avoids critical network bottlenecks, including: The incremental value of the expert selection bias parameter for the logical expert node is added to the original route score value of the logical expert node output by the router to obtain the corrected route score value. Divide the corrected route score by the distribution smoothing parameter to obtain the scaled route score; Perform exponential normalization on the scaled route scores of all logical expert nodes, and use the result as the updated route probability distribution.

Images