A method and apparatus for load balancing of hybrid expert large models through computing power in an intelligent computing cloud platform.

Abstract

This invention provides a method and apparatus for load balancing of hybrid expert models in an intelligent computing cloud platform using computing power, comprising: Step S1, calculating overload persistence index data based on the queue length, average waiting time, and communication time of each expert model; Step S2, determining whether a fission trigger condition is met in response to the overload persistence index data exceeding a judgment threshold; Step S3, fissioning the parent expert model to generate multiple child expert models; Step S4, obtaining a first-level score, obtaining the matching degree of each child expert model, and obtaining the real-time load; Step S5, calculating a second-level score based on the real-time load, the first-level score, and the matching degree, and performing load balancing judgment based on the second-level score to determine the target child expert model; Step S6, reclaiming the child expert models that meet the reclamation conditions. This invention can solve the problem of imperfect load balancing strategies for hybrid expert models in intelligent computing center cloud platforms in the prior art.

Description

Technical Field

[0001] This invention relates to the fields of intelligent computing centers, smart computing centers, computing infrastructure, and intelligent cloud computing technologies, specifically to a method and apparatus for achieving load balancing of hybrid expert large models through computing power in an intelligent computing cloud platform. Background Technology

[0002] With the rapid development of artificial intelligence technology, "intelligent computing centers" and "smart computing centers" have emerged.

[0003] An "intelligent computing center" refers to a facility that provides the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models) by utilizing large-scale heterogeneous computing resources, including general-purpose and intelligent computing power. Intelligent computing centers encompass facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0004] "Intelligent computing center" includes, but is not limited to, "intelligent computing center".

[0005] "Intelligent computing center" or artificial intelligence computing center is a type of computing infrastructure that provides computing power services, data services, and algorithm services required for artificial intelligence applications, based on artificial intelligence theory and adopting artificial intelligence computing architecture.

[0006] "Computing power" is the core of "intelligent computing center" and "smart computing center". It is the ability of computer equipment or computing / data center to process parameters. It is the ability of computer hardware and software to work together to execute a certain computing requirement. It is the computing power to achieve the target result output by processing parameter data. It is a new type of productivity that integrates parameter computing power, network carrying capacity and data storage capacity. It mainly provides services to society through computing power infrastructure.

[0007] As large-scale models continue to exceed trillions of parameters, Mixture of Experts (MoE) models, with their sparse activation mechanism, have become the mainstream architecture for ultra-large-scale language models and multimodal models. However, existing MoE models are deployed on local workstations or small-scale clusters, requiring runtime access to computing power in intelligent computing centers. The performance and stability of the MoE architecture are highly dependent on the rationality of the routing strategy. If the router cannot efficiently, evenly, and semantically consistently distribute input tokens to each expert model, it can easily lead to a series of systemic problems, including but not limited to: some expert models becoming overloaded due to continuously receiving high-density similar inputs, resulting in computational bottlenecks and increased long-tail latency; some expert models remaining idle for extended periods, causing parameter redundancy and resource waste; frequent cross-device communication exacerbating network congestion; and, more seriously, expert models may lose their specialization capabilities during training due to skewed input distribution, resulting in "semantic collapse" or "functional degradation." At the time, simply summarizing the technical solution as "replicating an expert model when it is overloaded" as in existing technologies was rather one-sided. The following issues should be further addressed: First, how to distinguish between occasional congestion and hotspot expert models that are continuously and repeatedly absorbed by similar inputs; second, how to determine whether the hotspot expert model has the semantic basis to be split into multiple sub-expert models; third, how to perform semantic-oriented secondary traffic splitting after replication, rather than just average traffic splitting; and fourth, how to reclaim the copy while maintaining the model's capabilities and avoid long-term resource occupation after the hotspot subsides.

[0008] It is evident that since the emergence of intelligent computing centers, how to leverage their computing power to implement a robust load balancing strategy for hybrid expert large models has been a pressing challenge in this field. Summary of the Invention

[0009] This invention provides a method and apparatus for load balancing of hybrid expert large models in an intelligent computing cloud platform through computing power, in order to solve the problem of imperfect load balancing strategies for hybrid expert large models in intelligent computing center cloud platforms in the prior art.

[0010] To solve the above problems, the present invention is implemented as follows: In a first aspect, the present invention provides a method for load balancing of hybrid expert large models through computing power in an intelligent computing cloud platform, comprising: Step S1: Based on the queue length, average waiting time, and communication time of each expert model in the intelligent computing cloud platform, calculate the overload persistence index data of the expert model. Step S2: In response to the overload persistence index data of the expert model being greater than the judgment threshold, determine whether the expert model meets the fission triggering condition; Step S3: Determine the expert model that meets the fission triggering condition as the parent expert model to be fissioned, and perform fission on the parent expert model to generate multiple child expert models; Step S4: Obtain the first-level score of the parent expert model, obtain the matching degree between the corresponding token hidden layer and each of the multiple sub-expert models within the current statistical window, and obtain the real-time load of the multiple sub-expert models. Step S5: Calculate the secondary scores corresponding to each of the multiple sub-expert models based on the real-time load, the primary score, and the matching degree, and perform load balancing judgment on the multiple sub-expert models based on the secondary scores to determine the target sub-expert model to perform the current calculation; Step S6: Determine whether the multiple sub-expert models meet the recycling conditions, and recycle the sub-expert models that meet the recycling conditions.

[0011] In one embodiment, step S2 includes: Step S2.1: Cluster the set of hidden token representations corresponding to the expert model within the statistical window to generate multiple clusters; Step S2.2: Determine whether the expert model meets the fission triggering condition based on the cluster.

[0012] In one embodiment, step S2.2 includes: Step S2.2.1: Obtain the silhouette coefficients and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics among the multiple clusters; Step S2.2.2: Calculate the splittable index data of the expert model based on the silhouette coefficient and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics; Step S2.2.3: In response to the splittable index data being greater than the split threshold, determine that the expert model meets the fission triggering condition.

[0013] In one embodiment, step S2.2.3 includes: Step S2.2.3.1: Determine the repeated selection penalty data and continuous high-load window data of the expert model; Step S2.2.3.2: In response to the fact that the repeated selection penalty data is greater than the repeated penalty threshold, the continuous high load window data is greater than the window number threshold, and the splittable index data is greater than the split threshold, it is determined that the expert model satisfies the fission triggering condition.

[0014] In one embodiment, step S3 includes: Step S3.1: Determine the sub-model parameters of each of the multiple sub-expert models based on the model parameters of the expert model; Step S3.2: Construct the sub-expert model based on the sub-model parameters.

[0015] In one embodiment, step S3.1 includes: Step S3.1.1: Determine the sub-model parameters of each of the multiple sub-expert models based on their respective lightweight difference parameters and the model parameters.

[0016] In one embodiment, step S3.2 includes: Step S3.2.1: Cluster the set of hidden token representations of the expert model to generate multiple clusters; Step S3.2.2: Construct the sub-expert model based on the cluster and the sub-model parameters.

[0017] Step S3.2.2 includes: Step S3.2.2.1: Determine the sub-cluster mean vector and / or principal direction vector and / or statistical descriptive data based on the clusters; Step S3.2.2.2: Determine the semantic prototype data of the cluster based on the mean vector of the sub-clusters and / or the principal direction vector and / or the statistical description data; Step S3.2.2.3: Construct the sub-expert model based on the semantic prototype data and the sub-model parameters.

[0018] Step S5 includes: Step S5.1: Select the sub-expert model with the highest secondary score as the target sub-expert model.

[0019] The recycling conditions include at least one of the following: The penalty value for repeated selection of the multiple sub-expert models is less than the first decision threshold; The overload persistence index values ​​of the multiple sub-expert models are less than the second judgment threshold; The utilization rate of the multiple sub-expert models is less than the third judgment threshold in M ​​consecutive calculations of the expert model. The similarity value of the semantic prototype data of the multiple sub-expert models is greater than the fourth judgment threshold. The parameter differences among the multiple sub-expert models are less than the fifth judgment threshold; The time delay loss value generated after the recycling of the multiple sub-expert models is less than the sixth judgment threshold.

[0020] Secondly, the present invention also provides a device for achieving load balancing of hybrid expert large models through computing power in an intelligent computing cloud platform, comprising: The calculation module is used to calculate the overload persistence index data of the expert models based on their respective queue lengths, average waiting times, and communication times in the intelligent computing cloud platform. The judgment module is used to determine whether the expert model meets the fission triggering condition in response to the overload persistence index data of the expert model being greater than the judgment threshold. The fission module is used to determine the expert model that meets the fission triggering condition as the parent expert model to be fissed, and to fiss the parent expert model to generate multiple child expert models. The acquisition module is used to obtain the first-level score of the parent expert model, obtain the matching degree between the corresponding token hidden layer and each of the multiple sub-expert models in the current statistical window, and obtain the real-time load of the multiple sub-expert models. The matching module is used to calculate the secondary scores corresponding to each of the multiple sub-expert models based on the real-time load, the primary score, and the matching degree, and to perform load balancing judgment on the multiple sub-expert models based on the secondary scores, so as to determine the target sub-expert model to perform the current calculation. The recycling module determines whether the multiple sub-expert models meet the recycling conditions and recycles the sub-expert models that meet the recycling conditions.

[0021] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps in the method for load balancing of a hybrid expert large model by computing power in the intelligent computing cloud platform described in the first aspect above.

[0022] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the method for load balancing of a hybrid expert large model by computing power in the intelligent computing cloud platform described in the first aspect above.

[0023] Fifthly, the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the method for load balancing of a hybrid expert large model by computing power in an intelligent computing cloud platform as described in the first aspect above.

[0024] In this invention, step S1 is to calculate the overload persistence index data of each expert model based on the queue length, average waiting time, and communication time of each expert model in the intelligent computing cloud platform; step S2 is to determine whether the expert model meets the fission triggering condition in response to the overload persistence index data of the expert model being greater than the judgment threshold; step S3 is to determine the expert model that meets the fission triggering condition as the parent expert model to be fissed, and to fiss the parent expert model to generate multiple child expert models; step S4 is to obtain the first-level score of the parent expert model, and to obtain the matching degree between the corresponding token hidden layer in the current statistical window and each of the multiple child expert models, and to obtain the real-time load of the multiple child expert models; step S5 is to calculate the second-level score of each of the multiple child expert models based on the real-time load, the first-level score, and the matching degree, and to perform load balancing judgment on the multiple child expert models based on the second-level score to determine the target child expert model to execute the current calculation; step S6 is to determine whether the multiple child expert models meet the recycling condition, and to recycle the child expert models that meet the recycling condition. Thus, compared to the static approach of deploying large hybrid expert models locally or in small-scale clusters in traditional technologies, this invention targets the high-concurrency, multi-tenant, and resource-heterogeneous cloud-native environment of intelligent computing cloud platforms. These platforms, through kilo-level GPU clusters, high-speed interconnect networks, and cloud-native scheduling systems, provide MoE with an unprecedented computing foundation and elastic scalability. In this context, MoE's core advantages are fully realized: for each input token, only a small number of expert sub-networks are dynamically activated by the router to participate in computation, effectively controlling the computational overhead of a single inference or training iteration while the total number of parameters expands rapidly. This is not simply a matter of "copying" high-load expert models to achieve coarse-grained scaling. Instead, it constructs a complete closed-loop mechanism encompassing perception, decision-making, execution, feedback, and recycling. It not only determines "whether there is overload" but also deeply analyzes the entire dynamic evolution system of "why there is overload," "how to split," "how to allocate," and "when to recycle." This enables the intelligent computing cloud platform to alleviate load while improving model expressiveness, optimizing communication topology, ensuring service stability, and effectively controlling computing resource consumption. It significantly improves the efficiency, accuracy, and sustainable operation capability of hybrid expert models on the intelligent computing cloud platform. Attached Figure Description

[0025] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1This is a flowchart of a method for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform, provided by the present invention. Figure 2 This is a flowchart of another method provided by the present invention for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform; Figure 3 This is a flowchart of another method provided by the present invention for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform; Figure 4 This is a flowchart of another method provided by the present invention for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform. Figure 5 This is a structural diagram of a device for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform provided by the present invention; Figure 6 This is a structural diagram of an electronic device provided by the present invention. Detailed Implementation

[0027] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] The “computing power” mentioned in this invention refers to: the ability of computer equipment or computing / data center to process information; the ability of computer hardware and software to work together to perform a certain computing requirement; the computing power to achieve the target result output by processing information data; and a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity, mainly providing services to society through computing power infrastructure.

[0029] The "computational power" (CP) described in this invention refers to the ability of a data center server to process data and output results. It is a comprehensive indicator of a data center's computing power, encompassing general computing power, supercomputing power, and intelligent computing power. The commonly used unit of measurement is floating-point operations per second (FLOPS, 1 EFLOPS = 10^18 FLOPS), with higher values ​​indicating stronger overall computing power. It is estimated that 1 EFLOPS is approximately the computing power output of 5 Tianhe-2A supercomputers, 500,000 mainstream server CPUs, or 2 million mainstream laptops. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .

[0030] The "Network Power" (NP) mentioned in this invention refers to the performance of data transmission capability of computing facilities, which includes comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, and involves network transmission within and between data centers. It is a comprehensive indicator for measuring network transmission scheduling capability.

[0031] The "Storage Power" (SP) described in this invention refers to the comprehensive capabilities of a data center in four aspects: data storage capacity, performance, security and reliability, and green and low-carbon operation. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and internal storage devices within servers. The commonly used unit of measurement for storage capacity is exabytes (EB, 1EB = 2^60 bytes), while the commonly used unit of measurement for performance is the number of read / write operations per second (IOPS / TB). Disaster recovery ratio is an important indicator of security and reliability.

[0032] The "computing infrastructure" mentioned in this invention refers to a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity, enabling centralized computing, storage, transmission, and application of information.

[0033] The "new information infrastructure" mentioned in this invention refers to network infrastructure such as 5G networks, fiber optic broadband networks, backbone networks, international communication networks, and satellite internet; computing infrastructure such as data centers, general computing centers, intelligent computing centers, and supercomputing centers; and new technology facilities such as artificial intelligence, blockchain, and quantum computing.

[0034] The “computing power” mentioned in this invention includes: general computing power, intelligent computing power, and supercomputing power.

[0035] The "general computing power" mentioned in this invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0036] The "intelligent computing power" mentioned in this invention refers to: a computing platform deployed on a large scale based on dedicated chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and ASIC (Application Specific Integrated Circuit) for various artificial intelligence innovative applications, such as natural language processing and machine vision.

[0037] The “supercomputing power” mentioned in this invention refers to the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for computing in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0038] The "intelligent computing center" described in this invention refers to a facility that utilizes large-scale heterogeneous computing resources, including general-purpose computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.), to primarily provide the necessary computing power, data, and algorithms for artificial intelligence applications (such as the development, training, and inference of deep learning models). The intelligent computing center encompasses facilities, hardware, and software, and can provide full-stack capabilities from underlying computing power to top-level application enablement.

[0039] The "intelligent computing center cloud platform" mentioned in this invention, abbreviated as "intelligent computing cloud", refers to a cloud computing platform that integrates hardware and software resources based on an intelligent computing center.

[0040] The "intelligent computing center" mentioned in this invention includes, but is not limited to, "smart computing center".

[0041] The "intelligent computing center" mentioned in this invention, also known as an artificial intelligence computing center, is a type of computing infrastructure that provides computing power services, data services, and algorithm services required for artificial intelligence applications, based on artificial intelligence theory and adopting an artificial intelligence computing architecture.

[0042] The "computing center" mentioned in this invention refers to a facility that is mainly composed of infrastructure such as wind, thermal, hydro, and electricity, and IT hardware and software equipment, and has computing power, carrying capacity, and storage capacity, including general data centers, intelligent computing centers, supercomputing centers, etc.

[0043] The "supercomputing center" mentioned in this invention refers to a supercomputing data center, which is a data center based on supercomputers or large-scale computing clusters. It can provide large-scale computing, storage and network services and is widely used in aerospace, defense, oil exploration, climate modeling and genome sequencing and other application scenarios.

[0044] The “computing resources” mentioned in this invention refer to the technologies and facilities required for the development of the digital society that have the ability to compute, transmit, store and apply information, including but not limited to computing resources such as CPUs and GPUs, network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guaranteeing resources such as wind, fire, water and electricity.

[0045] The "large model" mentioned in this invention refers to a deep neural network model with a parameter count of billions or even trillions, mainly including ultra-large-scale language models (such as LLM), multimodal large models (integrating text, images, speech, etc.), and special large models for scientific computing or industry intelligence.

[0046] The "Hybrid Expert Large Model" described in this invention refers to an ultra-large-scale deep neural network model employing a Mixture of Experts (MoE) architecture, deployed and running on an intelligent computing cloud platform. It captures complex semantics and knowledge structures through hundreds of billions of parameters, while relying on dynamic routing and sparse activation mechanisms to control the computational cost of a single inference iteration. This provides powerful generalization and generation capabilities while adapting to the computing power and latency constraints of real-world production environments. Its core feature lies in dividing the model parameters into multiple relatively independent "expert networks" and using a learnable router to dynamically select and activate a small number of experts for each input token, thus keeping the computational cost of a single forward or backward propagation controllable even with a total number of parameters reaching hundreds of billions or even trillions. This architecture is widely used in current mainstream large language models and multimodal large models, deployed on intelligent computing cloud platforms, aiming to balance model capacity, expressive power, and inference efficiency, and improving semantic modeling capabilities for diverse tasks through the specialized division of labor among experts.

[0047] It is crucial to emphasize that the load balancing of large Mixture of Experts (MoE) models through intelligent computing cloud platforms is extremely important. The fundamental reason is that while the sparse activation mechanism of MoE decouples parameter size from computational cost, it also introduces a highly dynamic, non-uniform, and semantically driven computational load distribution. If this load imbalance cannot be effectively managed and controlled at the computing power level of the intelligent computing cloud platform, it will directly lead to system performance collapse, resource waste, training instability, and even model performance degradation. Specifically, this manifests in the following ways: 1. In existing technologies, "occasional high load" and "persistent semantic hotspots" are often confused.

[0048] However, in reality, some expert models, due to their expertise in handling specific semantic domains (such as "technical terms" and "code generation"), are repeatedly activated by similar inputs over a long period of time, forming structural hotspots.

[0049] Relying solely on algorithmic layer "punishment" Scoring alone cannot solve the fundamental problem. A smart computing cloud platform must provide elastically scalable computing containers to persistently reserve resources or deploy replicas of these high-value, hot-topic expert models, ensuring they receive sufficient computing bandwidth without sacrificing semantic matching and preventing them from becoming a system bottleneck.

[0050] 2. To achieve professionalization of computing power scheduling services.

[0051] In existing technologies, forcibly "driving" tokens away from overloaded expert models through routing bias may balance usage in the short term, but it may also misassign "financial text" to "sports expert models," damaging the specialized structure of the models and leading to a decrease in accuracy.

[0052] Intelligent computing cloud platforms can expand service capacity through computing power replication, rather than distorting routing logic. These platforms need to support semantically aware replication and placement of expert models, ensuring that replicas inherit the capabilities of their parent expert models and are deployed on low-latency devices, thereby improving throughput while maintaining natural popularity.

[0053] 3. The computing architecture needs to support functional differentiation.

[0054] Current replication schemes often treat replicas as undifferentiated "splitters," lacking the ability to explore the semantic separability within expert models (e.g., an "animal expert model" actually contains two subgroups: "pets" and "wild animals").

[0055] The intelligent computing cloud platform can provide fine-grained computing power units (such as a single card can host multiple light quantum expert models), and in conjunction with the semantic clustering signals of the model layer, it can automatically trigger expert model splitting during replication, splitting the original expert model into multiple specialized sub-expert models.

[0056] 4. Rely on intelligent computing cloud platform lifecycle management.

[0057] Current technologies prioritize "the timing of replication" while neglecting "subsequent governance." However, if copies persist for a long time, it will lead to the expansion of hybrid expert models, memory overflow, and training divergence.

[0058] Intelligent computing cloud platforms can provide dynamic route updates, parameter synchronization mechanisms, and heat monitoring and recycling. These capabilities cannot be achieved solely through model algorithms; they require infrastructure support from the intelligent computing cloud platform.

[0059] Please see Figure 1 , Figure 1 This is a flowchart of a method for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform, as provided by the present invention. Figure 1 As shown, the method includes: Step S1: Calculate the overload persistence index data of the expert models based on their respective queue lengths, average waiting times, and communication times in the intelligent computing cloud platform.

[0060] It should be noted that the overload persistence index data of the computational expert model is constructed from the runtime state of the intelligent computing cloud platform. It is a quantitative, dynamic, and time-aware sustainability monitoring index used to distinguish between instantaneous congestion and structural, persistent hotspots, providing a reliable decision-making basis for whether to trigger expert model fission, replication, or resource reallocation.

[0061] The queue length represents the number of tokens waiting to be processed by the expert model at any given time. It reflects the immediate load pressure on the expert model, with a high value indicating a request backlog.

[0062] The average waiting time is the average latency from when a token is enqueued to when calculation begins within a time window. It is used to measure the service response capability of the expert model, and a continuous increase indicates insufficient processing capacity.

[0063] Communication duration is the average communication overhead between the device containing the expert model and other devices due to routing across nodes.

[0064] In one possible implementation, the overload persistence index data can be calculated using the following formula.

[0065]

[0066] in, Indicates the queue length. Indicates a waiting or execution delay. This indicates communication overhead.

[0067] Based on this real-time load, overload sustainability metrics can be further defined:

[0068] when When the load remains above the threshold, it indicates that the target expert model is not experiencing a short-term load spike, but rather is in a state of congestion for an extended period across multiple windows.

[0069] The beneficial technical effects achieved by step S1 are as follows: Based on the queue length, average waiting time, and communication time of each expert model in the intelligent computing cloud platform, the overload persistence index data of the expert model is calculated, which provides a multi-faceted data foundation for the accuracy and effectiveness of subsequent decisions. It transforms the vague "expert model is very busy" into the clear "the expert model is under long-term structural overload, and the root cause of the problem can be attributed." Therefore, this step is not only a load monitoring method, but also the perceptual cornerstone for the intelligent computing cloud platform to achieve precise scheduling, stable service, efficient resource utilization, and autonomous evolution capabilities.

[0070] Step S2: In response to the overload persistence index data of the expert model being greater than the judgment threshold, determine whether the expert model meets the fission triggering condition.

[0071] In this invention, the expert model is determined to meet the fission triggering condition only when both of the following conditions are met simultaneously: The overload persistence index of the expert model has exceeded the preset threshold, indicating that it is facing long-term, structural load pressure, not instantaneous fluctuations.

[0072] If the overload persistence index data is greater than the judgment threshold, and the expert model satisfies this condition, then the expert model can be considered to have the potential basis to split into two or more specialized sub-expert models.

[0073] It should be noted that the fission triggering conditions are pre-designed and can be changed according to actual design needs; no restrictions are imposed here.

[0074] It should be noted that, in order to avoid "copying simply because one is busy," this invention further determines whether the tokens handled by the target expert model are separable in the semantic space. Let the target expert model be e. The set of hidden token representations within the statistical window is H. e ={h1, h2, ..., h m The intelligent computing cloud platform clusters this set and constructs semantically separable index data S based on at least one statistical measure, such as silhouette coefficient, inter-cluster distance to intra-cluster distance ratio, multimodality index, and covariance spectrum characteristics. solit (l, e) When this indicator is higher than the preset threshold, it can be considered that the hotspot expert model is simultaneously undertaking multiple separable sub-tasks and has the basis for performing expert model fission.

[0075] In another possible implementation, the fission trigger condition is dynamically adjusted. In the actual operation of ultra-large-scale hybrid expert models on intelligent computing cloud platforms, factors such as input data distribution, system resource status, business priorities, and model training stages all evolve over time. Using static, fixed fission thresholds and rules would be difficult to adapt to the complex and ever-changing production environment, potentially leading to resource waste, service instability, or model capability degradation. Fission trigger conditions are a key design principle for achieving efficient, robust, and sustainable MoE systems. For example, the judgment threshold in the fission trigger condition can be dynamically adjusted, and the fission trigger condition can also be dynamically adjusted based on context-aware combinations of conditions.

[0076] The beneficial technical effects achieved by step S2 are as follows: Step S2 does not simply replicate all high-load expert models, but rather identifies those expert models that have both continuous computational pressure and inherent semantic separability through multi-dimensional criteria. This ensures that the fission operation can both alleviate system bottlenecks and improve model expressiveness, avoiding resource waste and model redundancy caused by blind expansion.

[0077] Step S3: Determine the expert model that meets the fission triggering condition as the parent expert model to be fissed, and perform fission on the parent expert model to generate multiple child expert models.

[0078] Based on the semantic subgroup structure identified in the early stage, the parent expert model is split to generate multiple sub-expert models. The parent expert model is then subjected to supervised parameter differentiation and functional decoupling to generate several sub-expert models that perform their respective functions and focus their capabilities in the semantic space. This not only alleviates the load on the intelligent computing cloud platform, but also improves the overall expressive ability of the hybrid expert model.

[0079] In this invention, after confirming that the parent expert model meets the fission condition, the system freezes its parameters and extracts the set of hidden layer representations of tokens that have been processed in the past. Two or more semantic subgroups are identified through clustering, and the corresponding cluster centers are calculated.

[0080] Secondly, based on these semantic prototypes, the original parameters of the parent expert model are subjected to directional initialization perturbations to generate multiple sub-expert models. Various methods can be used to generate multiple sub-expert models, and no limitations are imposed here. For example: small gradient shifts guided by semantic prototypes can be superimposed on the weights of the parent expert model; a low-rank adaptation (LoRA) structure can be used to inject a trainable lightweight quantum network into each sub-expert model; or the output of each sub-cluster token can be knowledge distilled using the parent expert model to fine-tune specialized initial values.

[0081] Finally, a unique ID is assigned to each sub-expert model, it is registered with the router, and matching rules are established with the corresponding semantic prototype. The intelligent computing cloud platform simultaneously completes resource allocation and deployment optimization (such as placing semantically similar sub-expert models on the same computing node), ensuring that efficient, low-communication-overhead inference services can be immediately deployed after the fission. The entire process achieves capability decoupling and functional evolution from "one generalist" to "multiple specialists."

[0082] Compared to the crude scaling strategy of simply "copying" high-load expert models to offload requests in existing technologies, this invention achieves a fundamental shift from "homogeneous redundancy" to "heterogeneous specialization" by introducing a structured fission mechanism driven by semantic separability. This improvement avoids problems such as wasted GPU memory, increased communication overhead, and diluted model capabilities caused by blind copying. It not only effectively alleviates system overload but also enhances the model's ability to express and process multiple types of tasks by generating semantically focused sub-expert models. Therefore, this technical solution not only solves the load problem but also transforms resource investment into a substantial evolution of model intelligence, significantly outperforming the traditional one-sided approach of "copying simply because it's busy."

[0083] The beneficial technical effects achieved by step S3: Step S3 generates multiple functionally specialized and semantically focused sub-expert models by performing semantically driven structured fission on the parent expert model that meets the fission triggering conditions. This not only effectively alleviates the processing latency and queue congestion caused by hot expert models and significantly improves service stability and SLA guarantee level, but also avoids the waste of computing power and memory caused by traditional homogeneous replication, greatly improves the utilization efficiency of computing power resources, and provides a unified foundation for the capability evolution of hybrid expert large models and the elastic scaling of the system, thereby promoting the MoE architecture from passive expansion to an intelligent new paradigm of proactive collaborative evolution.

[0084] Step S4: Obtain the first-level score of the parent expert model, obtain the matching degree between the corresponding token hidden layer and each of the multiple sub-expert models within the current statistical window, and obtain the real-time load of the multiple sub-expert models.

[0085] In this invention, the intelligent computing cloud platform establishes a comprehensive evaluation mechanism that integrates semantic matching, historical preferences, and system load to achieve a smooth transition from "coarse-grained parent expert model routing" to "fine-grained child expert model allocation." This step provides a complete decision-making basis for subsequent dynamic secondary routing strategies by simultaneously collecting three types of key information.

[0086] It should be noted that the first-level score represents the original confidence level of the token to be processed assigned to the parent expert model before the split, reflecting its fit with the overall semantic domain of the parent expert model.

[0087] The Token Hidden Layer refers to the context-aware vector obtained after the current token passes through the front shared layer of the MoE model. It encodes the token's semantics, context, and task intent.

[0088] The matching score is a scalar value that quantifies the degree of similarity to the semantic type that a particular sub-expert model excels at handling. A higher matching score indicates that the sub-expert model is more suitable for processing this token.

[0089] The beneficial technical effects achieved in step S4 are as follows: By synchronously acquiring the first-level score of the parent expert model, the semantic matching degree between the current token hidden layer and each child expert model, and the real-time load of the child expert models, a multi-dimensional decision context that integrates the original routing intent, semantic affinity, and system state is constructed. This enables accurate, balanced, and low-latency second-level routing after the expert model fission: It ensures that the token is assigned to the semantically most suitable child expert model to maintain the performance of the hybrid expert model, while avoiding directing requests to overloaded nodes to ensure service stability, effectively preventing hotspot shifts, and improving the utilization rate of computing resources and the overall inference efficiency of the hybrid expert model.

[0090] Through deep collaboration between the intelligent computing cloud platform and the MoE model, this step achieves: semantic fidelity, load balancing, and smooth evolution.

[0091] Step S5: Calculate the secondary scores corresponding to each of the multiple sub-expert models based on real-time load, primary score and matching degree, and perform load balancing judgment on the multiple sub-expert models based on the secondary scores to determine the target sub-expert model to execute the current calculation.

[0092] It should be noted that this invention preferably employs a two-level routing mechanism, rather than distributing subsequent tokens evenly across all replicas. The first-level routing is still executed within the original expert model space to ensure compatibility with existing token-choice or Top-K routing. If a parent expert model has split, the system continues to execute second-level routing for tokens that were originally routed from the first-level model to that parent expert model.

[0093] For the j-th sub-expert model, its second-level score can be expressed as:

[0094] in, The first-level score is the result of correction to the parent expert model. This indicates the degree of matching between the current token's hidden layer representation and the semantic prototype of the sub-expert model. This represents the real-time load of the j-th sub-expert model. Finally, one or more sub-expert models with the highest scores are selected to perform the actual computation.

[0095] With this design, the newly added replicas are not indiscriminate parallel capacity buckets, but will gradually evolve into sub-expert models with different semantic biases, thereby achieving load distribution and specialization enhancement at the same time.

[0096] Step S5 is the core step in achieving refined and intelligent routing decisions after the expert model fission. Its goal is to dynamically select the most suitable execution unit from multiple newly generated sub-expert models to process the current token, balancing semantic accuracy, system load balancing, and the original routing intent. This step calculates a "secondary score" by fusing multi-source information and uses this score as a basis for load balancing judgment, thus completing a smooth transition from coarse-grained parent expert model allocation to fine-grained sub-expert model scheduling.

[0097] Specifically, the intelligent computing cloud platform constructs a weighted fusion function based on the first-level score (reflecting the original route confidence), the semantic matching degree between the token hidden layer and each sub-expert model, and the real-time load of each sub-expert model (such as queue length, waiting time, GPU utilization, etc.) to generate a second-level score corresponding to each sub-expert model.

[0098] Subsequently, the intelligent computing cloud platform performs load balancing judgment on all sub-expert models based on the secondary scores.

[0099] One possible approach is to select the sub-expert model with the highest score at the second level as the optimal model. This can be combined with a Top-k strategy (such as selecting the two highest-scoring sub-expert models and distributing them proportionally), or by introducing random perturbations to enhance the exploration capability. More importantly, this judgment process is adaptive: when the overall cluster resources are strained, it automatically reduces the load to prioritize load balancing; when the service is in a low-peak period, it increases the load to prioritize semantic accuracy.

[0100] Ultimately, the sub-expert model with the highest score is selected as the target sub-expert model to perform the current computation, receives the token, and completes forward inference. This mechanism ensures that the expert model system after the fission can leverage its specialization advantages without creating new performance bottlenecks due to uneven distribution.

[0101] The beneficial technical effects achieved in step S5 are as follows: By integrating the first-level score of the parent expert model, the semantic matching degree of the child expert model, and the real-time load, the intelligent computing cloud platform dynamically calculates the second-level score of each child expert model and makes intelligent load balancing decisions accordingly. This effectively avoids the generation of new hotspots while ensuring the semantic accuracy of the task. It not only significantly improves the rationality of the allocation of inference requests and the utilization rate of computing resources, but also reduces end-to-end latency and long-tail jitter, ensuring that the professional capabilities after the fission of expert models are truly transformed into highly stable, efficient, and accurate service capabilities.

[0102] Step S6: Determine whether multiple sub-expert models meet the recycling conditions, and recycle the sub-expert models that meet the recycling conditions.

[0103] In this invention, the key final stage of the intelligent computing cloud platform to achieve closed-loop autonomous management has the core objective of: after the expert model has completed its phased service mission, timely identify and safely reclaim redundant or inefficient sub-expert models to prevent the disorderly expansion of model size, continuous waste of resources, and loss of control over system complexity.

[0104] The intelligent computing cloud platform continuously monitors the running status of each sub-expert model and judges whether it meets the recycling conditions based on multi-dimensional indicators. Typical criteria include: (1) long-term low load - the queue length is close to zero and the GPU utilization is lower than the threshold in multiple consecutive statistical windows; (2) semantic degradation or overlap - the semantic distribution of the tokens it processes is highly overlapping with that of the sibling sub-expert model or the parent expert model (which can be measured by the silhouette coefficient or KL divergence); (3) performance benefit decay - the fission does not bring a significant decrease in latency or an increase in accuracy, or even the overall efficiency of the hybrid expert model decreases due to communication overhead; (4) lifecycle overdue - the preset survival time has been exceeded since its creation and there is no active traffic. When one or more of the above conditions are met at the same time, the sub-expert model is marked as "recyclable".

[0105] In one possible implementation, after determining that the recycling conditions are met, the intelligent computing cloud platform will not immediately destroy the sub-expert model, but will initiate a smooth shutdown process: first, it will be removed from the router's candidate list, and new requests will no longer be allocated to it; then, it will wait for the existing tasks in its current queue to be processed (i.e., the "drain" phase); finally, it will release the GPU memory, computing context, and network resources it occupies, and update the expert model registry and metadata. If similar semantic traffic surges again in the future, the intelligent computing cloud platform can also quickly rebuild it based on historical records, rather than permanently losing its capabilities.

[0106] In this invention, the recycling conditions include at least one of the following: The penalty value for repeated selection by multiple sub-expert models is less than the first decision threshold; The overload persistence index values ​​of multiple sub-expert models are less than the second judgment threshold; The utilization rate of the expert model in M ​​consecutive calculations of multiple sub-expert models is less than the third judgment threshold; The similarity value of the semantic prototype data of multiple sub-expert models is greater than the fourth judgment threshold; The parameter differences among multiple sub-expert models are less than the fifth judgment threshold; The time delay loss generated after the recycling of multiple sub-expert models is less than the sixth judgment threshold.

[0107] In one possible implementation, multiple sub-expert models can be weighted and merged according to utilization or contribution:

[0108] in, This represents the normalized merge weight of the j-th sub-expert model. After merging, the corresponding sub-expert model can be deleted to restore the parent expert model structure or a small number of active sub-expert models can be retained to continue running.

[0109] The beneficial technical effects achieved in step S6 are as follows: By recycling the sub-expert models, on the one hand, the parameter scale and deployment overhead of the MoE model are effectively controlled, avoiding a resource black hole that only increases and never decreases; on the other hand, the simplicity and high signal-to-noise ratio of the expert model system are maintained, ensuring that each active expert model has a clear functional value. More importantly, it gives the entire MoE architecture a "metabolism" capability similar to a biological system—it can grow and differentiate according to needs, and it can also actively streamline and optimize when the environment changes, thereby supporting the intelligent computing cloud platform to maintain high efficiency, stability and sustainable evolution in long-term operation.

[0110] In this invention, step S1 is to calculate the overload persistence index data of each expert model based on the queue length, average waiting time, and communication time of each expert model in the intelligent computing cloud platform; step S2 is to determine whether the expert model meets the fission triggering condition in response to the overload persistence index data of the expert model being greater than the judgment threshold; step S3 is to determine the expert model that meets the fission triggering condition as the parent expert model to be fissed, and to fiss the parent expert model to generate multiple child expert models; step S4 is to obtain the first-level score of the parent expert model, and to obtain the matching degree between the corresponding token hidden layer in the current statistical window and each of the multiple child expert models, and to obtain the real-time load of the multiple child expert models; step S5 is to calculate the second-level score of each of the multiple child expert models based on the real-time load, the first-level score, and the matching degree, and to perform load balancing judgment on the multiple child expert models based on the second-level score to determine the target child expert model to execute the current calculation; step S6 is to determine whether the multiple child expert models meet the recycling condition, and to recycle the child expert models that meet the recycling condition. In contrast to the traditional static approach of deploying large hybrid expert models locally or in small-scale clusters, this invention targets the high-concurrency, multi-tenant, and resource-heterogeneous cloud-native environment of intelligent computing cloud platforms. It goes beyond simply "copying" high-load expert models for coarse-grained scaling; instead, it constructs a complete closed-loop mechanism encompassing perception, decision-making, execution, feedback, and recycling. This mechanism not only determines "overload" but also deeply analyzes the entire dynamic evolutionary process, including "why overload," "how to fission," "how to allocate resources," and "when to reclaim resources." This allows the intelligent computing cloud platform to alleviate load while simultaneously improving model expressiveness, optimizing communication topology, ensuring service stability, and effectively controlling computing resource consumption. Consequently, it significantly enhances the efficiency, accuracy, and sustainable operation of large hybrid expert models on intelligent computing cloud platforms.

[0111] Figure 2 This is a flowchart of another method for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform provided by the present invention. In an embodiment of the present invention, step S2 includes: Step S2.1: Cluster the set of hidden token representations corresponding to the expert model within the statistical window to generate multiple clusters.

[0112] In this invention, the hidden layer representations of all input tokens received by the expert model within the current statistical window (e.g., the last second or several thousand tokens) are first collected and output through a shared front-end network, forming a high-dimensional vector set. These hidden layer vectors contain the contextual semantics, task intent, and domain features of the tokens, serving as the foundational data for semantic analysis.

[0113] Subsequently, the system performs clustering analysis on the hidden layer set. Various clustering methods can be used, such as K-means, Gaussian Mixture Model (GMM), or density-based DBSCAN. The goal of clustering is to identify whether multiple significantly separated semantic subgroups exist in the data. For example, an expert model originally labeled as belonging to the "technology" domain might naturally differentiate into two semantic clusters: "artificial intelligence" and "semiconductor hardware." By setting a reasonable number of clusters (usually for the initial attempt), the algorithm divides the token hidden layer representation set into several clusters, each representing a potential, independently modelable sub-task domain.

[0114] The clustering process not only outputs the cluster member affiliations but also generates a semantic prototype for each cluster, namely the cluster center vector, which can be regarded as a typical representation of the sub-semantic domain. These prototypes will be used in subsequent steps to calculate the matching degree between tokens and sub-expert models and guide the initialization direction of sub-expert model parameters. More importantly, the clustering results provide an objective basis for "whether fission is worthwhile": if all tokens are highly clustered in a single region (only one effective cluster), it indicates that although the workload is high, the semantics are homogeneous, and replication is sufficient; if there are two or more clearly separated and similarly sized clusters, it indicates that the expert model actually undertakes heterogeneous tasks and has the potential to achieve specialized division of labor through fission.

[0115] The beneficial technical effects achieved by step S2.1 are as follows: By clustering the set of hidden representations of tokens to generate multiple clusters, a deep insight from the load phenomenon to the semantic essence is achieved, avoiding blind fission based solely on resource indicators, and ensuring that each model structure evolution is based on the real data distribution pattern, providing an accurate data foundation for subsequent efficient, accurate, and meaningful expert model fission.

[0116] Step S2.2: Determine whether the expert model meets the fission triggering conditions based on the clustering cluster.

[0117] In this invention, step S2.2 includes: Step S2.2.1: Obtain the silhouette coefficients and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics among multiple clusters.

[0118] Step S2.2.2: Calculate the splittable index data of the expert model based on the silhouette coefficient and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics.

[0119] Step S2.2.3: In response to the splittable index data being greater than the split threshold, determine that the expert model meets the fission triggering condition.

[0120] In this invention, to avoid "copying simply because one is busy," the invention further determines whether the tokens received by the target expert model are separable in the semantic space. Let the target expert model be e. The set of hidden token representations within the statistical window is H. e ={h1, h2, ..., hm}. The intelligent computing cloud platform clusters this set and constructs semantically separable index data Ssolit(l, e) based on at least one statistical measure such as silhouette coefficient, inter-cluster distance to intra-cluster distance ratio, multimodality index, and covariance spectrum characteristics. When this indicator is higher than the preset threshold, it can be considered that the hotspot expert model is simultaneously undertaking multiple separable sub-tasks and has the basis for performing expert model fission.

[0121] It should be noted that the silhouette coefficient measures how closely each sample is clustered with its own cluster and how well it is separated from other clusters. The value ranges from (-1, 1) and the closer it is to 1, the better the clustering effect.

[0122] The inter-cluster distance to intra-cluster distance ratio is a quantitative metric for measuring clustering quality. It is used to determine whether data has naturally split into multiple separate but internally compact subgroups. It is crucial in expert model splitting decisions; only when this ratio is sufficiently large does it indicate that the requests processed by the expert model truly come from multiple different semantic domains, making it worthwhile to split into multiple specialized sub-expert models.

[0123] The multimodality index is used to determine whether the distribution of a set of data (such as the hidden representation of tokens processed by an expert model) has multiple obvious peaks (i.e., multiple high-density regions), thereby reflecting whether there are multiple different semantic patterns or subgroups inherently. In this invention, if the index exceeds a threshold, it indicates that the data is not a single semantic cluster, but naturally contains multiple separable semantic branches, providing an important basis for the fission of expert models.

[0124] In this invention, a weighted fusion function can be constructed using multidimensional quantitative indicators such as aggregatable contour coefficients, inter-cluster / intra-cluster distance ratios (reflecting the distinguishing strength of semantic subgroups), multimodality indicators, and covariance spectrum features, to generate unified splittable indicator data.

[0125] When the indicator exceeds the preset threshold, it is determined that the expert model has a clear, stable and meaningful multi-cluster structure in semantics, and meets the fission triggering conditions, thereby ensuring that the fission decision is not only based on load pressure, but also conforms to the real data distribution pattern.

[0126] The beneficial technical effects achieved in step S2.2.3 are as follows: By fusing multi-dimensional semantic structure indicators such as contour coefficients, inter-cluster / intra-cluster distance ratios, multimodality indicators, and covariance spectrum features, the divisibility of expert models is scientifically quantified, effectively avoiding the problem of blindly splitting based solely on load thresholds. This method ensures that splitting is triggered only when a high-load expert model is indeed processing multiple semantically clear and distributed subtasks, thereby significantly improving the effectiveness of splitting and the return on resource investment. It not only prevents the waste of memory and communication overhead caused by invalid replication, but also ensures the rationality of the specialized division of labor among sub-expert models, laying a key foundation for realizing a precise, efficient, and sustainable dynamic MoE architecture.

[0127] In another possible implementation, such as Figure 3 As shown, step S2.2.3 includes: Step S2.2.3.1: Determine the repeated selection penalty data and continuous high-load window data of the expert model.

[0128] It should be noted that the repeated selection penalty data refers to a dynamic penalty value applied by the system to expert models that have been selected multiple times recently during the hybrid expert model routing process, in order to avoid uneven load or overfitting caused by a particular expert model being selected continuously or frequently. This penalty is usually applied to the expert model score in the form of a decaying weight (e.g., subtracting a term related to historical selection frequency from the original score), thereby reducing the probability of it being selected again in subsequent routing decisions, promoting a more balanced distribution of requests among multiple expert models, and improving the overall robustness and resource utilization of the system.

[0129] In this invention, the repeated selection penalty data can be calculated using the following formula: Suppose that in the l-th layer, the e-th expert model receives 1 / t the number of tokens within the time window t. The total number of tokens in this window is The selection ratio of the expert model in this window can then be defined as:

[0130] Let the average selection ratio of all expert models in this layer within window t be:

[0131] in, This represents the number of expert models in layer l. To characterize whether a certain expert model is continuously and repeatedly selected over a period of time, this invention defines the repeated selection penalty data as:

[0132] Where W is the length of the sliding window, As a time decay weight; This represents the number of windows in the expert model that are continuously in the high-load set; This indicates the degree to which similar inputs repeatedly hit the expert model across multiple windows. This invention particularly emphasizes "persistence" and "repeatability," focusing not on whether the expert model is currently busy, but rather on whether the expert model is consistently and persistently attracted to one or more similar types of inputs.

[0133] Continuous high-load window data refers to the time-series statistics recorded in the intelligent computing cloud platform monitoring, showing that an expert model is continuously under high load (e.g., queue length, waiting latency, etc. repeatedly exceed thresholds) across multiple consecutive time windows. This data is used to distinguish between "instantaneous traffic spikes" and "long-term structural congestion." Only when the high load is continuous over time will deep governance operations such as fission be triggered, thereby avoiding unnecessary changes to the model structure due to occasional fluctuations.

[0134] In this invention, continuous high-load window data can be calculated using the following formula: In one possible implementation, the overload persistence index data can be calculated using the following formula.

[0135]

[0136] in, Indicates the queue length. Indicates a waiting or execution delay. This indicates communication overhead.

[0137] Based on this real-time load, overload sustainability metrics can be further defined:

[0138] when When the load remains above the threshold, it indicates that the target expert model is not experiencing a short-term load spike, but rather is in a state of congestion for an extended period across multiple windows.

[0139] Step S2.2.3.2: In response to the fact that the repeated selection penalty data is greater than the repeated penalty threshold, the continuous high load window data is greater than the window number threshold, and the splittable index data is greater than the split threshold, the expert model is determined to meet the fission triggering condition.

[0140] In this invention, a triple-condition joint judgment is used: the expert model not only needs load balancing intervention due to repeated selection (repeated selection penalty for exceeding data limits), but its high-load state also has temporal persistence (the number of consecutive high-load windows exceeds the limit), and the requests it processes are semantically separable (the separability index data meets the standard). Only then is the fission trigger condition finally confirmed. This mechanism effectively avoids invalid fission caused by misjudgment of a single index, ensuring that fission decisions have load necessity, temporal stability, and semantic rationality, thereby significantly improving the efficiency of computing resource investment, preventing disorderly expansion of the model, and ensuring that the sub-expert models after fission can truly undertake specialized tasks, achieving synergistic optimization of performance, accuracy, and system stability.

[0141] Figure 4 This is a flowchart of another method for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform provided by the present invention. In an embodiment of the present invention, step S3 includes: Step S3.1: Determine the sub-model parameters of each of the multiple sub-expert models based on the model parameters of the expert model.

[0142] In this invention, the sub-model parameters of multiple sub-expert models can be determined based on their respective lightweight difference parameters and model parameters.

[0143] When the parent expert model meets the fission triggering condition, the system generates m child expert models. In the simplest implementation, each child expert model can directly inherit the parameters of the parent expert model.

[0144] In a more preferred implementation, the parameters of the sub-expert model can be set as follows:

[0145] in, The lightweight difference parameter representing the j-th sub-expert model can take the form of low-rank increments, bias increments, gated increments, or locally trainable sub-modules. By inheriting the lightweight difference evolution of the master parameter, a clearer functional division can be gradually formed among different sub-expert models while maintaining simplicity of implementation.

[0146] Step S3.2: Construct a sub-expert model based on the sub-model parameters.

[0147] In this invention, step S3.2 includes: Step S3.2.1: Cluster the set of hidden token representations of the expert model to generate multiple clusters.

[0148] Step S3.2.2: Construct a sub-expert model based on clusters and sub-model parameters.

[0149] In this invention, step S3.2.2 includes: Step S3.2.2.1: Determine the sub-cluster mean vector and / or principal direction vector and / or statistical descriptive data based on the clustering clusters.

[0150] Step S3.2.2.2: Determine the semantic prototype data of the clusters based on the sub-cluster mean vector and / or principal direction vector and / or statistical description data.

[0151] It should be noted that the sub-cluster mean vector refers to the arithmetic mean of all hidden vectors in each sub-cluster (i.e., semantic subgroup) after clustering the hidden representations of tokens processed by the expert model. It represents the "central position" or "typical semantic prototype" of the sub-cluster in the high-dimensional semantic space and is often used to measure the semantic similarity between a new token and the sub-cluster (e.g., calculated using cosine similarity). It is also a key basis for subsequent initialization of sub-expert model parameters or construction of secondary routing matching functions.

[0152] The principal direction vector (PDR) typically refers to the direction of the first principal component (i.e., the eigenvector corresponding to the largest eigenvalue of the covariance matrix) obtained by performing principal component analysis (PCA) on the hidden layer data of a sub-cluster. It reveals the direction of the most significant semantic changes within that sub-cluster, reflecting the main variation patterns of that semantic subdomain. In expert model fission, the PDR can be used to guide the differential perturbation of sub-expert model parameters (e.g., fine-tuning weights along this direction), making the sub-expert model more sensitive to changes in its specific semantic dimensions, thereby enhancing its specialization capabilities.

[0153] Statistical descriptors are multi-dimensional quantitative summaries of the distribution characteristics of subcluster data, including but not limited to: the number of samples within a cluster, mean, variance, skewness, kurtosis, L2 norm mean, and effective dimensions (such as the trace or condition number of the covariance matrix). These descriptions are not only used to assess the size and stability of subclusters (e.g., excessively small clusters may be noisy), but also serve as auxiliary features for determining separability or managing the lifecycle of sub-expert models (e.g., excessive variance may indicate semantic confusion), providing the system with a more comprehensive data-driven decision-making basis.

[0154] Step S3.2.2.3: Construct a sub-expert model based on semantic prototype data and sub-model parameters.

[0155] In this invention, the hidden layer representations of tokens assigned to the parent expert model within the most recent windows are clustered to obtain m sub-cluster centers. Each sub-expert model is initially bound to a semantic prototype, which can be the sub-cluster mean vector, the principal direction vector, semantic prototype data generated from statistical descriptive data, or semantic prototype data generated by a small network.

[0156] In this invention, step S5 includes: Step S5.1: Select the sub-expert model with the highest second-level score as the target sub-expert model.

[0157] The intelligent computing cloud platform sorts all sub-expert models generated from the parent expert model in descending order based on their secondary scores calculated in step S5 (which integrates the original route confidence of the parent expert model, the semantic matching degree between the current token and the sub-expert model, and the real-time load status of the sub-expert model). The sub-expert model with the highest secondary score is selected as the target sub-expert model. This sorting process ensures that the sub-expert model with the best semantic fit and the most abundant current resources is ranked first, providing an ordered candidate set for subsequent selections.

[0158] In another possible implementation, the intelligent computing cloud platform may not be limited to selecting only the top-ranked sub-expert model, but may instead select the top N ranked models (e.g., N=2 or N=3) as the target sub-expert model candidate set. This design supports a Top-N routing strategy: requests can be dynamically distributed to multiple high-quality sub-expert models according to their scores, further improving throughput and fault tolerance.

[0159] Please see Figure 5 , Figure 5 This is a structural diagram of a device provided by the present invention that enables load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform. Figure 5 As shown, the device 500 for achieving load balancing of hybrid expert large models through computing power in the intelligent computing cloud platform includes: The calculation module 501 is used to calculate the overload persistence index data of the expert models based on the queue length, average waiting time and communication time of each expert model in the intelligent computing cloud platform.

[0160] The judgment module 502 is used to determine whether the expert model meets the fission triggering condition in response to the overload persistence index data of the expert model being greater than the judgment threshold.

[0161] The fission module 503 is used to determine the expert model that meets the fission triggering conditions as the parent expert model to be fissed, and to fiss the parent expert model to generate multiple child expert models.

[0162] The acquisition module 504 is used to obtain the first-level score of the parent expert model, the matching degree between the corresponding token hidden layer and multiple child expert models within the current statistical window, and the real-time load of multiple child expert models.

[0163] The matching module 505 is used to calculate the secondary scores corresponding to each of the multiple sub-expert models based on real-time load, primary score and matching degree, and to perform load balancing judgment on the multiple sub-expert models based on the secondary scores in order to determine the target sub-expert model to perform the current calculation.

[0164] The recycling module 506 determines whether multiple sub-expert models meet the recycling conditions and recycles the sub-expert models that meet the recycling conditions.

[0165] In one embodiment, the determination module 502 includes: Clustering units are used to cluster the set of hidden token representations corresponding to the expert model within the statistical window to generate multiple clusters.

[0166] The judgment unit is used to determine whether the expert model meets the fission triggering conditions based on the clustering cluster.

[0167] In one embodiment, the determining unit is further configured to: Obtain the silhouette coefficients and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics among multiple clusters; Based on the silhouette coefficient and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics, the splittable index data of the expert model are calculated. When the separable index data is greater than the separable threshold, the expert model is determined to meet the fission triggering condition.

[0168] In one embodiment, the determining unit is further configured to: Identify the repeated selection penalty data and continuous high-load window data for the expert model; In response to the fact that repeated selection penalty data is greater than the repeated penalty threshold, continuous high load window data is greater than the window number threshold, and splittable index data is greater than the split threshold, the expert model is determined to meet the fission triggering condition.

[0169] In one embodiment, the fission module 503 includes: The determination unit is used to determine the sub-model parameters of multiple sub-expert models based on the model parameters of the expert model; The building unit is used to construct sub-expert models based on the parameters of the sub-model.

[0170] In one embodiment, the determining unit is further configured to: The sub-model parameters of each of the multiple sub-expert models are determined based on their respective lightweight difference parameters and model parameters.

[0171] In one embodiment, the determining unit is further configured to: Cluster the set of hidden token representations of the expert model to generate multiple clusters; Sub-expert models are constructed based on clusters and sub-model parameters.

[0172] In one embodiment, the determining unit is further configured to: Based on cluster analysis, sub-cluster mean vectors and / or principal direction vectors and / or statistical descriptive data are determined. The semantic prototype data of the clusters are determined based on the sub-cluster mean vector and / or principal direction vector and / or statistical description data; Sub-expert models are constructed based on semantic prototype data and sub-model parameters.

[0173] In one embodiment, the matching module 505 includes: The sorting unit is used to sort multiple sub-expert models in order of their secondary scores from smallest to largest. The identification unit is used to select the top N sub-expert models as the target sub-expert models.

[0174] The device for achieving load balancing of hybrid expert large models through computing power in the intelligent computing cloud platform provided by the present invention is capable of realizing the various processes of the above-mentioned method for achieving load balancing of hybrid expert large models through computing power in the intelligent computing cloud platform. The technical features are one-to-one and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0175] It should be noted that the device for achieving load balancing of hybrid expert large models through computing power in the intelligent computing cloud platform of this invention can be a device, or a component, integrated circuit, or chip in an electronic device.

[0176] The present invention also provides an electronic device, see below. Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device includes a memory 601, a processor 602, and a program or instructions stored in the memory 601 that run on the processor 602. When the program or instructions are executed by the processor 602, they can achieve the following: Figure 1 The corresponding intelligent computing cloud platform achieves the same beneficial effect through any step in the method embodiment of load balancing of hybrid expert large models by computing power, and will not be elaborated here.

[0177] The processor 602 can be a CPU, ASIC, FPGA or GPU.

[0178] Those skilled in the art will understand that all or part of the steps of the above-described method embodiment for achieving load balancing of hybrid expert large models through computing power in the intelligent computing cloud platform can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0179] The present invention also provides a readable storage medium on which a computer program is stored, and which, when executed by a processor, can perform the above-described functions. Figure 1 The corresponding intelligent computing cloud platform achieves any step in the method embodiment of load balancing for hybrid expert large models through computing power, and can achieve the same technical effect. To avoid repetition, it will not be described again here. The storage medium mentioned is such as read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc.

[0180] The present invention also provides a computer program product, including computer instructions that, when executed by a processor, implement the above-described... Figure 1 The corresponding intelligent computing cloud platform achieves the same technical effect by implementing the method of load balancing of hybrid expert large models through computing power. Therefore, it will not be described in detail here to avoid repetition.

[0181] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or second terminal device, etc.) to execute the methods of the various embodiments of the present invention.

[0182] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A method for load balancing of a hybrid expert large model using computing power in an intelligent computing cloud platform, characterized in that, include: Step S1: Based on the queue length, average waiting time, and communication time of each expert model in the intelligent computing cloud platform, calculate the overload persistence index data of the expert model. Step S2: In response to the overload persistence index data of the expert model being greater than the judgment threshold, determine whether the expert model meets the fission triggering condition; Step S3: Determine the expert model that meets the fission triggering condition as the parent expert model to be fissioned, and perform fission on the parent expert model to generate multiple child expert models; Step S4: Obtain the first-level score of the parent expert model, obtain the matching degree between the corresponding token hidden layer and each of the multiple sub-expert models within the current statistical window, and obtain the real-time load of the multiple sub-expert models. Step S5: Calculate the secondary scores corresponding to each of the multiple sub-expert models based on the real-time load, the primary score, and the matching degree, and perform load balancing judgment on the multiple sub-expert models based on the secondary scores to determine the target sub-expert model to perform the current calculation; Step S6: Determine whether the multiple sub-expert models meet the recycling conditions, and recycle the sub-expert models that meet the recycling conditions.

2. The method according to claim 1, characterized in that, Step S2 includes: Step S2.1: Cluster the set of hidden token representations corresponding to the expert model within the statistical window to generate multiple clusters; Step S2.2: Determine whether the expert model meets the fission triggering condition based on the cluster.

3. The method according to claim 2, characterized in that, Step S2.2 includes: Step S2.2.1: Obtain the silhouette coefficients and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics among the multiple clusters; Step S2.2.2: Calculate the splittable index data of the expert model based on the silhouette coefficient and / or the ratio of inter-cluster distance to intra-cluster distance and / or the multimodality index and / or the covariance spectrum characteristics; Step S2.2.3: In response to the splittable index data being greater than the split threshold, determine that the expert model meets the fission triggering condition.

4. The method according to claim 3, characterized in that, Step S2.2.3 includes: Step S2.2.3.1: Determine the repeated selection penalty data and continuous high-load window data of the expert model; Step S2.2.3.2: In response to the fact that the repeated selection penalty data is greater than the repeated penalty threshold, the continuous high load window data is greater than the window number threshold, and the splittable index data is greater than the split threshold, it is determined that the expert model satisfies the fission triggering condition.

5. The method according to claim 1, characterized in that, Step S3 includes: Step S3.1: Determine the sub-model parameters of each of the multiple sub-expert models based on the model parameters of the expert model; Step S3.2: Construct the sub-expert model based on the sub-model parameters.

6. The method according to claim 5, characterized in that, Step S3.1 includes: Step S3.1.1: Determine the sub-model parameters of each of the multiple sub-expert models based on their respective lightweight difference parameters and the model parameters.

7. The method according to claim 5, characterized in that, Step S3.2 includes: Step S3.2.1: Cluster the set of hidden token representations of the expert model to generate multiple clusters; Step S3.2.2: Construct the sub-expert model based on the cluster and the sub-model parameters.

8. The method according to claim 7, characterized in that, Step S3.2.2 includes: Step S3.2.2.1: Determine the sub-cluster mean vector and / or principal direction vector and / or statistical descriptive data based on the clusters; Step S3.2.2.2: Determine the semantic prototype data of the cluster based on the mean vector of the sub-clusters and / or the principal direction vector and / or the statistical description data; Step S3.2.2.3: Construct the sub-expert model based on the semantic prototype data and the sub-model parameters.

9. The method according to claim 1, characterized in that, Step S5 includes: Step S5.1: Select the sub-expert model with the highest secondary score as the target sub-expert model.

10. The method according to claim 1, characterized in that, The recycling conditions include at least one of the following: The penalty value for repeated selection of the multiple sub-expert models is less than the first decision threshold; The overload persistence index values ​​of the multiple sub-expert models are less than the second judgment threshold; The utilization rate of the multiple sub-expert models is less than the third judgment threshold in M ​​consecutive calculations of the expert model. The similarity value of the semantic prototype data of the multiple sub-expert models is greater than the fourth judgment threshold. The parameter differences among the multiple sub-expert models are less than the fifth judgment threshold; The time delay loss value generated after the recycling of the multiple sub-expert models is less than the sixth judgment threshold.

11. A device for load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform, characterized in that, include: The calculation module is used to calculate the overload persistence index data of the expert models based on their respective queue lengths, average waiting times, and communication times in the intelligent computing cloud platform. The judgment module is used to determine whether the expert model meets the fission triggering condition in response to the overload persistence index data of the expert model being greater than the judgment threshold. The fission module is used to determine the expert model that meets the fission triggering condition as the parent expert model to be fissed, and to fiss the parent expert model to generate multiple child expert models. The acquisition module is used to obtain the first-level score of the parent expert model, obtain the matching degree between the corresponding token hidden layer and each of the multiple sub-expert models in the current statistical window, and obtain the real-time load of the multiple sub-expert models. The matching module is used to calculate the secondary scores corresponding to each of the multiple sub-expert models based on the real-time load, the primary score, and the matching degree, and to perform load balancing judgment on the multiple sub-expert models based on the secondary scores, so as to determine the target sub-expert model to perform the current calculation. The recycling module determines whether the multiple sub-expert models meet the recycling conditions and recycles the sub-expert models that meet the recycling conditions.

12. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when executed by the processor, the program implements the steps of the method for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, The method includes computer instructions that, when executed by a processor, implement the steps of the method for achieving load balancing of a hybrid expert large model through computing power in an intelligent computing cloud platform as described in any one of claims 1 to 10.

Images