An expert hybrid model cataloging expert node atomic routing method and system
By combining node directory classification and atomic-level directional routing, the problems of scheduling latency and high power consumption in expert hybrid models are solved, achieving efficient and low-power expert node scheduling, which is suitable for various model sizes and scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 陈立波
- Filing Date
- 2026-03-22
- Publication Date
- 2026-06-05
Abstract
Description
Technical Field
[0001] This invention relates to the field of underlying computing power scheduling technology for expert hybrid models (MoE), specifically to an atomic routing method and system for cataloged expert nodes in expert hybrid models, which falls under the category of native underlying hardware / software scheduling architecture technology for large models.
[0002] 2. Definition of terms The atomic-level directional routing described in this invention refers to a routing scheduling method that directly reaches the target expert node point-to-point based on mapping relationships without any software or hardware relay links; The intelligent fallback routing described in this invention refers to a local precise routing scheduling that is triggered only when atomic-level directional route matching fails, and is executed using a non-global traversal method, without performing global or equivalent global traversal retrieval. The directory nodes and structured directory forms described in this invention include at least one of single directory structure, multi-directory structure, and hierarchical nested structure. The number of directories and the structural form are not limited and can be dynamically adapted and adjusted according to the operational requirements of the expert hybrid model. Background Technology
[0003] In the process of computing power scheduling, existing expert hybrid models generally adopt global traversal retrieval or equivalent global traversal retrieval methods for the allocation and invocation of expert nodes. This results in technical problems such as high routing latency, large loss of ineffective computing power, and high overall power consumption. Existing optimization solutions in the industry only focus on adjusting the expert activation ratio and do not build an expert node catalog classification mechanism from the bottom layer. The routing scheduling still relies on traversal and investigation logic. At the same time, it lacks low-power targeted scheduling design, which cannot achieve efficient and low-power precise atomic routing and is difficult to adapt to the commercial deployment requirements of large-scale expert hybrid models. Summary of the Invention
[0004] This invention aims to solve the technical problems of high expert node scheduling traversal loss, large routing delay, and insufficient power consumption control in existing expert hybrid models, and provides an atomic routing method and system for cataloged expert nodes in expert hybrid models.
[0005] This invention adopts an underlying architecture that separates node directory classification from atomic routing execution. The node directory classification module completes the directory classification of expert nodes according to computing power call characteristics, establishes standardized mapping relationships, and realizes orderly partitioned management of expert nodes. The atomic routing execution module performs atomic-level directional routing based on mapping relationships. Global traversal and equivalent global traversal are not performed during the routing process, thereby reducing scheduling computational power consumption and routing latency from the root. The system supports multiple implementation forms and deployment methods, and has strong versatility and underlying compatibility.
[0006] This invention provides a native underlying scheduling technology solution for expert hybrid models. It is not limited to specific model sizes, the number of expert nodes, or business scenarios. Its scope of protection covers all scheduling implementation methods that employ expert node cataloging, atomic-level directional routing, and non-global or equivalent global traversal retrieval. Detailed Implementation
[0007] This system supports various implementation forms, including pure software, pure hardware, and hybrid software and hardware. It can be non-intrusively embedded into the kernel of expert hybrid models or deployed externally, achieving universal adaptability without modifying the top-level inference logic of the model.
[0008] The node directory classification module classifies each expert node in the expert hybrid model according to its computing power call characteristics. It adopts a structured directory format and establishes a mapping relationship between expert node characteristics and directory nodes. Directory nodes can be in the form of physical partitions or logical partitions.
[0009] This system preferably adopts a multi-directory classification architecture, which can achieve accurate classification based on the active calling status of expert nodes, and achieve optimal technical effects in atomic routing scheduling and overall power consumption control. The directory structure can be dynamically expanded or switched according to the running status.
[0010] The atomic routing execution module only receives expert hybrid model running instructions and directory mapping signals, and performs atomic-level directional routing based on the mapping relationship. It does not traverse non-target expert node directories, does not process redundant scheduling data, and does not perform global traversal retrieval or equivalent global traversal retrieval during the routing process.
[0011] Atomic-level directed routing is the primary execution path. When atomic-level directed routing fails to match, it automatically switches to intelligent fallback routing to complete backup scheduling. The backup routing process still does not perform global or equivalent global traversal retrieval.
[0012] This system can be adapted to expert hybrid models of various sizes, achieving low-latency and accurate scheduling during the model inference phase and targeted low-power management during the idle phase, thus meeting the full-scenario operation requirements of expert hybrid models.
[0013] 6. Technical Effects: When applied to various expert hybrid models, this system can completely eliminate the invalid computing power loss caused by global traversal retrieval in routing scheduling, and significantly reduce the scheduling delay of expert nodes. Directory-based classification mechanisms can significantly improve routing accuracy and reduce redundant scheduling overhead; atomic-level directional routing can further reduce power consumption and latency losses caused by transit links. Its multi-form implementation and multi-mode deployment characteristics ensure seamless compatibility between the system and various expert hybrid models, and it has stable underlying scheduling and adaptation capabilities.
[0014] 7. Conclusion on Technical Effectiveness 1. This invention reconstructs the scheduling logic of the expert hybrid model from the bottom layer by classifying expert nodes into a directory and implementing atomic-level directed routing, thereby solving the problems of wasted computing power and high power consumption caused by global traversal retrieval during the scheduling process.
[0015] 2. This invention can significantly reduce routing scheduling latency, improve the overall operating efficiency of expert hybrid models, and greatly reduce computing power deployment costs.
[0016] 3. This invention has strong universality and adaptability, and is compatible with various directory structures, implementation forms and deployment methods. It is the core foundational technology for the underlying scheduling of large-scale expert hybrid models.
[0017] 8. List of Existing Technologies Single point 1: Traditional MoE expert random scheduling architecture Core mechanism: Expert nodes are randomly activated based on input features, which requires traversing the candidate expert pool to complete the matching, and is a forced global traversal scheduling mechanism. Objective differences: No directory-based classification was used, no atomic-level directed routing was used, and global traversal retrieval was still performed. Single point 2: MoE scheduling scheme based on gating network Core mechanism: The computational power for allocating expert weights is calculated through a gating network. This involves multiple layers of weight calculation intermediaries, and it is still necessary to traverse all experts to calculate the weights. Objective differences: Relies on gating weight calculation, lacks fixed directory mapping, has multiple layers of transit, and does not abandon global traversal logic. Single point 3: General-purpose computing power load balancing scheduling system Core mechanism: Tasks are dynamically allocated based on computing power load, focusing only on load balancing and failing to perform feature-based categorization of expert nodes. Objective differences: It only implements load balancing, lacks expert node feature classification, and has no atomic-level directed routing design. Single point 4: Model parallel computing power scheduling technology Core mechanism: The model parallel tasks are divided according to hardware nodes, which is a hardware-level scheduling mechanism and does not perform software-level catalog management for expert nodes. Objective differences: It belongs to hardware parallel scheduling, lacks expert node catalog classification, and lacks atomic-level point-to-point routing.
Claims
1. A cataloged expert node atomic routing system of expert hybrid models, characterized by, It includes a node directory classification module and an atomic routing execution module; the node directory classification module classifies each expert node of the expert hybrid model into a directory based on the computing power call characteristics, and establishes a mapping relationship between expert node characteristics and directory nodes; The atomic routing execution module performs atomic-level directed routing to the target expert node based on the mapping relationship. During the routing process, no global traversal retrieval is performed, nor is an equivalent global traversal retrieval.
2. The system according to claim 1, characterized in that, The directory node can be in the form of a physical partition or a logical partition.
3. The system according to claim 1, characterized in that, The atomic-level directional routing is a point-to-point direct route without any software or hardware intermediaries.
4. The system according to claim 1, characterized in that, The catalog classification adopts a structured catalog format.
5. The system according to claim 1, characterized in that, The system can be implemented as pure software, pure hardware, or a hybrid of software and hardware.
6. The system according to claim 1, characterized in that, The system can be non-intrusively embedded into the expert hybrid model kernel or deployed externally.
7. The system according to claim 1, characterized in that, The atomic routing execution module only performs routing scheduling operations and does not participate in the core inference operations of the expert hybrid model.
8. The system according to claim 1, characterized in that, The atomic-level directed route is the primary execution path. When the atomic-level directed route fails to match, the intelligent fallback route is automatically triggered as the backup execution path. The backup route still does not perform global or equivalent global traversal retrieval.
9. An expert hybrid model cataloging expert node atomic routing method, characterized in that, include: (1) Classify each expert node in the expert hybrid model according to the computing power call characteristics, and establish the mapping relationship between the expert node characteristics and the directory nodes; (2) Based on the mapping relationship, perform atomic-level directional routing on the target expert node. During the routing process, do not perform global traversal retrieval and do not perform equivalent global traversal retrieval.
10. The method according to claim 9, characterized in that, The node cataloging and atomic routing operations are triggered in real time by the expert hybrid model's execution instructions.