A cross-expert continuous fusion execution method and device, a terminal and a medium

Abstract

The application discloses a cross-expert continuous fusion execution method and device, a terminal and a medium. The method comprises the following steps: performing routing solving, word statistics collection, continuous cache arrangement and index table construction operations on a word sequence to obtain routing weights, continuous word cache areas, expert offset tables and recovery index tables; based on the continuous word cache areas and the expert offset tables, input projection calculation of all activated experts is performed to obtain a continuous projection cache area arranged in sections; based on the expert offset tables and the continuous projection cache area, unit element boundary insertion and parallel prefix scanning are performed to obtain an effective scanning result sequence; and the effective scanning result sequence is taken as input, index inverse mapping is performed according to the recovery index table, and multi-expert output is weighted and aggregated by using the routing weights to obtain a target output sequence arranged in the order of the original word sequence. The application reduces operator overhead, updates the state in parallel and improves reasoning efficiency by proposing a cross-expert continuous fusion mechanism.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence deep learning technology, and in particular to a cross-expert continuous fusion execution method, apparatus, terminal and medium. Background Technology

[0002] With the rapid development of large language models, expanding model capacity and optimizing inference efficiency have become core research directions in the field of sequence modeling. Mixture of Experts (MoE) models, with their core mechanism of sparse activation, can effectively reduce computational overhead during inference while increasing model parameter capacity and enhancing feature learning capabilities. This overcomes the pain point of traditional dense models where capacity and computational cost are strongly tied. Currently, a series of mainstream high-performance models such as Mixtral-8x7B, Qwen3.5-MoE, and DeepSeek-MoE have emerged. The core of the MoE architecture is based on the SparseMoeBlock module to implement sparse computation. Through the Top-K gating mechanism of the Router network, a suitable subset of experts is selected for each text token. Multiple parallel MLP feedforward network experts complete feature calculations, and the results are aggregated and output based on routing weights. With its high computational efficiency, it is widely used in various large model pre-training and inference deployment scenarios. To further optimize sequence modeling efficiency and address the computational bottleneck of the quadratic complexity of traditional Transformer architectures, State Space Models (SSMs) have rapidly emerged as a sequence modeling architecture, with representative models including the Mamba series, S4, and S5. SSMs construct sequence dependencies through temporal recursive propagation of hidden states, achieving sequence modeling in linear time complexity. This not only endows the model with powerful long-sequence memory capabilities through its iterative update mechanism of hidden states but also reduces the computational overhead of long text processing compared to traditional dense models, making it a core technical solution to replace traditional MLP structures and optimize sequence modeling performance. To integrate the dual advantages of MoE sparse activation and SSM linear complexity, the industry has gradually conducted research on the fusion of these two architectures, proposing hybrid MoE-SSM architectures such as MoE-Mamba, BlackMamba, and Swimba. These architectures replace MLP experts in traditional MoE modules with SSM experts, enabling each expert to possess independent hidden states, aiming to achieve high-capacity, low-overhead, and high-efficiency sequence modeling. Currently, mainstream MoE optimization technologies are designed for traditional MLP expert architectures. Based on the core characteristics of MLP experts—independent tokens, natural parallelism, and statelessness—optimization strategies such as Grouped GEMM, Token Packing, and Expert Batching are used to achieve cross-expert matrix multiplication fusion and continuous memory layout. This effectively improves the inference speed and computing power utilization of traditional MoE models, fully meets the batch parallel computing needs of MLP experts, and has become a general acceleration solution for traditional MoE models.However, the existing MoE-SSM hybrid architecture still has inherent technical defects in practical applications. The temporal recursive dependency characteristics of SSM experts make it impossible to directly reuse the optimization schemes of traditional MoE, resulting in many unavoidable technical bottlenecks that severely restrict the inference performance and deployment effectiveness of the hybrid architecture. Specifically, firstly, there is a core bottleneck of serialized state updates. The hidden states of SSM experts have strict temporal dependencies. The current state needs to be updated iteratively based on the state of the previous state, forming a long-distance serial dependency chain. When multiple experts are activated simultaneously, state updates must be completed independently and serially, making batch parallel computation impossible. Furthermore, traditional TokenPacking optimization will cause cross-expert state pollution, which is a core technical challenge unique to the MoE-SSM architecture. Secondly, the existing solutions have extremely high operator scheduling overhead, requiring iterative traversal of multiple activated experts. The computation process of each SSM expert includes multiple independent operator calls and kernel startups. In multi-expert scenarios, the fixed scheduling overhead accumulates continuously, slowing down inference efficiency. Meanwhile, the model suffers from discontinuous memory access. Tokens are stored in a scattered manner according to their original sequence. Cross-expert computation requires frequent data collection through index gathers operations, resulting in low cache hit rate and poor memory access efficiency, which further exacerbates the model inference latency.

[0003] In summary, the existing MoE-SSM hybrid architecture is limited by the temporal recursive characteristics of SSM experts, making it unsuitable for traditional MoE optimization strategies. It suffers from multiple drawbacks, such as serialized state updates, high operator scheduling overhead, and low memory access efficiency. In particular, the unique serial dependency bottleneck limits the parallel computing capability of the model, making it impossible to fully leverage the dual advantages of MoE sparse activation and SSM linear modeling. This makes it difficult to meet the requirements for efficient, low-latency, and high-throughput inference deployment of large models.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a cross-expert continuous fusion execution method, device, terminal and medium to address the above-mentioned defects of the prior art. The aim is to solve the problem that the structural differences between SSM experts and MLP experts in the MoE-SSM architecture make it difficult to achieve cross-expert continuous execution, and the high overhead of operator calls and the serial execution of state update processes result in low model inference efficiency.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a cross-expert continuous fusion execution method, wherein the method includes: Perform routing, lexical statistics aggregation, continuous cache arrangement and index table construction operations on the lexical sequence to obtain the routing weight, continuous lexical cache area, expert offset table and recovery index table; Based on the continuous lexical buffer and the expert offset table, the input projection calculation of all activated experts is performed to obtain a segmented continuous projection buffer. Based on the expert offset table and the continuous projection buffer, identity boundary insertion and parallel prefix scanning are performed to obtain an effective scan result sequence. Using the effective scan result sequence as input, perform index inverse mapping according to the recovery index table, and use the routing weight to perform weighted aggregation of multiple expert outputs to obtain the target output sequence arranged in the order of the original word sequence.

[0007] In one implementation, the method further includes: Based on the expert offset table and the effective scan result sequence, the updated hidden state corresponding to each activated expert segment is located and extracted, and then overwritten to the state cache with the expert identifier as the key to obtain the updated state cache.

[0008] In one implementation, the steps of performing route solving, lexical statistical aggregation, continuous cache arrangement, and index table construction on the lexical sequence to obtain route weights, continuous lexical cache areas, expert offset tables, and restored index tables include: Based on the input word sequence, the routing probability of each word for each expert is calculated through the routing network, and the expert identifier list and corresponding routing weight of each word are determined according to the routing probability. The expert identifier list is traversed and counted to determine the number of lexical units corresponding to each activated expert, thus obtaining an expert lexical unit count array. Based on the expert lexical counting array, contiguous memory is allocated and the corresponding intervals of each expert are divided. Lexicals belonging to the same expert are copied to their respective intervals in the order of the original sequence, resulting in a continuous lexical cache area with segmented and continuous arrangement of expert lexicals. Based on the layout information of the expert lexical counting array and the continuous lexical cache area, an expert offset table and a recovery index table are generated.

[0009] In one implementation, the step of performing input projection calculations on all activated experts based on the continuous lexical buffer and the expert offset table to obtain a segmented, continuously arranged continuous projection buffer includes: Extract the input projection weight matrix of each activated expert from the expert parameter set to obtain a weight list arranged in expert order; Using a continuous term buffer as input data, the input data is grouped according to the expert offset table, and the weight list is used as the calculation parameter. The grouped general matrix multiplication operation is performed through a single kernel startup to complete the input projection calculation of all activated experts, resulting in a segmented and continuously arranged continuous projection buffer.

[0010] In one implementation, the step of performing identity boundary insertion and parallel prefix scanning based on the expert offset table and the continuous projection buffer to obtain an effective scan result sequence includes: Based on the expert offset table and the continuous projection buffer, identity element boundary insertion is performed to obtain an extended binary tuple sequence containing identity elements. Perform a parallel prefix scan on the extended binary sequence to obtain the scan result sequence; Based on the scan result sequence, all scan items caused by the insertion of identical placeholders are removed, and the prefix scan results corresponding to a preset number of valid word positions are extracted to obtain the valid scan result sequence.

[0011] In one implementation, the step of inserting identity elements based on the expert offset table and the continuous projection buffer to obtain an extended binary sequence containing identity elements includes: Traverse all lexical positions in the continuous projection buffer, calculate the state update tuple corresponding to each lexical position based on the parameters of the attribution expert state space model corresponding to each lexical position, and generate a tuple sequence composed of all state update tuples. By using an expert offset table to locate the boundaries of each segment of the binary sequence, and inserting identity elements between adjacent segments of the binary sequence, an extended binary sequence containing identity elements is obtained.

[0012] In one implementation, the step of taking the effective scan result sequence as input, performing index inverse mapping according to the restored index table, and using the routing weights to perform weighted aggregation of multiple expert outputs to obtain a target output sequence arranged in the original word sequence order includes: Using the valid scan result sequence as input, perform index inverse mapping according to the restored index table to inverse map the scan results of the continuous layout order back to the original word sequence order, and obtain the result sequence arranged in the original order; Based on the resulting sequence, the outputs of multiple experts are weighted and aggregated using the routing weights to obtain the target output sequence arranged in the order of the original word sequence.

[0013] Secondly, embodiments of the present invention also provide a cross-expert continuous fusion execution apparatus, wherein the apparatus includes: The module for obtaining route weights, continuous lexical cache, expert offset table, and recovery index table is used to perform route solving, lexical statistical aggregation, continuous cache arrangement, and index table construction operations on the lexical sequence to obtain route weights, continuous lexical cache, expert offset table, and recovery index table. The continuous projection buffer acquisition module is used to perform input projection calculations for all activated experts based on the continuous lexical buffer and the expert offset table, so as to obtain a segmented continuous projection buffer. The effective scan result sequence acquisition module is used to perform identity boundary insertion and parallel prefix scanning based on the expert offset table and the continuous projection buffer to obtain the effective scan result sequence; The target output sequence acquisition module is used to take the effective scan result sequence as input, perform index inverse mapping according to the recovery index table, and use the routing weight to perform weighted aggregation of multiple expert outputs to obtain the target output sequence arranged in the order of the original word sequence.

[0014] Thirdly, embodiments of the present invention also provide a terminal, wherein the terminal includes a memory, a processor, and a cross-expert continuous fusion execution program stored in the memory and executable on the processor, wherein when the processor executes the cross-expert continuous fusion execution program, it implements the steps of the cross-expert continuous fusion execution method described in any of the above schemes.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a cross-expert continuous fusion execution control program, and when the cross-expert continuous fusion execution program is executed by a processor, it implements the steps of the cross-expert continuous fusion execution method described in any of the above schemes.

[0016] Beneficial Effects: This invention provides a cross-expert continuous fusion execution method. Compared with existing technologies, this invention first performs routing solutions, lexical statistical aggregation, continuous cache arrangement, and index table construction operations on the lexical sequence to obtain routing weights, continuous lexical cache areas, expert offset tables, and restored index tables. This achieves continuous and structured storage of discrete lexical metadata from the data layer, improving the problems of disordered lexical storage, inefficient data retrieval, and ambiguous expert correspondences in traditional technologies. It provides a regular, unified, and quickly accessible data foundation and index support for all subsequent computational stages, avoiding data corruption at the source. The chaotic calculations lead to biases and resource waste. Next, based on the continuous lexical buffer and the expert offset table, the input projection calculation for all activated experts is performed, resulting in a segmented, continuously arranged continuous projection buffer. This step, based on pre-structured data, achieves batch and regularized calculation of activated expert input projections, abandoning the traditional fragmented calculation mode of per-lexical and per-expert. It effectively unifies the arrangement format of each expert's input data, reduces the fragmented overhead of projection calculations, improves the overall efficiency of multi-expert input projection calculations, and simultaneously ensures the integrity and accuracy of each activated expert's input data, thus facilitating subsequent scanning operations. The computational foundation is solidified; subsequently, based on the expert offset table and the continuous projection buffer, unit boundary insertion and parallel prefix scanning are performed to obtain an effective scan result sequence. By accurately defining the boundary range of each unit and expert calculation unit, parallel collaborative operation of multi-unit prefix scanning is achieved, breaking through the technical bottlenecks of low efficiency, ambiguous boundary recognition, and large amount of invalid computation in traditional serial scanning. This improves the parallelism and speed of data scanning processing, while accurately filtering and retaining effective data and eliminating invalid and redundant information, ensuring the effectiveness, accuracy, and completeness of the scan results; then, based on the effective... The scan result sequence is used as input. Based on the restored index table, index inverse mapping is performed, and the routing weights are used to perform weighted aggregation of the multi-expert outputs to obtain the target output sequence arranged in the original word sequence order. This step repairs the disordered word order in the previous segmented calculation and regularization process through index inverse mapping, ensuring the consistency of the output sequence with the original input sequence. At the same time, based on the accurate routing weights, the differentiated weighted fusion of the multi-expert outputs is completed, avoiding the problems of feature loss and insufficient output accuracy caused by traditional average aggregation and fixed weight aggregation, thus improving the accuracy and adaptability of the output results.Overall, this invention constructs a complete multi-expert lexical sequence processing system through end-to-end optimization, including pre-process data structuring and regularization, efficient parallel computing in the middle stage, and precise sorting and aggregation in the back end. It comprehensively optimizes the shortcomings of traditional technologies from multiple dimensions, including data storage, computational architecture, processing logic, and output accuracy. This effectively reduces computational redundancy and resource consumption in multi-expert model lexical processing, improves the parallel computing efficiency, data processing accuracy, and output stability of sequence processing, and achieves efficient regularization of discrete lexical data, collaborative parallel advancement of multi-expert computing, and accurate restoration and high-quality aggregation of output sequences. This comprehensively enhances the overall performance and application effect of multi-expert models in lexical sequence processing tasks. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a specific implementation of the cross-expert continuous fusion execution method provided in this embodiment of the invention.

[0018] Figure 2 This is a schematic diagram of the expert continuity layout in the cross-expert continuous fusion execution method provided in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the Segment boundary isolation state propagation in the cross-expert continuous fusion execution method provided in this embodiment of the invention.

[0020] Figure 4 This is a comparison diagram of the traditional solution and the cross-expert continuous fusion execution method of the present invention.

[0021] Figure 5 This is a schematic diagram of the cross-expert continuous fusion execution device provided in the embodiment of the present invention.

[0022] Figure 6 This is a structural diagram of a preferred embodiment of the cross-expert continuous fusion execution device provided in this invention.

[0023] Figure 7 This is a block diagram illustrating the internal structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0025] With the rapid development of large language models, expanding model capacity and optimizing inference efficiency have become core research directions in the field of sequence modeling. Mixture of Experts (MoE) models, with their core mechanism of sparse activation, can effectively reduce computational overhead during inference while increasing model parameter capacity and enhancing feature learning capabilities. This overcomes the pain point of traditional dense models where capacity and computational cost are strongly tied. Currently, a series of mainstream high-performance models such as Mixtral-8x7B, Qwen3.5-MoE, and DeepSeek-MoE have emerged. The core of the MoE architecture is based on the SparseMoeBlock module to implement sparse computation. Through the Top-K gating mechanism of the Router network, a suitable subset of experts is selected for each text token. Multiple parallel MLP feedforward network experts complete feature calculations, and the results are aggregated and output based on routing weights. With its high computational efficiency, it is widely used in various large model pre-training and inference deployment scenarios. To further optimize sequence modeling efficiency and address the computational bottleneck of the quadratic complexity of traditional Transformer architectures, State Space Models (SSMs) have rapidly emerged as a sequence modeling architecture, with representative models including the Mamba series, S4, and S5. SSMs construct sequence dependencies through temporal recursive propagation of hidden states, achieving sequence modeling in linear time complexity. This not only endows the model with powerful long-sequence memory capabilities through its iterative update mechanism of hidden states but also reduces the computational overhead of long text processing compared to traditional dense models, making it a core technical solution to replace traditional MLP structures and optimize sequence modeling performance. To integrate the dual advantages of MoE sparse activation and SSM linear complexity, the industry has gradually conducted research on the fusion of these two architectures, proposing hybrid MoE-SSM architectures such as MoE-Mamba, BlackMamba, and Swimba. These architectures replace MLP experts in traditional MoE modules with SSM experts, enabling each expert to possess independent hidden states, aiming to achieve high-capacity, low-overhead, and high-efficiency sequence modeling. Currently, mainstream MoE optimization technologies are designed for traditional MLP expert architectures. Based on the core characteristics of MLP experts—independent tokens, natural parallelism, and statelessness—optimization strategies such as Grouped GEMM, Token Packing, and Expert Batching are used to achieve cross-expert matrix multiplication fusion and continuous memory layout. This effectively improves the inference speed and computing power utilization of traditional MoE models, fully meets the batch parallel computing needs of MLP experts, and has become a general acceleration solution for traditional MoE models.However, the existing MoE-SSM hybrid architecture still has inherent technical defects in practical applications. The temporal recursive dependency characteristics of SSM experts make it impossible to directly reuse the optimization schemes of traditional MoE, resulting in many unavoidable technical bottlenecks that severely restrict the inference performance and deployment effectiveness of the hybrid architecture. Specifically, firstly, there is a core bottleneck of serialized state updates. The hidden states of SSM experts have strict temporal dependencies. The current state needs to be updated iteratively based on the state of the previous state, forming a long-distance serial dependency chain. When multiple experts are activated simultaneously, state updates must be completed independently and serially, making batch parallel computation impossible. Furthermore, traditional TokenPacking optimization will cause cross-expert state pollution, which is a core technical challenge unique to the MoE-SSM architecture. Secondly, the existing solutions have extremely high operator scheduling overhead, requiring iterative traversal of multiple activated experts. The computation process of each SSM expert includes multiple independent operator calls and kernel startups. In multi-expert scenarios, the fixed scheduling overhead accumulates continuously, slowing down inference efficiency. Meanwhile, the model suffers from discontinuous memory access. Tokens are stored in a scattered manner according to their original sequence. Cross-expert computation requires frequent data collection through index gathers operations, resulting in low cache hit rate and poor memory access efficiency, which further exacerbates the model inference latency.

[0026] In summary, the existing MoE-SSM hybrid architecture is limited by the temporal recursive characteristics of SSM experts, making it unsuitable for traditional MoE optimization strategies. It suffers from multiple drawbacks, such as serialized state updates, high operator scheduling overhead, and low memory access efficiency. In particular, the unique serial dependency bottleneck limits the parallel computing capability of the model, making it impossible to fully leverage the dual advantages of MoE sparse activation and SSM linear modeling. This makes it difficult to meet the requirements for efficient, low-latency, and high-throughput inference deployment of large models.

[0027] To address the aforementioned issues, this embodiment provides a cross-expert continuous fusion execution method. Specifically, this embodiment first performs routing calculations, lexical statistical aggregation, continuous cache arrangement, and index table construction operations on the lexical sequence to obtain routing weights, continuous lexical cache areas, expert offset tables, and a restored index table. This achieves continuous and structured storage of discrete lexical metadata from the data layer, improving upon the problems of disordered lexical storage, inefficient data retrieval, and ambiguous expert correspondences in traditional technologies. It provides a well-organized, unified, and quickly accessible data foundation and index support for all subsequent computational stages, thus avoiding problems at the source. This process avoids computational biases and resource waste caused by data disorder. Next, based on the continuous lexical buffer and the expert offset table, input projection calculations are performed for all activated experts, resulting in a segmented, continuously arranged continuous projection buffer. This step, based on pre-structured data, achieves batch and regularized calculations of activated expert input projections, abandoning the traditional fragmented calculation mode of per-lexical and per-expert. It effectively unifies the arrangement format of each expert's input data, reduces the fragmented overhead of projection calculations, improves the overall efficiency of multi-expert input projection calculations, and simultaneously ensures the integrity and accuracy of each activated expert's input data, thus providing a basis for subsequent scanning. The initial mapping operation lays a solid computational foundation. Subsequently, based on the expert offset table and the continuous projection buffer, unit boundary insertion and parallel prefix scanning are performed to obtain an effective scan result sequence. By accurately defining the boundary range of each unit and expert calculation unit, parallel collaborative operation of multi-unit prefix scanning is achieved, breaking through the technical bottlenecks of low efficiency, ambiguous boundary recognition, and large amount of invalid computation in traditional serial scanning. This improves the parallelism and computation speed of data scanning processing, while accurately filtering and retaining effective data and eliminating invalid and redundant information, ensuring the effectiveness, accuracy, and completeness of the scan results. Then, based on the existing... The effective scan result sequence is used as input. The index inverse mapping is performed according to the restored index table, and the multi-expert outputs are weighted and aggregated using the routing weights to obtain the target output sequence arranged in the original word sequence order. This step repairs the disordered word order in the previous segmented calculation and regularization process through index inverse mapping, ensuring the consistency of the output sequence with the original input sequence. At the same time, the differentiated weighted fusion of multi-expert outputs is completed based on accurate routing weights, avoiding the problems of feature loss and insufficient output accuracy caused by traditional average aggregation and fixed weight aggregation, thus improving the accuracy and adaptability of the output results.Overall, this invention constructs a complete multi-expert lexical sequence processing system through end-to-end optimization, including pre-process data structuring and regularization, efficient parallel computing in the middle stage, and precise sorting and aggregation in the back end. It comprehensively optimizes the shortcomings of traditional technologies from multiple dimensions, including data storage, computational architecture, processing logic, and output accuracy. This effectively reduces computational redundancy and resource consumption in multi-expert model lexical processing, improves the parallel computing efficiency, data processing accuracy, and output stability of sequence processing, and achieves efficient regularization of discrete lexical data, collaborative parallel advancement of multi-expert computing, and accurate restoration and high-quality aggregation of output sequences. This comprehensively enhances the overall performance and application effect of multi-expert models in lexical sequence processing tasks.

[0028] The cross-expert continuous fusion execution method provided in this embodiment can be applied to smart terminals, such as... Figure 1 As shown, the specific steps include the following: Step S100: Perform route solving, lexical statistical aggregation, continuous cache arrangement and index table construction operations on the lexical sequence to obtain the route weight, continuous lexical cache area, expert offset table and recovery index table.

[0029] In this embodiment, firstly, a routing solution operation is performed on the word sequence to calculate the distribution weight corresponding to each word, providing a core basis for subsequent word classification and traffic matching, ensuring the rationality and accuracy of word allocation. Next, word statistical aggregation is performed, which batch sorts, classifies, summarizes, and regularizes the disordered and scattered original word sequences, effectively eliminating redundant and scattered data, clarifying the distribution patterns and quantitative characteristics of words, and laying a regular data foundation for subsequent cache layout and index construction. Then, a continuous cache layout operation is performed, organizing the statistically aggregated words according to continuous rules for cache layout, overcoming the drawbacks of scattered word storage, achieving continuous and efficient storage of word data, improving the speed of subsequent word reading and retrieval, and reducing data access latency. Finally, the index table construction is completed, simultaneously generating an expert offset table and a recovery index table, which can record the storage location, offset information, and corresponding mapping relationships of various words, providing index support for the rapid location, retrieval, and restoration of word data. After the entire set of operations is completed, the final output consists of four core data categories: routing weights, continuous lexical cache, expert offset table, and recovery index table. Overall, it realizes intelligent distribution and matching of lexical sequences, standardized data organization, efficient caching and storage, and accurate index positioning. It optimizes the efficiency of lexical processing throughout the entire process from data allocation and storage layout to query and restoration, ensuring the stability and efficiency of subsequent model lexical calls and inference operations.

[0030] Specifically, step S100 includes the following steps: Step S101: Based on the input word sequence, calculate the routing probability of each word for each expert through the routing network, and determine the expert identifier list and corresponding routing weight for each word according to the routing probability. Step S102: Traverse and count the expert identifier list to determine the number of tokens corresponding to each activated expert, and obtain the expert token count array; Step S103: Allocate contiguous memory according to the expert word count array and divide the corresponding intervals of each expert. Copy the words belonging to the same expert to their respective intervals in the order of the original sequence to obtain a continuous word cache area in which the words of each expert are arranged in segments. Step S104: Based on the layout information of the expert lexical counting array and the continuous lexical cache area, generate an expert offset table and a recovery index table.

[0031] In one implementation, expert-aware reordering is used to reorganize the scattered tokens output by the routing network into a continuous layout. Simultaneously, it constructs auxiliary data tables required for subsequent computational stages, fundamentally eliminating the discrete distribution of tokens along the expert dimension, improving cache hit rate, and establishing a unified and standardized data layout for cross-expert batch computation. The overall process consists of four coherent sub-steps, and the specific implementation process is as follows: First, perform routing computation operations, defining the input token sequence as... After the sequence is processed by a routing network involving linear transformation and Softmax activation, the routing probability of each expert corresponding to each Token can be calculated. Then, the set of activated experts and the matching routing weights corresponding to each Token are determined by a Top-K selection method. Finally, the output includes a list of expert IDs (expert identifier list) and a routing weight matrix containing each Token, providing core input data for the subsequent Token grouping and statistics step. After completing the routing calculation, Token grouping and statistics are performed. By traversing the expert ID list output in the previous steps, the number of Tokens corresponding to each activated expert is counted one by one. The process involves generating a corresponding expert token count array, which reflects the token carrying capacity of each expert, providing accurate data for the subsequent allocation of capacity in the contiguous memory buffer. Next, a contiguous buffer (cache) construction operation is performed. Based on the obtained expert token count array, contiguous memory buffer space is allocated, and dedicated memory intervals are divided according to the number of tokens for each expert. Then, following the original input sequence, all tokens belonging to the same expert are copied sequentially to their corresponding dedicated memory intervals. After copying, a contiguous token buffer (contiguous lexical buffer) with segmented and continuous expert tokens is obtained, solving the problem of scattered token distribution. This contiguous token buffer can be directly used as input data for the next stage of computation. Finally, auxiliary tables are generated. Combining the expert token count array and the overall layout information of the continuous token buffer, two core auxiliary data tables supporting subsequent multi-stage operations are generated: the expert offset table and the recovery index table. The expert offset table has a length of E+1 and is used to record the start and end indices of the token segment corresponding to each expert in the continuous buffer, which can provide position index support for subsequent stages of operation. The recovery index table has a length consistent with the total length N of the input token sequence and is used to record the original token sequence position corresponding to each position in the rearranged buffer, which can provide a precise mapping basis for the sequence recovery operation in subsequent stages.

[0032] In one implementation, this embodiment provides a schematic diagram of the expert continuity layout in the cross-expert continuous fusion execution method, such as... Figure 2As shown, the initial distributed layout contains four tokens: T0, T1, T2, and T3. T0 is assigned to expert 2, T1 to expert 0, T2 to expert 1, and T3 to expert 0. This results in tokens belonging to the same expert being stored in a distributed manner, leading to a low cache hit rate. The tokens are reorganized and optimized through expert-aware rearrangement S1. During the process, an expert offset table is constructed, which records three types of information in sequence: expert number, token range, and offset. Expert 0 corresponds to the token range [0,2] and offset 0, expert 1 corresponds to the token range [2,3] and offset 2, and expert 2 corresponds to the token range [3,4] and offset 3. An end marker End is set to correspond to offset 4. This expert offset table can support O(1) range queries, allowing downstream modules to quickly locate the boundaries of the token segments corresponding to each expert. After the tokens are reorganized according to the expert offset table, a continuous layout is obtained. After reorganization, T1 and T3 are assigned to expert 0 region, T2 to expert 1 region, and T0 to expert 2 region, achieving continuous storage of tokens belonging to the same expert and improving cache hit rate. A recovery index table is generated simultaneously with the reorganization operation. The recovery index table records the mapping relationship between the original position of the token and its reorganized position. In the example, T0 with original position pos=0 corresponds to expert 2 region with reorganized position pos=3, and T1 with original position pos=1 corresponds to expert 0 region with reorganized position pos=0. The recovery index table allows the output result to be mapped back to the original token order after the calculation, achieving lossless restoration of sequence information. The prerequisite for cross-expert continuous execution is that the tokens are arranged continuously according to the experts. This embodiment can reorganize scattered tokens into a continuous layout with a single reorganization, simultaneously generating two auxiliary data structures: the expert offset table and the recovery index table. The expert offset table provides segment boundary information for downstream modules, and the recovery index table ensures that the output sequence can be restored to its initial order. The entire process improves token access characteristics while completely preserving the original input sequence order information.

[0033] Step S200: Based on the continuous lexical buffer and the expert offset table, perform input projection calculation for all activated experts to obtain a segmented continuous projection buffer.

[0034] In this embodiment, input projection calculations for all activated experts are performed based on the constructed continuous lexical buffer and expert offset table. The continuous lexical buffer can completely retain the input data of the lexical to be calculated, ensuring that the input information is complete, continuous, and ordered, avoiding computational gaps and information loss caused by the discrete arrangement of lexical data. At the same time, the expert offset table can locate the lexical data range and offset position corresponding to each activated expert, providing accurate data indexing basis for multi-expert parallel projection calculations, effectively avoiding the problems of expert data mismatch and calculation misalignment. After the input projection operation is completed based on the collaborative support of the two types of data structures, a segmented continuous projection buffer is finally generated, allowing the feature data after projection transformation to be stored in an orderly manner according to the corresponding segmentation rules, realizing the continuous and structured arrangement of projection features. The overall process completes the input feature projection transformation of all activated experts by binding lexical cache data and expert offset index. This not only improves the accuracy and orderliness of input projection calculation in multi-expert mode and avoids redundant errors caused by discrete data calculation, but also organizes the storage structure of output features, providing regular, continuous and efficiently callable projection feature data for subsequent deep model operations, effectively improving the overall model's computational efficiency and operational stability.

[0035] Specifically, step S200 includes the following steps: Step S201: Based on the continuous lexical buffer and the expert offset table, perform input projection calculation for all activated experts to obtain a segmented continuous projection buffer. Step S202: Using the continuous lexical buffer as input data, group the input data according to the expert offset table, and use the weight list as calculation parameters. Perform the grouped general matrix multiplication operation through a single kernel startup to complete the input projection calculation of all activated experts and obtain the segmented continuous projection buffer.

[0036] In one implementation, input projection calculations for all activated experts are performed based on the obtained continuous lexical buffer and expert offset table. This yields a segmented, continuously arranged continuous projection buffer. The overall process integrates the projection calculations of multiple experts into a unified batch operation by standardizing the data layout and merging calculation tasks, enabling cross-expert fusion projection and improving computational efficiency. The specific execution process includes two stages: The first stage is the projection weight organization stage. This step mainly involves organizing and standardizing the calculation parameters of each activated expert, providing standardized parameter support for batch calculations. Specifically, it extracts the input projection weight matrix corresponding to all activated experts from the expert parameter set, adapting to expert weight matrices of different dimensions to effectively support heterogeneous expert calculation scenarios. Then, it organizes and generates a weight list according to a preset expert order, providing core calculation parameters for subsequent batch matrix multiplication operations. The second stage is the input projection batch calculation stage. Based on the aforementioned standardized data layout and parameters, this step enables the completion of all expert calculations in a single kernel startup. The previously processed continuous lexical buffer is used as the model input data, and the pre-built expert offset table is used as the grouping basis for the input data. Simultaneously, combined with the calculation parameters corresponding to the weight list, the grouped general matrix multiplication operation is called. GEMM (Generative Term Modeling) executes the input projection calculation tasks of all activation experts in a single kernel launch. After the calculation is completed, the projection results of each expert are written to the output buffer in strict segmented order, ensuring that the continuous projection buffer maintains the same segmented and continuous layout as the continuous lexical buffer, which can be directly used as input data for the next stage of calculation. This entire implementation simplifies the E independent matrix multiplication operations in traditional schemes into a single batch matrix multiplication operation, reducing the number of operator calls. It also avoids the problem of intermediate results being scattered and written back during multiple rounds of independent calculations, continuously and stably improving the hardware utilization of matrix calculation units such as Tensor Cores and CubeUnits. While ensuring the complete and accurate execution of the input projection calculations of all activation experts, it optimizes the computational performance and hardware resource utilization efficiency of model inference.

[0037] Step S300: Based on the expert offset table and the continuous projection buffer, perform identity boundary insertion and parallel prefix scanning to obtain an effective scan result sequence.

[0038] In this embodiment, identity element boundary insertion is performed based on the expert offset table and the continuous projection buffer. This step uses the offset parameters of the expert offset table as a benchmark to define the boundary range of each processing identity element. Simultaneously, it combines the continuous projection data temporarily stored in the continuous projection buffer to complete the projection association information corresponding to the identity element boundaries, correcting boundary ambiguity and data misalignment issues. This allows the discrete processing identity elements to form a standardized and continuous boundary system, defining a standard data processing range for subsequent scanning operations and avoiding computational deviations caused by boundary chaos. After completing the identity element boundary insertion, a parallel prefix scan operation is performed. This step, based on the standardized identity element boundaries, uses a parallel computing mechanism to batch synchronously traverse, statistically analyze, and integrate the prefix data of all identity elements, sorting out the effective data information corresponding to each identity element, filtering redundant and invalid data, and quickly aggregating an ordered and complete dataset. Overall, by first standardizing the boundaries of unit metadata and then performing parallel batch scanning to integrate the data, the accuracy and efficiency of projection data scanning processing can be improved. This solves the problems of poor boundary adaptation, long computation time, and messy results in traditional serial scanning. Finally, it outputs a regular, accurate, and directly callable effective scan result sequence, providing reliable data support for subsequent related data operations and projection analysis.

[0039] Specifically, step S300 includes the following steps: Step S301: Based on the expert offset table and the continuous projection buffer, perform identity element boundary insertion to obtain an extended binary sequence containing identity elements; Step S302: Perform a parallel prefix scan on the extended binary sequence to obtain the scan result sequence; Step S303: Based on the scan result sequence, remove all scan items caused by inserting identical placeholders, and extract the prefix scan results corresponding to a preset number of valid word positions to obtain a valid scan result sequence.

[0040] In one implementation, addressing the core issue of recursive state dependencies among SSM experts, this invention proposes a multi-segment parallel scanning mechanism based on segment boundary isolation. Using the Mamba-2 architecture as the application platform, and leveraging the associative operator and identity element boundary isolation characteristics, it efficiently completes independent state updates for multiple experts within a unified continuous execution flow. This avoids the problems of serial recursion, cross-expert state interference, and redundant branch judgments inherent in traditional solutions. The specific implementation process is as follows: First, perform binary parameterization processing, traversing all token positions within the continuous projection buffer. For each token position t in the continuous projection buffer output from the above steps, match the state space model parameters corresponding to the expert at that token position, i.e., the diagonal state transition matrix A. t Input projection matrix B t Combined with the token vector x projected from the current positiont The SSM state update tuple corresponding to each word position is calculated. Generate a complete sequence of binary tuples consisting of state update tuples corresponding to all word elements. To achieve parallel cumulative computation of sequence states, this mechanism defines an association operator. Its operation rules are as follows This operator strictly satisfies the associative law and has the following properties: Based on the mathematical properties of this property, the state update of a sequence of any length N can be transformed into a prefix accumulation form, i.e. This approach breaks free from the constraints of traditional SSM state updates, which involve sequential recursion step-by-step, providing mathematical support for parallel scan computation. Building upon this, Identity Element boundary insertion is implemented. To achieve complete isolation of states across different expert segments and prevent cross-segment state propagation, this mechanism defines an Identity Element I = (1, 0). This Identity Element possesses strict mathematical properties for both left and right Identity Elements, with the left Identity Element property being... The right identity element has the following properties: This means that after the identity element is associated with any state update tuple, the parameters and state information of the original tuple will not change. In actual execution, the boundary positions of the corresponding segments of each expert in the tuple sequence are located using an expert offset table. An identity element I is inserted between the tuple sequences of adjacent expert segments, resulting in an extended tuple sequence containing the identity element. This makes the overall sequence form a structured arrangement of "Segment0 + I + Segment1 + I + Segment2...". Based on the isolation characteristics of the identity element, no additional conditional branches or runtime boundary checks are needed, laying the foundation for the independent evolution of each expert's state. Subsequently, a parallel prefix scan operation is performed on the extended tuple sequence to obtain the scan result sequence. Specifically, the classic Up-Sweep + Down-Sweep algorithm of Blech Scan is used, with the association operator... To achieve global parallel scanning of the core computational rules, thanks to the associative property of the association operator, the scanning process can be completed in O(log(N+E)) parallel steps (where N is the number of original tokens and E is the number of experts), which is far superior to the time complexity of traditional single-sequence state updates (O(N)) and multi-expert serial updates (O(E×N)). During the scanning process, the isolation effect of the identity element automatically takes effect. The cumulative operation of the binary tuples within each segment is not affected by the states of other segments before and after it. The state propagation of adjacent segments is precisely blocked by the identity element at the boundary. Finally, the cumulative state result of each segment is completely equivalent to the result obtained by independent serial scanning of each segment, realizing parallel and independent state updates of multiple experts. Finally, the effective result extraction operation is performed. The result sequence obtained after the above parallel scanning is N+E-1 in length, which contains invalid scan items corresponding to all identity element insertion positions. Therefore, it is necessary to filter the scan result sequence to remove all invalid scan items caused by the insertion of identity placeholders, and only retain the prefix scan results corresponding to the original valid token positions. Finally, a valid scan result sequence of length N is obtained as the subsequent input data. This entire mechanism transforms the traditional serial SSM state update process into a highly efficient parallel computing process through mathematical operators and identity element design. This reduces the computation time for state updates in multi-expert scenarios. Simultaneously, based on a non-intrusive identity element boundary isolation scheme, it ensures that the state spaces of different experts evolve independently without crosstalk. It eliminates the branching logic and decision-making overhead required by traditional boundary isolation, and allows multiple experts to uniformly reuse parallel computing hardware resources such as GPU warp and NPU vector units, maximizing the utilization of hardware parallel computing power while balancing computational efficiency, result accuracy, and hardware adaptability. The meanings and dimensions of the various symbols mentioned above are explained in the table below:

[0041] In one implementation, this embodiment provides a schematic diagram of Segment boundary isolation state propagation in a cross-expert continuous fusion execution method, such as... Figure 3 As shown, the traditional scheme adopts a serial recursive architecture, which has O(N) serial dependency. The data processing flow needs to perform calculations one token at a time, and sequentially complete the serial state propagation of Expert0 from h0 to h1, h1 to h2, and so on. This processing method cannot fully utilize hardware computing power and has low parallel hardware utilization. This invention builds a unified continuous execution flow as the overall computing framework, dividing the complete input sequence into multiple independent segments: Segment0, Segment1, and Segment2. Each Segment corresponds to Expert0, Expert1, and Expert2, respectively. Each Segment contains several computing units. Segment0 contains E00 E 01 E 02 Segment1 contains E 10 E 11 Segment2 contains E 20 E 21 E 22 Boundaries are set between segments to isolate states, enabling the entire architecture to achieve O(log N) parallel execution. To ensure that the states between segments do not interfere with each other, an Identity element mechanism is used to block the propagation of states across segments. IdentityElement is defined as... It possesses two mathematical properties: as well as This characteristic ensures that Identity elements do not modify existing state values ​​during computation, while simultaneously cutting off state propagation paths across segments. The insertion position of Identity elements is set at the boundary of each Segment, resulting in the following complete sequence arrangement: By using Identity elements inserted at the boundary, state isolation between Segments can be automatically guaranteed. In terms of state isolation effectiveness, the final state of Expert0 satisfies... The result of the operation is not affected by the operation process of Expert1 and Expert2, and the final state of Expert1 satisfies The result of the operation is not affected by the operation process of Expert0 and Expert2, and the final state of Expert2 satisfies The calculation result is not affected by the calculation process of Expert0 and Expert1; the state update process of Mamba-2 can be expressed as a tuple. Association operations between And this association operation The associative law is satisfied, therefore identity elements are inserted at the boundaries of each segment. Afterwards, the entire sequence can be updated by a single Parallel Prefix Scan operation to update the state of all experts. The identity element I will automatically block state propagation across segments, and the entire parallel computing process does not require the introduction of any conditional branches.

[0042] In one implementation, the method further includes a state continuous propagation step. This step is used to extract the final state of each expert segment from the scan results of the preceding steps and update the state cache, thereby achieving continuous iterative reuse of states during inference. The specific execution logic is divided into two core sub-steps, which are calculated based on the expert offset table generated in the preceding steps and the sequence of valid scan results. Finally, the updated state cache is output for reuse in subsequent inference stages. The first step is segment final state localization and extraction. This step, based on the expert offset table generated in the preceding steps, accurately locates the end position of the corresponding expert segment for each activated expert k. It reads the scan result corresponding to the end position from the valid scan result sequence. The scan result at this position contains the prefix of all elements within the corresponding expert segment. The first step is to accumulate the computation results, which are equivalent to the final state obtained by serially recursively calculating each expert token by token. This step accurately extracts the updated hidden state h_new corresponding to each activated expert, providing effective input data for subsequent state cache updates and ensuring the accuracy and integrity of the state data. The second step is to update the state cache. Based on obtaining the updated hidden state h_new for each expert, the new hidden state corresponding to each expert is overwritten into the state cache using the expert identifier as the key, replacing the old state stored in the cache. This completes the overall update of the state cache. The state cache adopts a persistent storage method. In subsequent inference processes, whether it is inference at the next time step or computation at the next network layer, when the corresponding expert is reactivated, the latest state stored in the cache will be used as the initial state h_init for the SSM recursive computation to participate in the early scanning computation process. The entire continuous state propagation process can effectively maintain the continuity of the state during inference, without re-initializing the state every time an expert is activated, simplifying the computation process. At the same time, all state extraction and update operations are completed within a unified cache buffer, without the need for cross-buffer data copying, effectively reducing the read and write overhead of state data. Furthermore, the persistent residency of the state cache can avoid redundant operations of repeatedly calculating historical states during the model autoregression generation stage, thereby improving the overall execution efficiency of the model autoregression generation stage. The updated state cache will continuously provide initial state support for the subsequent scanning calculation stage of inference, enabling efficient and continuous iterative computation of the model inference process.

[0043] Step S400: Using the effective scan result sequence as input, perform index inverse mapping according to the recovery index table, and use the routing weight to perform weighted aggregation of the multi-expert outputs to obtain the target output sequence arranged in the order of the original word sequence.

[0044] In this embodiment, firstly, the selected valid scan result sequence is used as the core input data, providing compliant and effective basic data for subsequent index restoration and result aggregation, eliminating interference from invalid and redundant data, and ensuring the accuracy and efficiency of subsequent operations. Next, based on a preset recovery index table, an index inverse mapping operation is performed, restoring the scrambled and reorganized lexical index information from the scanning process, matching the lexical correspondences of the original text, correcting sequence errors caused by the scanning process, and laying a solid foundation for restoring the complete original lexical sequence. Then, weighted aggregation processing is performed on the multi-expert model's multivariate output results using preset routing weights. Weights are assigned based on the adaptability and advantages of different expert outputs, integrating effective features from each branch, mitigating the biases and defects of individual outputs, and integrating a more semantically logical and more accurate comprehensive output. Finally, based on the above series of operations, the output content order is regularized, ultimately generating a target output sequence that strictly conforms to the original lexical sequence arrangement rules. The entire process achieves a complete closed loop of scanning data error correction and restoration, multi-model advantage fusion, and accurate sequence restoration, effectively improving the completeness, accuracy, and regularity of the model output results, and ensuring that the final output content is highly matched with the structure and semantics of the original input text.

[0045] Specifically, step S400 includes the following steps: Step S401: Using the valid scan result sequence as input, perform index inverse mapping according to the restored index table to inverse map the scan results of the continuous layout order back to the original word sequence order, and obtain the result sequence arranged in the original order; Step S402: Based on the result sequence, the multi-expert outputs are weighted and aggregated using the routing weights to obtain the target output sequence arranged in the order of the original word sequence.

[0046] In one implementation, the core of the Fused Writeback Restore is used to restore the results of continuous execution flow to the original token sequence order. It consists of two coherent sub-steps: index inverse mapping and multi-expert output weighted aggregation. This achieves a lossless transformation from a unified execution flow to the original sequence layout, while avoiding additional output reorganization overhead. The fusion of index mapping and weight aggregation processes reduces the number of writeback operations, ensuring efficient and accurate model inference. The specific execution flow is as follows: First, the index inverse mapping sub-step is executed. This step takes the effective scan result sequence as input and uses the restored index table as the core basis for inverse mapping. The aim is to restore the scan results, after continuous layout adjustments, to the original token sequence arrangement. Specifically, for each position i in the original token sequence, the corresponding position in the continuous layout is... Through formula After completing the position-by-position mapping operation, the final output is a sequence of results arranged in the original token order, providing compliant input for subsequent weighted aggregation operations. After completing the index inverse mapping, the multi-expert output weighted aggregation sub-step is executed. This step is primarily for scenarios where a single original token is routed to multiple expert networks. In a multi-expert activation scenario, routing weights are used as weighting coefficients to perform a weighted summation operation on the K expert outputs corresponding to the same original token, as output from the aforementioned index inverse mapping step. This results in the aggregated output for that token. When K=1, meaning each original token activates only a single expert network, this weighted aggregation process automatically degenerates into an identity mapping, requiring no additional computation. The final output dimension of this sub-step is... The target output sequence Y is arranged strictly according to the original word sequence order. This sequence can be directly passed to the next layer of the network to continue to participate in the operation, or directly used as the final inference result of the model. The entire fusion write-back and recovery process, through the deep integration of two functions, simplifies the data recombination and write-back operations while fully restoring the original sequence layout and ensuring lossless data conversion. This effectively reduces the computational overhead and timing loss in the model inference process.

[0047] In one implementation, this invention uses an autoregressive sequence reasoning scenario as its application carrier. Based on Mamba-2 SSM experts, it constructs a cross-expert continuous fusion execution method, which can effectively achieve efficient collaborative reasoning among multiple SSM experts. It is adaptable to both regular and ultra-long sequence reasoning scenarios. The overall solution has a closed-loop logic and is feasible for implementation. The specific implementation process is as follows: The core execution unit of this method is a single Mamba-2 SSM expert. Its forward computation includes three core steps. The first step is input projection, which completes the input dimension mapping through linear transformation. The calculation method is as follows: The dimension can be The original input is converted into dimensions. The intermediate features; the second step is a recursive state update, which iteratively calculates the hidden state step by step. The core formula is: ,in Let be a diagonal state transition matrix, with diagonal elements denoted as . B[t] and C[t] are the input and output projection vectors, respectively. This formula can realize the fusion and iteration of historical hidden states and current input features to complete the temporal state evolution; the third step is output projection, which uses Linear(y) to project the vectors of dimension [N, D]. stateThe state features are restored to the final output features of dimension [N, D], ensuring that the output dimension matches the input dimension. Furthermore, a core feature of Mamba-2 is that this recursive state update process can be equivalently converted into a parallel prefix scan form. Based on this parallel equivalence property, which has been verified in the original Mamba-2 paper, the serial bottleneck of traditional recursive computation can be overcome, providing fundamental support for multi-expert parallel inference. This scheme requires four types of pre-set core input data during execution: an input hidden state sequence of dimension [N, D]. Routing network parameters Multiple SSM expert parameters, each expert contains And an expert state cache that stores the latest hidden states of each expert. The final output is a sequence of hidden states of dimension [N, D]. Simultaneously, the expert state cache is iteratively updated to provide continuous state support for subsequent inference. To achieve efficient continuous execution across experts, four types of dedicated data structures need to be pre-built. Among them, the Expert Offset Table is used to record the start and end positions of the corresponding token segments of each expert in the continuous buffer, which can achieve segment boundary positioning with O(1) complexity, for example... This corresponds to Expert0 occupying the interval [0, 128), Expert1 occupying the interval [128, 256), and Expert2 occupying the interval [256, 400). A continuous Token Buffer stores the rearranged Token sequence, grouping Tokens corresponding to the same expert into a continuous interval to form a regular segmented memory layout [Expert0 Tokens][Expert1 Tokens][Expert2 Tokens]... A recovery index table records the mapping relationship between the rearranged Token positions and their original positions, providing a basis for restoring the final output sequence. A state cache uses the expert ID as a unique key, storing each expert's dimension as [D...]. stateThe latest hidden state of [[]] enables iterative reuse of inference states. Based on the above data structure, this solution completes cross-expert fusion inference through a five-step continuous process from S1 to S5. Each step works in concert and progresses step by step to comprehensively optimize the efficiency of multi-expert inference. Among them, S1 is expert-aware rearrangement, the core function of which is to organize the data storage structure. First, the routing probability of each token corresponding to each expert is calculated through the routing network. After Softmax normalization and Top-K screening, the set of active experts for a single token is determined. Then, all tokens are reordered according to the expert identifier, so that the tokens corresponding to the same expert form continuous storage segments. Finally, a continuous token buffer, an expert offset table and a recovery index table are generated. This process only adjusts the token storage order and does not change the expert affiliation, routing weight and expert inherent parameters of the token, ensuring that each segment always corresponds to the original expert's exclusive calculation path. S2 is a cross-expert fusion projection. Based on the regular data structure generated by S1, it realizes batch optimization of the computation operator. Based on the continuous TokenBuffer and expert offset table, it adopts hardware-adapted batch matrix multiplication operators such as Grouped GEMM or Ascend NPU npu_grouped_matmul to integrate the input projection operation of all active experts into a single kernel call. The operator distinguishes the token grouping boundary of each expert through the expert offset table, completes the input projection calculation of all experts in batches, and writes the projection result directly to the continuous buffer, eliminating the redundant operation of writing back intermediate data in a scattered manner, reducing the frequency of operator calls and data read and write overhead. S3 is a multi-segment parallel scan, which implements independent parallel state evolution of multiple experts in segments. For the multiple continuous token segments formed by S2, such as Expert0 corresponding to Segment0, Expert1 corresponding to Segment1, Expert2 corresponding to Segment2, etc., all segment elements are uniformly expanded into the global continuous execution flow, and identity units I=(1,0) are inserted at the boundaries of adjacent segments. The blocking property of identity units is used to achieve isolated evolution of different expert states. Then, parallel prefix scan operation is performed on the entire continuous sequence. Implementation methods such as Blech Scan (up-sweep + down-sweep two-pass scan), CUDA CUB library, Triton associative_scan, or custom NPU kernel can be selected. Based on the associative property of prefix operation, synchronous parallel computation of all segments is achieved. During the operation, the state only propagates within a single segment, and each expert maintains an independent state evolution path. At the same time, all segments reuse the same parallel execution flow, abandoning the traditional expert-by-expert serial scan mode.S4 is for continuous state propagation, enabling iterative updates and persistence of the inference state. After parallel scanning and computation, the end position of each token segment is accurately located using the expert offset table. The final hidden state corresponding to each segment is extracted, establishing a one-to-one correspondence between segments and experts. That is, the latest inference state of the expert corresponding to the end state of each segment is written back to the expert state cache, making it the initial state for the next round of inference, ensuring the continuity and iterative nature of expert states during temporal inference. S5 is for fusion, write-back, and recovery, ensuring the original temporal sequence and computational accuracy of the output sequence. Based on the recovery index table constructed in S1, the output results arranged in a continuous execution flow are restored to the original token temporal order. For the case where a single token corresponds to multiple expert outputs in the Top-K>1 scenario, multiple sets of output results are weighted and aggregated according to the routing weight of each expert, finally obtaining an output sequence Y that perfectly matches the order and dimension of the original input sequence. Compared to the traditional MoE-SSM implementation, the entire basic process eliminates the redundancy of expert-level loop scheduling in traditional solutions by constructing a unified continuous execution flow across experts. This reduces the overall number of Kernel Launch calls from E×3 to 5 or less. At the same time, it optimizes the computational complexity of multi-expert state updates from a serial complexity of O(E×N) to a parallel complexity of O(E+log N). Based on the segmented boundary identity unit mechanism, it naturally ensures the independent evolution of each expert's state, avoiding interference between expert states. The continuous memory layout can effectively improve the cache hit rate of CPU and NPU hardware. Furthermore, through index mapping and weighted aggregation mechanisms throughout the process, it ensures that the optimized output results are completely consistent with the results of traditional expert-by-expert independent execution, thus comprehensively improving inference efficiency without compromising inference accuracy. For scenarios involving ultra-long sequence inference, where the input sequence length N is too large and the memory usage of a single parallel scan exceeds the hardware's capacity limit, this method extends the block-based inference strategy to achieve a balance between hardware adaptation and efficiency. Specifically, the complete input sequence is equally divided into K data blocks along the time dimension, with each block having a sequence length of N / K. Within each data block, the aforementioned S1-S5 cross-expert continuous fusion execution process is fully reused to complete token rearrangement, batch projection, parallel scanning, state update, and result recovery within the block, achieving efficient parallel computation within the block. At the same time, a block-level prefix propagation mechanism is added, using the final hidden state of each expert in the previous data block as the initial state for the inference of the corresponding expert's data block in the next data block, ensuring the state continuity of segmented inference for ultra-long sequences. Finally, the output results of all data blocks are concatenated to form a complete long sequence output. This expansion strategy can optimize the memory usage from the original O(N×D_state) to a controllable O(N / K×D_state), solving the memory overflow problem of long-sequence single-parallel scanning, while avoiding the efficiency decay problem of overall parallel computing of ultra-long sequences. Moreover, only the final hidden state of the core is passed between blocks, resulting in minimal data communication, thus balancing the stability, efficiency, and accuracy consistency of long-sequence inference.

[0048] In one implementation, this embodiment provides a comparison diagram between traditional solutions and the cross-expert continuous fusion execution method of this invention, such as... Figure 4 As shown. This invention proposes a cross-expert continuous fusion execution flow for the MoE-SSM architecture to address the inherent shortcomings of traditional expert-by-expert serial execution schemes, where the SSM recursive state dependency isolates tokens into multiple independent serial chains according to experts, making batch parallel computation impossible. Traditional schemes require 64 iterations of the expert process, with token collection, gate_up projection, state update, and down projection operations completed sequentially within each iteration. In a single-layer inference scenario, if the number of activated experts E=8, this results in E×3 Kernel Launches, totaling 24 times. Frequent Gather and Scatter memory accesses occur, the state update time complexity reaches O(E×N), and the overall operator call count is no less than 24. The execution flow of this invention is divided into five stages, S1 to S5, to collaboratively complete the computation. Based on a contiguous memory layout of experts, after inputting a Token[N,D] carrying a hidden state sequence, the process first enters the S1 expert-aware rearrangement stage. Based on the output of the routing network Router, and combined with three auxiliary tables—the contiguous Token Buffer, the expert offset table, and the recovery index table—the Token is rearranged according to the expert ID, constructing a contiguous storage layout in memory. This optimizes the memory access mode, enabling subsequent computations to use sequential access instead of the original distributed storage mode's random Gather access, improving cache hit rate. Next, the S2 cross-expert fusion projection stage is entered, employing Grouped GEMM operations. The projection calculation for all experts is completed in only one Kernel Launch, merging the E independent Matmul operations of the traditional scheme and reducing the expert dimension loop scheduling overhead. Then, the S3 multi-segment parallel scanning stage is entered. By inserting Identity Elements between sequence segments corresponding to different experts, cross-expert state dependencies are severed, and multiple independent serial chains are spliced ​​into a unified sequence that can be scanned in parallel. Parallel prefix scanning transforms the original O(E×N) serial steps of state updates into O(E+log N) parallel steps, eliminating the O(N) serial bottleneck. This enables parallel scanning within a segment and ensures independent evolution of each expert state through boundary isolation. Next, in the S4 continuous state propagation phase, the initial state h_init from the previous time step is read from the expert state cache. The states of each segment are extracted to update the expert state cache, resulting in h_new. A unified buffer is used for state extraction and continuous write-back operations, reducing state I / O overhead. Finally, in the S5 fusion write-back recovery phase, the inverse mapping of the joint index and the aggregation of routing weights are performed. Based on the recovery index table, the original order of the tokens is restored, and the token[N,D] carrying the updated hidden state sequence is output. This optimizes the output recovery process and effectively reduces the number of write-backs compared to the traditional multi-Scatter write-back mode. The entire S1-S5 pipeline integrates and compresses the number of operator kernel launches to no more than 5, a reduction of approximately 80% compared to traditional solutions. This includes 3 core kernel launches for fusion projection, parallel scanning, and write-back, a reduction of approximately 87% compared to the original solution. The number of write-backs in the output recovery stage is reduced by approximately E times compared to the original solution. The entire execution flow completely transforms the original expert-per-expert serial execution MoE-SSM computation process into a unified continuous parallel execution flow, comprehensively reducing operator scheduling overhead. In a sequence length N=2048 scenario, the parallelism of the state update stage is improved by N / log N times compared to traditional solutions. Existing optimization methods such as Grouped GEMM, TokenPacking, and Expert Batching cannot solve the serial state dependency problem of SSM. This solution breaks through this bottleneck at its root by relying on multi-stage parallel scanning. Each stage cooperates to complete the entire chain processing of data reorganization, batch projection, parallel state evolution, persistent state update, and original sequence restoration, completely constructing a deployable MoE-SSM cross-expert continuous parallel inference path.

[0049] In summary, this embodiment first performs routing calculations, lexical statistical aggregation, continuous cache arrangement, and index table construction operations on the lexical sequence to obtain routing weights, continuous lexical cache areas, expert offset tables, and restored index tables. This achieves continuous and structured storage of discrete lexical metadata from the data layer, improving upon the problems of disordered lexical storage, inefficient data retrieval, and ambiguous expert correspondences in traditional technologies. It provides a regular, unified, and quickly accessible data foundation and index support for all subsequent computational stages, avoiding computational biases and resource waste caused by data chaos from the source. Then, based on the aforementioned... The continuous lexical buffer and the expert offset table are used to perform input projection calculations for all activated experts, resulting in a segmented, continuously arranged continuous projection buffer. This step, based on pre-structured data, achieves batch and regular calculations of activated expert input projections, abandoning the traditional fragmented calculation mode of per-lexical and per-expert. It effectively unifies the arrangement format of each expert's input data, reduces the fragmented overhead of projection calculations, improves the overall efficiency of multi-expert input projection calculations, and ensures the integrity and accuracy of each activated expert's input data, laying a solid computational foundation for subsequent scanning operations. Subsequently, based on the... The expert offset table and the continuous projection buffer are used to perform unit boundary insertion and parallel prefix scanning to obtain an effective scan result sequence. By accurately defining the boundary range of each word and each expert calculation unit, parallel collaborative operation of multi-unit prefix scanning is realized, breaking through the technical bottlenecks of low efficiency, ambiguous boundary recognition, and large amount of invalid calculation in traditional serial scanning. This improves the parallelism and operation speed of data scanning processing. At the same time, it accurately filters and retains effective data and removes invalid and redundant information to ensure the effectiveness, accuracy, and completeness of the scan results. Then, using the effective scan result sequence as input, the index inverse mapping is performed according to the recovery index table, and the routing weights are used to perform weighted aggregation of the multi-expert outputs to obtain the target output sequence arranged in the order of the original word sequence. This step repairs the disordered word order in the previous segmented calculation and regularization process through index inverse mapping, ensuring the consistency of the output sequence with the original input sequence. At the same time, based on the precise routing weights, the differentiated weighted fusion of the multi-expert outputs is completed, avoiding the problems of feature loss and insufficient output accuracy caused by traditional average aggregation and fixed weight aggregation, thus improving the accuracy and adaptability of the output results.Overall, this invention constructs a complete multi-expert lexical sequence processing system through end-to-end optimization, including pre-process data structuring and regularization, efficient parallel computing in the middle stage, and precise sorting and aggregation in the back end. It comprehensively optimizes the shortcomings of traditional technologies from multiple dimensions, including data storage, computational architecture, processing logic, and output accuracy. This effectively reduces computational redundancy and resource consumption in multi-expert model lexical processing, improves the parallel computing efficiency, data processing accuracy, and output stability of sequence processing, and achieves efficient regularization of discrete lexical data, collaborative parallel advancement of multi-expert computing, and accurate restoration and high-quality aggregation of output sequences. This comprehensively enhances the overall performance and application effect of multi-expert models in lexical sequence processing tasks.

[0050] like Figure 5 As shown in the illustration, this embodiment also provides a cross-expert continuous fusion execution device, which includes: a routing weight, continuous lexical cache, expert offset table, and recovery index table acquisition module 10, a continuous projection cache acquisition module 20, an effective scan result sequence acquisition module 30, and a target output sequence acquisition module 40. Specifically, the routing weight, continuous lexical cache, expert offset table, and recovery index table acquisition module 10 is used to perform routing solution, lexical statistical aggregation, continuous cache arrangement, and index table construction operations on the lexical sequence to obtain the routing weight, continuous lexical cache, expert offset table, and recovery index table. The continuous projection cache acquisition module 20 is used to perform input projection calculation of all activated experts based on the continuous lexical cache and the expert offset table to obtain a segmented continuous projection cache. The effective scan result sequence acquisition module 30 is used to perform identity boundary insertion and parallel prefix scanning based on the expert offset table and the continuous projection cache to obtain the effective scan result sequence. The target output sequence acquisition module 40 is used to take the effective scan result sequence as input, perform index inverse mapping according to the recovery index table, and use the routing weight to perform weighted aggregation of multiple expert outputs to obtain the target output sequence arranged in the order of the original word sequence.

[0051] In one implementation, the device further includes a state cache update module, which is used to locate and extract the updated hidden state corresponding to each activated expert segment based on the expert offset table and the effective scan result sequence, and overwrite it to the state cache with the expert identifier as the key to obtain the updated state cache.

[0052] In one implementation, the routing weight, continuous lexical cache, expert offset table, and recovery index table acquisition module 10 includes: The routing weight acquisition unit is used to calculate the routing probability of each word for each expert through the routing network based on the input word sequence, and determine the expert identifier list and corresponding routing weight for each word based on the routing probability. The expert lexical count array acquisition unit is used to traverse and count the expert identifier list, determine the number of lexicals corresponding to each activated expert, and obtain the expert lexical count array. The continuous word buffer acquisition unit is used to allocate continuous memory according to the expert word count array and divide the corresponding intervals of each expert. Words belonging to the same expert are copied to their respective intervals in the order of the original sequence to obtain a continuous word buffer in which the word segments of each expert are continuously arranged. The expert offset table and recovery index table acquisition unit is used to generate the expert offset table and recovery index table based on the layout information of the expert lexical count array and the continuous lexical cache area.

[0053] In one implementation, the continuous projection buffer acquisition module 20 includes: The weight list acquisition unit is used to extract the input projection weight matrix of each activated expert from the expert parameter set, and obtain a weight list arranged in expert order; The continuous projection buffer acquisition unit is used to take the continuous lexical buffer as input data, group the input data according to the expert offset table, and use the weight list as calculation parameters. It performs a grouped general matrix multiplication operation through a single kernel startup to complete the input projection calculation of all activated experts and obtain a segmented continuous projection buffer.

[0054] In one implementation, the effective scan result sequence acquisition module 30 includes: An extended binary sequence acquisition unit is used to insert identity element boundaries based on the expert offset table and the continuous projection buffer to obtain an extended binary sequence containing identity elements. The scan result sequence acquisition unit is used to perform parallel prefix scanning on the extended binary sequence to obtain the scan result sequence; The effective scan result sequence acquisition unit is used to remove all scan items caused by the insertion of identical placeholders based on the scan result sequence, and extract the prefix scan results corresponding to a preset number of valid word positions to obtain the effective scan result sequence.

[0055] In one implementation, the extended binary sequence acquisition unit includes: The binary sequence acquisition subunit is used to traverse all word positions in the continuous projection buffer, calculate the state update binary corresponding to each word position based on the parameters of the attribution expert state space model corresponding to each word position, and generate a binary sequence composed of all state update binary. The extended binary sequence obtains sub-units, which are used to locate the boundaries of each segment of the binary sequence through an expert offset table. Unit elements are inserted between adjacent segments of the binary sequence to obtain an extended binary sequence containing unit elements.

[0056] In one implementation, the target output sequence acquisition module 40 includes: The result sequence acquisition unit is used to take the valid scan result sequence as input, perform index inverse mapping according to the recovery index table, and inverse map the scan result of continuous layout order back to the original word sequence order to obtain the result sequence arranged in the original order; The target output sequence acquisition unit is used to perform weighted aggregation of multiple expert outputs based on the result sequence and using the routing weights to obtain a target output sequence arranged in the order of the original word sequence.

[0057] The working principle of each module in the cross-expert continuous fusion execution device of this embodiment is the same as that of each step in the above method embodiment, and will not be repeated here.

[0058] In one implementation, this embodiment provides a preferred embodiment structural diagram of a cross-expert continuous fusion execution device. The cross-expert continuous fusion execution device adopts a pipelined five-module architecture, such as... Figure 6As shown, the overall data flow is as follows: input data, expert reordering module, continuous layout management module, fusion execution module, state propagation module, write-back recovery module, and finally output data is generated. An independent state cache is configured to interact with the fusion execution module. The state cache stores the running states of each expert, such as Expert0 State, Expert1 State, and Expert2 State. The module for acquiring routing weights, continuous token caches, expert offset tables, and recovery index tables constitutes the Expert Reordering Module. This module receives input data, performs routing probability calculations, and reorders tokens according to expert categories. The module for acquiring continuous projection caches constitutes the Continuous Layout Management Module. This module receives the output data stream from the Expert Reordering Module, continuously maintains the expert offset table and recovery index table, and establishes a data path with the state cache. The module for acquiring effective scan result sequences constitutes the Fused Execution Module, which is the core of the device. This module integrates the fusion projection submodule, the multi-segment scan submodule, and the fusion write-back submodule, corresponding to the three processing logics in the method flow: S2 fusion projection, S3 multi-segment parallel scan, and S5 fusion write-back. It can read the expert states stored in the state cache to complete the fusion calculation. The state cache update module is the State Propagation Module. The first module receives the processing results from the fusion execution module, completes segment state extraction, and synchronously updates the expert state cache to ensure the continuity of expert states during cross-time step processing. The second module, the target output sequence acquisition module (also known as the Writeback Restore Module), receives the output information from the state propagation module, performs index mapping recovery, and merges the outputs to finally output the data. This device architecture maps the complete method flow from S1 to S5 described above into the five software and hardware functional modules. Each module collaboratively executes all processing steps in the aforementioned method embodiment along the data flow sequence. Based on the ordered data flow between modules and the state cache, cross-time step state continuity is achieved, forming a complete and feasible cross-expert continuous fusion execution software and hardware implementation plan.

[0059] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 7 As shown. The terminal may include one or more processors 100 ( Figure 7(Only one is shown in the diagram), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100, such as a cross-expert continuous fusion execution program. When one or more processors 100 execute computer program 102, they can implement the various steps in the cross-expert continuous fusion execution method embodiment. Alternatively, when one or more processors 100 execute computer program 102, they can implement the functions of various modules / units in the cross-expert continuous fusion execution method embodiment, which is not limited here.

[0060] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0061] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or RAM. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, memory 101 may include both internal and external storage units. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 can also be used to temporarily store data that has been output or will be output.

[0062] Those skilled in the art will understand that Figure 7 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0063] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, operational databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operating data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-expert continuous fusion execution method, characterized in that, The method includes: Perform routing, lexical statistics aggregation, continuous cache arrangement and index table construction operations on the lexical sequence to obtain the routing weight, continuous lexical cache area, expert offset table and recovery index table; Based on the continuous lexical buffer and the expert offset table, the input projection calculation of all activated experts is performed to obtain a segmented continuous projection buffer. Based on the expert offset table and the continuous projection buffer, identity boundary insertion and parallel prefix scanning are performed to obtain an effective scan result sequence. Using the effective scan result sequence as input, perform index inverse mapping according to the recovery index table, and use the routing weight to perform weighted aggregation of multiple expert outputs to obtain the target output sequence arranged in the order of the original word sequence.

2. The cross-expert continuous fusion execution method according to claim 1, characterized in that, The method further includes: Based on the expert offset table and the effective scan result sequence, the updated hidden state corresponding to each activated expert segment is located and extracted, and then overwritten to the state cache with the expert identifier as the key to obtain the updated state cache.

3. The cross-expert continuous fusion execution method according to claim 1, characterized in that, The process of performing route solving, lexical statistical aggregation, continuous cache arrangement, and index table construction on the lexical sequence yields route weights, continuous lexical cache areas, expert offset tables, and a restored index table, including: Based on the input word sequence, the routing probability of each word for each expert is calculated through the routing network, and the expert identifier list and corresponding routing weight of each word are determined according to the routing probability. The expert identifier list is traversed and counted to determine the number of lexical units corresponding to each activated expert, thus obtaining an expert lexical unit count array. Based on the expert lexical counting array, contiguous memory is allocated and the corresponding intervals of each expert are divided. Lexicals belonging to the same expert are copied to their respective intervals in the order of the original sequence, resulting in a continuous lexical cache area with segmented and continuous arrangement of expert lexicals. Based on the layout information of the expert lexical counting array and the continuous lexical cache area, an expert offset table and a recovery index table are generated.

4. The cross-expert continuous fusion execution method according to claim 1, characterized in that, The step of performing input projection calculations on all activated experts based on the continuous lexical buffer and the expert offset table to obtain a segmented, continuously arranged continuous projection buffer includes: Extract the input projection weight matrix of each activated expert from the expert parameter set to obtain a weight list arranged in expert order; Using a continuous term buffer as input data, the input data is grouped according to the expert offset table, and the weight list is used as the calculation parameter. The grouped general matrix multiplication operation is performed through a single kernel startup to complete the input projection calculation of all activated experts, resulting in a segmented and continuously arranged continuous projection buffer.

5. The cross-expert continuous fusion execution method according to claim 1, characterized in that, The process of performing identity boundary insertion and parallel prefix scanning based on the expert offset table and the continuous projection buffer to obtain an effective scan result sequence includes: Based on the expert offset table and the continuous projection buffer, identity element boundary insertion is performed to obtain an extended binary tuple sequence containing identity elements. Perform a parallel prefix scan on the extended binary sequence to obtain the scan result sequence; Based on the scan result sequence, all scan items caused by the insertion of identical placeholders are removed, and the prefix scan results corresponding to a preset number of valid word positions are extracted to obtain the valid scan result sequence.

6. The cross-expert continuous fusion execution method according to claim 5, characterized in that, The step of inserting identity elements based on the expert offset table and the continuous projection buffer to obtain an extended binary sequence containing identity elements includes: Traverse all lexical positions in the continuous projection buffer, calculate the state update tuple corresponding to each lexical position based on the parameters of the attribution expert state space model corresponding to each lexical position, and generate a tuple sequence composed of all state update tuples. By using an expert offset table to locate the boundaries of each segment of the binary sequence, and inserting identity elements between adjacent segments of the binary sequence, an extended binary sequence containing identity elements is obtained.

7. The cross-expert continuous fusion execution method according to claim 1, characterized in that, The process of taking the effective scan result sequence as input, performing index inverse mapping according to the restored index table, and using the routing weights to perform weighted aggregation of multiple expert outputs to obtain the target output sequence arranged in the original word sequence order includes: Using the valid scan result sequence as input, perform index inverse mapping according to the restored index table to inverse map the scan results of the continuous layout order back to the original word sequence order, and obtain the result sequence arranged in the original order; Based on the resulting sequence, the outputs of multiple experts are weighted and aggregated using the routing weights to obtain the target output sequence arranged in the order of the original word sequence.

8. A cross-expert continuous fusion execution device, characterized in that, The device includes: The module for obtaining route weights, continuous lexical cache, expert offset table, and recovery index table is used to perform route solving, lexical statistical aggregation, continuous cache arrangement, and index table construction operations on the lexical sequence to obtain route weights, continuous lexical cache, expert offset table, and recovery index table. The continuous projection buffer acquisition module is used to perform input projection calculations for all activated experts based on the continuous lexical buffer and the expert offset table, so as to obtain a segmented continuous projection buffer. The effective scan result sequence acquisition module is used to perform identity boundary insertion and parallel prefix scanning based on the expert offset table and the continuous projection buffer to obtain the effective scan result sequence; The target output sequence acquisition module is used to take the effective scan result sequence as input, perform index inverse mapping according to the recovery index table, and use the routing weight to perform weighted aggregation of multiple expert outputs to obtain the target output sequence arranged in the order of the original word sequence.

9. A terminal, characterized in that, The terminal includes a memory, a processor, and a cross-expert continuous fusion execution program stored in the memory and executable on the processor. When the processor executes the cross-expert continuous fusion execution program, it implements the steps of the cross-expert continuous fusion execution method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a cross-expert continuous fusion execution program, which, when executed by a processor, implements the steps of the cross-expert continuous fusion execution method as described in any one of claims 1-7.

Images