Hybrid expert model multi-lexical element prediction method and device based on dense-sparse parallel computing, medium, program product and terminal

Abstract

According to the hybrid expert model multi-lexical element prediction method and device based on dense-sparse parallel calculation, the medium, the program product and the terminal provided by the invention, parallel execution is realized in a Tensor Core and a CUDA Core through a dense-sparse collaborative scheduling mechanism, the problem of low hardware utilization rate caused by expert activation fragmentation when MTP and MoE are combined is solved, and the prediction efficiency of the hybrid expert model multi-lexical element prediction method and device based on dense-sparse parallel calculation is improved. The hardware efficiency and throughput performance of MTP and MoE joint reasoning are obviously improved; according to the method, the structure of the MoE model does not need to be modified or additionally trained, high-performance reasoning can be realized only through optimization in the operation process, and the method is suitable for a production scene in which the model cannot be modified; according to the dynamic pruning strategy based on the activation frequency and the gating weight, redundant calculation is effectively eliminated, the model reasoning speed is remarkably increased while the precision of the output result is guaranteed, and the stability and reliability of the model generation quality are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language model prediction, in particular to a mixed expert model multi-word element prediction method and device based on dense-sparse parallel computing, a medium, a program product and a terminal. BACKGROUND

[0002] At present, the dynamic activation mechanism of the mixed expert model (Mixture of Experts, MoE) brings new challenges: due to the different expert combinations activated by each input, the system faces complexity in resource scheduling, load balancing and inter-expert communication, which is easy to cause delay or performance bottleneck, so special optimization strategies are needed to efficiently deploy and run such models.

[0003] The existing mixed expert model inference optimization methods can be systematically divided into three levels: model level, system level and hardware level optimization. The optimization at the model level mainly improves the efficiency of the mixed expert model itself by improving the model structure, adjusting the parameters and designing more efficient algorithms, such as designing more efficient model architectures (such as attention mechanisms and feedforward networks based on mixed experts), using model compression techniques (such as expert pruning, quantization, knowledge distillation and matrix decomposition), and improving algorithm strategies (such as dynamic gating mechanisms and skipping part of the expert's calculation). The optimization at the system level makes full use of the inherent sparse activation characteristics of the mixed expert model, focusing on solving the running efficiency problem in the distributed environment, typical methods including expert parallelism (allocating different experts to different computing nodes), expert offloading (temporarily moving part of the experts to memory or external storage to save resources), and optimizing the "all-to-all" communication overhead between nodes. The optimization at the hardware level focuses on accelerating the mixed expert model on specific hardware platforms to make up for the mismatch between traditional hardware (mainly designed for dense computation) and the dynamic sparse computation mode of the mixed expert model. These three types of optimization each have their own advantages: the model level reduces the computational demand from the algorithm root, the system level improves the resource utilization in the distributed environment, and the hardware level maximizes the computational efficiency of physical devices. The three work together to promote the more efficient and practical deployment of mixed expert models in various computing environments.

[0004] Multi-Token Prediction (MTP) is a technique that enables language models to predict multiple future tokens simultaneously (rather than generating them sequentially). This allows the model to generate multiple tokens in a single forward computation, significantly improving inference efficiency. During training, MTP learns richer contextual relationships by predicting multiple tokens simultaneously, thus improving sample utilization efficiency. During inference, it drastically reduces generation time, making it particularly suitable for latency-sensitive applications. Furthermore, MTP endows the model with "self-predictive decoding" capabilities—meaning the model can predict and validate multiple tokens internally without relying on external components.

[0005] Therefore, among various algorithm advancements, multi-word prediction technology is a particularly effective technique. However, fine-tuning-based multi-word prediction implementations typically require extensive and computationally expensive fine-tuning of the base model to ensure consistency among multiple prediction results, which significantly limits its practical application. To avoid this overhead, researchers have proposed a lightweight "draft-validation" multi-word prediction framework. However, when this "draft-validation" method is directly applied to hybrid expert models, compatibility issues arise.

[0006] Specifically, combining the "draft-verification" multi-word prediction method with a hybrid expert model leads to severe hardware underutilization and additional overhead. This incompatibility manifests in the execution dynamics of the hybrid expert model: when processing a single input, the hybrid expert model can leverage its sparsity to activate only a small number of experts for each word, thus operating efficiently; however, in the parallel verification phase of multi-word prediction, the system needs to process multiple candidate words simultaneously (i.e., a batch), which forces almost all experts to be activated. This is because the routing mechanism of the hybrid expert model depends on the specific input, and different draft words in the same batch often trigger different subsets of experts. This fragmentation of expert activation makes parallel computation difficult to achieve—each expert must process non-overlapping word groups sequentially, failing to effectively utilize the hardware's parallel capabilities. Ultimately, the throughput advantage theoretically brought by multi-word prediction is offset by the overhead in actual execution. Experiments show that although multiple words can be generated per inference, the overall generation latency increases significantly due to the forced serialization of expert computation, with the total generation time being up to 1.64 times that of the traditional autoregressive word-by-word generation method.

[0007] Therefore, it is necessary to provide a method, device, medium, terminal and program product for multi-term prediction based on a hybrid expert model with dense-sparse parallel computing to solve the above-mentioned problems in the prior art. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the present invention provides a method, device, medium, program product and terminal for multi-term prediction based on a hybrid expert model of dense-sparse parallel computing, which solves the problems in the prior art where the combination of term prediction method and hybrid expert model results in a significant increase in overall generation delay and a higher total generation time than traditional methods due to the forced serialization of expert calculations.

[0009] To achieve the above and other related objectives, a first aspect of this application provides a multi-term prediction method based on a hybrid expert model using dense-sparse parallel computation, applied to a GPU including Tensor Cores and CUDA Cores. The method includes: processing input data of the hybrid expert model using a draft model to generate several draft terms; wherein the hybrid expert model includes a gating network and several expert networks; inputting the several draft terms into the gating network of the hybrid expert model for computation to obtain the activation count and total weight value corresponding to each expert network; performing thresholding on the several expert networks based on the activation count and total weight value corresponding to each expert network to obtain a dense expert network subset and a sparse expert network subset; optimizing the sparse expert network subset to obtain an optimized sparse expert network subset; sending the computation task of the dense expert network subset to the Tensor Core and the computation task of the optimized sparse expert network subset to the CUDA Core, so as to perform parallel processing of the computation tasks in the Tensor Core and CUDA Core.

[0010] In some embodiments of the first aspect of this application, the process of inputting a plurality of draft lexical units into a gating network of a hybrid expert model for calculation to obtain the activation count and total weight value corresponding to each expert network includes: inputting a plurality of draft lexical units into a gating network of a hybrid expert model, the gating network calculating the weight value of each draft lexical unit corresponding to each expert network; judging whether each expert network is activated based on the weight value of each draft lexical unit corresponding to each expert network, and statistically obtaining the activation count and total weight value corresponding to each expert network.

[0011] In some embodiments of the first aspect of this application, the process of thresholding several expert networks based on the activation count and total weight value corresponding to each expert network to obtain a dense expert network subset and a sparse expert network subset includes: calculating the activation frequency corresponding to each expert network based on the activation count corresponding to each expert network; comparing the activation frequency corresponding to each expert network with a preset frequency threshold; if the activation frequency of the current expert network is greater than or equal to the preset frequency threshold, then the current expert network is classified as a dense expert network; if the activation frequency of the current expert network is less than the preset frequency threshold, then the current expert network is classified as a sparse expert network; finally, a dense expert network subset and a sparse expert network subset are obtained.

[0012] In some embodiments of the first aspect of this application, the process of optimizing a sparse expert network subset includes: performing expert pruning on the sparse expert network subset to obtain a pruned sparse expert network subset; and performing sparsity regularization on the pruned sparse expert network subset to obtain a structured sparse expert block.

[0013] In some embodiments of the first aspect of this application, the method further includes: obtaining the calculation results of the TensorCore and CUDA Core respectively and performing weighted fusion to obtain the final multi-word output result.

[0014] In some embodiments of the first aspect of this application, the preset frequency threshold is obtained by setting it according to the throughput ratio of Tensor Cores and CUDA Cores.

[0015] To achieve the above and other related objectives, a second aspect of this application provides a multi-term prediction device based on a hybrid expert model using dense-sparse parallel computing, applied in a GPU including Tensor Cores and CUDA Cores. The device includes: a draft model processing module for processing input data of the hybrid expert model using a draft model to generate several draft terms; wherein the hybrid expert model includes a gated network and several expert networks; a gated network computation module for inputting the several draft terms into the gated network of the hybrid expert model for computation to obtain the activation count and total weight value corresponding to each expert network; a threshold partitioning module for performing threshold partitioning on the several expert networks based on the activation count and total weight value corresponding to each expert network to obtain a dense expert network subset and a sparse expert network subset; an optimization processing module for optimizing the sparse expert network subset to obtain an optimized sparse expert network subset; and a parallel processing module for sending the computation task of the dense expert network subset to the Tensor Core and the computation task of the optimized sparse expert network subset to the CUDA Core, so as to achieve parallel processing on both the Tensor Core and CUDA Core. Parallel processing of computational tasks is performed in the Core.

[0016] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the hybrid expert model multi-word prediction method based on dense-sparse parallel computing.

[0017] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, enables the computer to implement the hybrid expert model multi-word prediction method based on dense-sparse parallel computing.

[0018] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the hybrid expert model multi-word prediction method based on dense-sparse parallel computing.

[0019] As described above, the multi-word prediction method, apparatus, medium, program product, and terminal based on dense-sparse parallel computing hybrid expert model provided in this application have the following beneficial effects:

[0020] (1) Efficient integration of MTP and MoE significantly improves inference speed: Through the dense-sparse collaborative scheduling mechanism, parallel execution is achieved in Tensor Core and CUDA Core, which solves the problem of low hardware utilization caused by expert activation fragmentation when MTP and MoE are combined, and significantly improves the hardware efficiency and throughput performance of joint inference of MTP and MoE.

[0021] (2) No need to fine-tune traditional models, reducing deployment costs: No need to modify the structure of the MoE model or perform additional training. High-performance inference can be achieved simply by optimizing during operation, making it suitable for production scenarios where the model cannot be modified.

[0022] (3) Low precision loss and guaranteed generation quality: The dynamic pruning strategy based on activation frequency and gating weight effectively eliminates redundant calculations. While ensuring the accuracy of the output results, it significantly improves the model inference speed and ensures the stability and reliability of the model generation quality. Attached Figure Description

[0023] Figure 1 The diagram shown is a flowchart of a multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing, according to an embodiment of this application.

[0024] Figure 2 The diagram shows a flowchart of another multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing, as described in one embodiment of this application.

[0025] Figure 3 The diagram shown is a schematic representation of an architecture combining traditional multi-word prediction with a hybrid expert model in one embodiment of this application.

[0026] Figure 4 The diagram shown is a schematic representation of an architecture combining multi-word prediction and a hybrid expert model in one embodiment of this application.

[0027] Figure 5 The diagram shown is a structural schematic of a hybrid expert model multi-word prediction device based on dense-sparse parallel computing in one embodiment of this application.

[0028] Figure 6 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0029] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0030] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0031] <1> Mixture of Experts (MoE) models are neural network models designed to solve complex tasks on large-scale datasets. They adaptively combine multiple expert networks to handle different subsets of data, thereby improving the model's generalization ability and performance.

[0032] <2> GPU (Graphics Processing Unit): A processor originally designed for graphics processing but now widely used in massively parallel computing. Its architecture, consisting of thousands of small cores, gives it unparalleled speed and efficiency advantages over CPUs when processing parallelizable tasks. Especially in the fields of artificial intelligence and high-performance computing, GPUs have become an indispensable core computing power foundation.

[0033] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 This document illustrates a flowchart of a multi-word prediction method based on a hybrid expert model using dense-sparse parallel computation, as described in an embodiment of the present invention. The method in this embodiment is applied to GPUs including Tensor Cores and CUDA Cores, and includes:

[0034] Step S11: The input data of the hybrid expert model is processed using a draft model to generate several draft lexical units; wherein, the hybrid expert model includes a gating network and several expert networks.

[0035] The draft model is a lightweight draft model, which includes, but is not limited to, the EAGLE series models and the Llama series models. Preferably, in this embodiment, the draft model is EAGLE-3.

[0036] It's important to note that multi-terminal prediction implementations mainly fall into two categories. The first category is based on fine-tuning: these methods add multiple specialized prediction heads to the original model, enabling it to generate different terms in parallel during a single forward computation. However, to ensure generation quality, these methods require additional fine-tuning of both the original model and the newly added prediction heads to ensure they work collaboratively. While this approach is highly effective in accelerating inference, the cost of modifying the model structure and retraining may be unacceptable in some deployment scenarios.

[0037] The second category is the "draft-verification" method: This method introduces a lightweight "draft model" to quickly generate several candidate lexical units, which are then verified in parallel by the original main model. Unlike fine-tuning methods, it does not require modifying the original model structure or retraining the main model; it only requires training a small draft model with minimal computational resources. During inference, the draft model is responsible for making predictions, while the main model verifies them in parallel—accepting correct predictions and rejecting incorrect outputs. This method achieves significant speedup while maintaining output quality, making it particularly suitable for scenarios where the main model cannot be modified. Its efficiency stems from the asymmetric workload distribution between the draft model and the verification process: the lightweight draft model handles the rapid generation task, while the complex verification work is efficiently completed by the main model.

[0038] The hybrid expert model is an important architecture widely used in Large Language Models (LLMs). Its core idea is "on-demand computation": faced with different inputs, the model only activates a subset of sub-networks (called "experts"), instead of calling all parameters each time as in traditional LLMs. Specifically, each input first passes through a routing mechanism (also called a gating network), which determines which experts are best suited to handle the current content. Then, the system only calls the selected experts to perform computations and weights and integrates their results to form the final output. This approach breaks down the originally large and dense model into multiple relatively independent expert modules, significantly improving model performance without drastically increasing the computational burden during actual inference. For this reason, hybrid expert models are gaining increasing importance in large language models because they increase model capacity while improving computational efficiency.

[0039] The hybrid expert model comprises multiple expert networks. Its core idea is division of labor and collaboration, specifically by breaking down complex inference computation tasks into multiple inference computation sub-tasks, each processed by a different expert network. In the sparse hybrid expert model, only a subset of expert networks are activated for each inference computation sub-task.

[0040] In this embodiment, the expert network is typically a neural network. The algorithm structure of this neural network can be a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM) network, or a model built with an attention network, such as a transformer model, a bidirectional encoder representation from transformers (BERT) model, a contrastive language-image pre-training (CLIP) model, etc., and is not limited in this application.

[0041] GPUs play a crucial role in deep learning model training and large language model inference due to their excellent general-purpose parallel computing capabilities and compatible software ecosystem. Typically, GPUs have two types of computing cores: Tensor Cores and Scalar Cores (CUDA Cores). Tensor Cores accelerate the underlying matrix multiplication of operators, while CUDA Cores accelerate other underlying computations. Therefore, the purpose of this embodiment is to fully utilize the two heterogeneous computing cores, CUDA Cores and Tensor Cores, in parallel, reducing resource waste.

[0042] It should be explained that the inference computation tasks of the hybrid expert model include medical diagnosis, image classification, image reconstruction, and medical image analysis. When performing corresponding inference tasks using the hybrid expert model, a lightweight draft model is introduced for assistance. The lightweight draft model processes the input data of the inference task; these words serve as preliminary inference results, providing input for parallel processing of the hybrid expert model, and decomposing complex tasks into simpler sub-tasks, reducing the computational complexity of the hybrid expert model. The hybrid expert model can perform refined inference based on draft words, thereby accelerating the overall inference speed. Furthermore, there is contextual relevance between the various draft words; this relevance provides the hybrid expert model with richer contextual information, enabling it to more accurately understand the task requirements.

[0043] Step S12: Input several draft lexical units into the gating network of the hybrid expert model for calculation to obtain the activation count and total weight value of each expert network.

[0044] It should be noted that the draft lexical units obtained through the draft model retain key task information (such as contextual relevance and task type labels) while removing redundant data. The gated network is a lightweight routing or scheduling network used to analyze the input and determine which expert networks to send it to for processing.

[0045] In some examples, the process of inputting several draft words into the gating network of the hybrid expert model to obtain the activation count and total weight value of each expert network includes: inputting several draft words into the gating network of the hybrid expert model, the gating network calculating the weight value of each draft word for each expert network; judging whether each expert network is activated based on the weight value of each draft word for each expert network, and statistically obtaining the activation count and total weight value of each expert network.

[0046] Specifically, in the gating network of the hybrid expert model in this embodiment, the gating network receives and processes several draft words simultaneously. The expert networks activated by each draft word are different, but because there is contextual relevance between the several draft words, some or completely overlapping expert networks may be dynamically activated.

[0047] For example, suppose we obtain several draft lexical units, namely draft lexical unit 1, draft lexical unit 2, and draft lexical unit 3. The hybrid expert model includes expert network 1, expert network 2, expert network 3, expert network 4, expert network 5, and expert network 6. Inputting draft lexical unit 1, draft lexical unit 2, and draft lexical unit 3 into a gating network yields the following results;

[0048] The weight values ​​of each expert network corresponding to the draft term 1 are as follows: expert network 1 has a weight value of 0.9, expert network 2 has a weight value of 0.8, expert network 3 has a weight value of 0.7, expert network 4 has a weight value of 0.2, expert network 5 has a weight value of 0.2, and expert network 6 has a weight value of 0.1.

[0049] The weight values ​​for each expert network corresponding to draft term 2 are as follows: expert network 1 has a weight value of 0.9, expert network 2 has a weight value of 0.8, expert network 3 has a weight value of 0.7, expert network 4 has a weight value of 0.7, expert network 5 has a weight value of 0.2, and expert network 6 has a weight value of 0.1.

[0050] The weight values ​​for each expert network corresponding to draft term 3 are as follows: expert network 1 has a weight value of 0.9, expert network 2 has a weight value of 0.2, expert network 3 has a weight value of 0.2, expert network 4 has a weight value of 0.7, expert network 5 has a weight value of 0.8, and expert network 6 has a weight value of 0.1.

[0051] Furthermore, after obtaining the weight values ​​of each expert network corresponding to each draft lexical unit, the weight values ​​are compared with the baseline activation threshold. If the weight value of the current expert network is greater than or equal to the baseline activation threshold, it indicates that the expert network is activated; if the weight value of the current expert network is less than the baseline activation threshold, it indicates that the expert network is not activated. The baseline activation threshold is set according to the actual inference task and is not limited here.

[0052] In this embodiment, after obtaining the weight values ​​of each expert network, the weight values ​​of each expert network can be sorted from high to low, and the expert networks with the top K weight values ​​can be selected as the activated expert networks. For example, the top 8 expert networks with the highest weight values ​​can be selected as the activated expert networks. The method for determining whether an expert network is activated is not limited in this embodiment and can be selected according to actual needs.

[0053] For example, with the baseline activation threshold set to 0.7, it can be seen that the expert network activated by draft lexical 1 includes expert network 1, expert network 2, and expert network 3; the expert network activated by draft lexical 2 includes expert network 1, expert network 2, expert network 3, and expert network 4; and the expert network activated by draft lexical 3 includes expert network 1, expert network 4, and expert network 5.

[0054] After determining whether each expert network is activated based on the weight value corresponding to each draft term, the activation count of each expert network is obtained. For example, expert network 1 is activated 3 times, expert network 2 is activated 2 times, expert network 3 is activated 2 times, expert network 4 is activated 2 times, expert network 5 is activated 1 time, and expert network 6 is activated 0 times.

[0055] Then, the total weight value is calculated based on the weight values ​​of each expert network corresponding to each draft term. For example, the total weight value of expert network 1 is 0.9 + 0.9 + 0.9 = 2.7; the total weight value of expert network 2 is 0.8 + 0.8 + 0.2 = 1.8; the total weight value of expert network 3 is 0.7 + 0.7 + 0.2 = 1.6; and so on, to obtain the total weight value of each expert network. It is understandable that an expert network activated three times with a weight value of 0.8 each time is significantly denser than an expert network activated once with a weight value of 0.9.

[0056] Step S13: Based on the activation count and total weight value of each expert network, perform threshold partitioning on several expert networks to obtain a dense expert network subset and a sparse expert network subset.

[0057] It's important to note that during the inference process of a hybrid expert model, the activation of the expert network is not randomly distributed but exhibits a clear concentration, meaning that a small number of experts undertake the majority of the activation tasks. This skewed distribution is particularly pronounced in multi-word prediction scenarios. This is because there is usually contextual association between the draft words, leading to highly overlapping expert subsets dynamically activating similar inputs. For example, a small number of expert networks (e.g., 7.8%) may account for as much as 70% of the total activations.

[0058] This embodiment utilizes the skewed characteristics of the expert network activation distribution to dynamically partition the expert network, enabling parallel computation in hardware and significantly improving hardware utilization, thereby increasing the overall computation speed. Specifically, all expert networks are partitioned based on indicators such as the number of activations and the total weight value, ultimately resulting in a dense expert network subset and a sparse expert network subset. The dense expert network subset contains expert networks with high-frequency activations, while the sparse expert network subset consists of low-frequency or low-weight expert networks.

[0059] In some examples, the process of thresholding several expert networks based on the activation count and total weight value of each expert network to obtain dense and sparse expert network subsets includes: calculating the activation frequency of each expert network based on the activation count; comparing the activation frequency of each expert network with a preset frequency threshold; if the activation frequency of the current expert network is greater than or equal to the preset frequency threshold, the current expert network is classified as a dense expert network; if the activation frequency of the current expert network is less than the preset frequency threshold, the current expert network is classified as a sparse expert network; finally, dense and sparse expert network subsets are obtained. The preset frequency threshold is obtained by setting it based on the throughput ratio of Tensor Cores and CUDA Cores.

[0060] It's important to explain that Tensor Cores and CUDA Cores are the computing cores in GPU hardware. Tensor Cores correspond to dense computing units, while CUDA Cores correspond to sparse computing units. By analyzing the throughput ratio of Tensor Cores and CUDA Cores in the current GPU, a preset frequency threshold is dynamically adjusted based on their throughput characteristics. The limit of the preset frequency threshold is set at a point that balances the throughput of Tensor Cores and CUDA Cores, ensuring a balanced load for the two types of computing units. For example, if the throughput ratio of Tensor Cores to CUDA Cores on the current GPU is obtained in real time, and Tensor Cores are significantly faster than CUDA Cores, the preset frequency threshold will be lower, aiming to allow more expert networks to enter the dense expert network subset. The setting of the preset frequency threshold depends on the actual GPU used and is not limited here.

[0061] In some examples, the process of thresholding several expert networks based on the activation count and total weight value of each expert network to obtain dense and sparse expert network subsets includes: comparing the total weight value of each expert network with a preset weight threshold; if the total weight value of the current expert network is greater than or equal to the preset weight threshold, the current expert network is classified as a dense expert network; if the total weight value of the current expert network is less than the preset weight threshold, the current expert network is classified as a sparse expert network; finally, dense and sparse expert network subsets are obtained. The setting method of the preset weight threshold is similar to that of the preset frequency threshold, and will not be elaborated here.

[0062] In this embodiment, the feature metric for thresholding several expert networks can be the activation frequency of each expert network or the total weight value of each expert network. After dividing the expert networks into dense and sparse subsets, they are sent to Tensor Core and CUDA Core respectively to perform parallel computing tasks, thereby improving hardware utilization and overall computing speed. Experimental data shows that the inference process is accelerated by 1.13 times in GPU hardware.

[0063] Step S14: Optimize the sparse expert network subset to obtain an optimized sparse expert network subset.

[0064] In some examples, the process of optimizing a sparse expert network subset includes: performing expert pruning on the sparse expert network subset to obtain a pruned sparse expert network subset; and performing sparsity regularization on the pruned sparse expert network subset to obtain a structured sparse expert block.

[0065] It should be noted that further processing of the sparse expert network subset, specifically expert pruning, involves removing low-frequency or low-weight expert networks. When an expert network with an activation frequency or total weight value below a preset removal threshold exists within the sparse expert network subset, this network is removed, effectively eliminating redundant computations. For example, removing an expert network with only one activation has almost no impact on the overall inference task output.

[0066] For example, in practical applications, approximately 90% of expert networks contribute less than 30% of the total activations. Therefore, expert networks with extremely low activation frequencies or very small weights are removed. This adaptive pruning strategy effectively eliminates redundant computations, significantly reduces CUDA Core latency during computation, and has almost no impact on output quality. The impact of low-frequency, low-weight expert networks on the final output of the model is negligible. Experimental data shows that expert pruning optimizes the dense-sparse computational balance and achieves a 1.21x speedup in hardware processing with less than a 0.2% loss in output accuracy.

[0067] Furthermore, after obtaining the pruned sparse expert network subset, it needs to undergo sparse regularization to obtain structured sparse expert blocks. It should be noted that unstructured sparse expert networks lead to serial control flow and irregular memory accesses, which conflicts with the dense-sparse hybrid parallel computing requirements of this embodiment. Structured activation regularization transforms the computational tasks of each sparse expert network subset into parallelizable computational tasks. Specifically, the computational tasks of the sparse expert network subset are divided into fixed-size matrix blocks, ensuring that each computational block is stored contiguously in memory, eliminating cache misses caused by random memory accesses.

[0068] In this embodiment, irregular sparse patterns are reconstructed into hardware-friendly, continuous computational blocks, and the sparse computation of the sparse expert network subset is aligned with the processing power of the dense computational units (Tensor Cores). Experimental data show that this design enables Tensor Cores (for dense computation) and CUDA Cores (for sparse computation) to work efficiently together at the hardware level, achieving a 4.71x speedup in hardware processing.

[0069] Step S15: Send the computation tasks of the dense expert network subset to the Tensor Core and the computation tasks of the optimized sparse expert network subset to the CUDA Core, so as to perform parallel processing of the computation tasks in the Tensor Core and the CUDA Core.

[0070] In this embodiment, the computation tasks of the dense expert network subset are sent to the Tensor Core for processing, and the computation tasks of the optimized sparse expert network subset are sent to the CUDA Core for processing. The Tensor Core and CUDA Core process in parallel.

[0071] like Figure 2 As shown, step S16: Obtain the calculation results of the Tensor Core and CUDA Core respectively and perform weighted fusion to obtain the final multi-word output result.

[0072] Specifically, after parallel execution, the Tensor Core and CUDA Core each output their computation results. Then, the computation results of the Tensor Core and CUDA Core are weighted and fused according to the original weights of the gating network to generate the final multi-term output. In the subsequent validation stage, correctly confirmed terms by the hybrid expert model are retained, while incorrect predictions are discarded to ensure output quality.

[0073] Combination Figure 3 and Figure 4 illustrate, Figure 3 It is a traditional draft model and a hybrid expert model architecture. Figure 4 This application presents an architecture combining a draft model and a hybrid expert model. After introducing multi-terminal prediction, the number of activations in the expert networks significantly increases. Therefore, the serially executed expert computation is replaced with a dense-sparse partitioning and parallel execution mechanism. The specific process of this architecture is as follows: When draft terms enter the hybrid expert model, the gating network first determines the activation state of each expert network and its corresponding weights. Then, based on the GPU hardware, the expert networks are dynamically divided into dense and sparse expert network subsets. Next, the sparse expert network subset is further pruned to remove low-frequency, low-weight experts, and sparse regularization is applied to reduce control overhead and redundant memory access. Finally, the dense expert network subset is processed by TensorCores, and the processed sparse expert network subset is processed by CUDA Cores, achieving parallel execution. This significantly improves the computational efficiency of the joint deployment of multi-terminal prediction and the hybrid expert model without reducing accuracy.

[0074] To facilitate understanding of the multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing proposed in this application, the following specific embodiments are provided for illustration. These specific embodiments are illustrated as follows... Figure 4 The framework is implemented in the process of large language model inference that supports hybrid expert models. The example is the deployment of the Qwen-MoE model on an NVIDIA A800 GPU combined with EAGLE-3's multi-word prediction technology.

[0075] Before using it, you need to adapt the hybrid expert model to EAGLE-3. EAGLE-3 currently only supports non-MoE models such as the LLaMA series and Qwen series. The use of MoE models requires manual adaptation. For specific adaptation methods, please refer to the official EAGLE-3 GitHub repo, which will not be elaborated here.

[0076] Step 1: Input data processing and routing stage.

[0077] After the user inputs data (such as prompt words), a "draft-verification" multi-term prediction strategy is used for processing. The specific process is as follows: First, a lightweight draft model (using EAGLE-3 or a model trained using EAGLE-3 code) generates multiple draft terms (e.g., 128 draft terms), forming a batch input. This batch of 128 draft terms is fed into the routing module (gating network) of the hybrid expert model's main model. The routing module independently calculates the gating weights (weight values) of each draft term for all expert networks and determines whether each expert network is activated.

[0078] Step 2: Expert activation analysis and dense-sparse partitioning stage.

[0079] The activation count and total weight of each expert network in this batch are counted. Then, based on the offline preset hardware awareness, the preset frequency threshold is divided (the preset frequency threshold is dynamically calibrated according to the throughput ratio of Tensor Core and CUDA Core). The experts are divided into two categories: (1) Dense expert network subset: experts with high activation frequency (such as activation frequency exceeding 6.25%) are handed over to Tensor Core for parallel computation in the form of dense matrix; (2) Sparse expert network subset: the remaining low frequency or low weight experts are retained in the sparse execution path and are processed by CUDA Core.

[0080] Step 3: Sparse subset optimization stage.

[0081] Two optimizations are implemented for the sparse subset: (1) Expert pruning: remove those with extremely low activation frequency (such as only 1 activation in a batch) to reduce redundant computation; (2) Sparse regularization: align the computation tasks of the remaining sparse experts according to the hardware warp size, reorganize them into structured sparse blocks, eliminate irregular memory access, and improve the utilization of CUDA Core.

[0082] Step 4: Parallel execution and result fusion stage.

[0083] The computational tasks for the dense and sparse expert network subsets are submitted to the Tensor Core and CUDA Core for parallel execution, respectively. The results from both are then weighted and fused according to the original routing weights to generate the final multi-term output. In the subsequent validation stage, only the terms correctly confirmed by the main model are retained, while incorrect predictions are discarded, thus ensuring output quality.

[0084] It should be noted that the entire process requires no modification to the MoE model structure or additional fine-tuning; all optimizations are performed dynamically during inference runtime. Through the above implementation method, this invention significantly improves the hardware efficiency and throughput performance of joint inference between MTP and MoE while maintaining model accuracy.

[0085] Meanwhile, experimental verification on an NVIDIA A800 GPU shows that this application, as Figure 4 The multi-term prediction-hybrid expert model architecture shown successfully solves the key problem between multi-term prediction and hybrid expert model architectures. Compared to direct multi-term prediction-hybrid expert model integration schemes (i.e., the traditional simple combination of EAGLE-3 and a hybrid expert model), the framework of this application achieves a 5.44x end-to-end speedup, and compared to the standard implementation in HuggingFace, it improves inference speed by 3.94x. Furthermore, after introducing low-frequency expert pruning, the multi-term prediction-hybrid expert model architecture further improves the above speedups to 6.81x and 4.53x respectively, with an accuracy loss of only 0.2%. This demonstrates that this application provides a practical path for the efficient collaborative deployment of MTP and MoE, without relying on costly model fine-tuning.

[0086] It should be emphasized that the multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing provided in this application has the following beneficial effects:

[0087] (1) Efficient integration of MTP and MoE significantly improves inference speed: Through a dense-sparse collaborative scheduling mechanism, parallel execution is achieved in Tensor Core and CUDA Core, which greatly improves hardware utilization and improves overall speed. It solves the problem of low hardware utilization caused by expert activation fragmentation when MTP and MoE are combined, and significantly improves the hardware efficiency and throughput performance of joint inference of MTP and MoE.

[0088] (2) No need to fine-tune traditional models, reducing deployment costs: No need to modify the structure of the MoE model or perform additional training. High-performance inference can be achieved simply by optimizing during operation, making it suitable for production scenarios where the model cannot be modified.

[0089] (3) Low precision loss and guaranteed generation quality: The dynamic pruning strategy based on activation frequency and gating weight effectively eliminates redundant calculations. While ensuring the accuracy of the output results, it significantly improves the model inference speed and ensures the stability and reliability of the model generation quality.

[0090] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect, without limiting their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0091] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0092] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0093] Figure 5 The schematic block diagram of the hybrid expert model multi-word prediction device based on dense-sparse parallel computing provided in this application embodiment is applied to a GPU including Tensor Cores and CUDA Cores, such as... Figure 5 As shown, the device 500 includes:

[0094] The draft model processing module 510 is used to process the input data of the hybrid expert model using a draft model to generate several draft lexical units; wherein, the hybrid expert model includes a gating network and several expert networks;

[0095] The gated network calculation module 520 is used to input several draft lexical units into the gated network of the hybrid expert model for calculation, so as to obtain the activation count and total weight value of each expert network.

[0096] The threshold partitioning module 530 is used to perform threshold partitioning on several expert networks based on the activation count and total weight value of each expert network, so as to obtain a dense expert network subset and a sparse expert network subset.

[0097] The optimization processing module 540 is used to optimize the sparse expert network subset to obtain the optimized sparse expert network subset.

[0098] The parallel processing module 550 is used to send computation tasks of dense expert network subsets to Tensor Cores and computation tasks of optimized sparse expert network subsets to CUDA Cores, so as to perform parallel processing of computation tasks in Tensor Cores and CUDA Cores.

[0099] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0100] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0101] Figure 6 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 6 As shown, the electronic terminal includes at least one processor 601, a memory 602, at least one network interface 603, and a user interface 605. The various components in the device are coupled together via a bus system 604. It is understood that the bus system 604 is used to implement communication between these components. In addition to a data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 6 The general will label all buses as bus systems.

[0102] The user interface 605 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0103] It is understood that memory 602 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0104] In this embodiment of the invention, the memory 602 is used to store various types of data to support the operation of the electronic terminal 600. Examples of this data include: any executable program for operation on the electronic terminal 600, such as the operating system 6021 and application programs 6022; the operating system 6021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 6022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The implementation of the hybrid expert model multi-word prediction method based on dense-sparse parallel computing provided in this embodiment of the invention can be included in the application program 6022.

[0105] The methods disclosed in the above embodiments of the present invention can be applied to processor 601, or implemented by processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 601 or by instructions in the form of software. The processor 601 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 601 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 601 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0106] In an exemplary embodiment, the electronic terminal 600 may be used to execute the aforementioned method by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs).

[0107] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute the hybrid expert model multi-word prediction method based on dense-sparse parallel computing in any of the embodiments shown.

[0108] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to execute the hybrid expert model multi-word prediction method based on dense-sparse parallel computing in any of the embodiments shown.

[0109] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0110] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0111] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0115] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0116] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0118] In summary, the multi-terminal prediction method, apparatus, medium, program product, and terminal based on dense-sparse parallel computing hybrid expert model provided in this application include: processing the input data of the hybrid expert model using a draft model to generate several draft terms; wherein the hybrid expert model includes a gating network and several expert networks; inputting the several draft terms into the gating network of the hybrid expert model for computation to obtain the activation count and total weight value corresponding to each expert network; performing threshold partitioning on the several expert networks based on the activation count and total weight value corresponding to each expert network to obtain a dense expert network subset and a sparse expert network subset; optimizing the sparse expert network subset to obtain an optimized sparse expert network subset; sending the computation task of the dense expert network subset to TensorCore and the computation task of the optimized sparse expert network subset to CUDA Core for parallel processing of computation tasks in TensorCore and CUDA Core.

[0119] This application achieves parallel execution in Tensor Cores and CUDA Cores through a dense-sparse collaborative scheduling mechanism, solving the problem of low hardware utilization caused by expert activation fragmentation when combining MTP and MoE, and significantly improving the hardware efficiency and throughput performance of joint MTP and MoE inference. This application does not require modification of the MoE model structure or additional training; high-performance inference can be achieved solely through in-process optimization, making it suitable for production scenarios where model modification is not possible. Based on a dynamic pruning strategy using activation frequency and gating weights, this application effectively eliminates redundant computations, significantly improving model inference speed while ensuring output accuracy, and guaranteeing the stability and reliability of model generation quality. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0120] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing, characterized in that, Applied to GPUs including Tensor Cores and CUDA Cores, the method includes: The input data of the hybrid expert model is processed using a draft model to generate several draft lexical units; wherein, the hybrid expert model includes a gating network and several expert networks; Several draft lexical units are input into the gating network of the hybrid expert model for calculation to obtain the activation count and total weight value of each expert network. Based on the number of activations and total weight value of each expert network, several expert networks are divided into dense expert network subsets and sparse expert network subsets. The sparse expert network subset is optimized to obtain an optimized sparse expert network subset. The computation tasks of the dense expert network subset are sent to the Tensor Core, and the computation tasks of the optimized sparse expert network subset are sent to the CUDA Core, so that the computation tasks can be processed in parallel in the Tensor Core and the CUDA Core.

2. The multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing according to claim 1, characterized in that, The process of inputting several draft lexical units into the gating network of the hybrid expert model for computation to obtain the activation count and total weight value for each expert network includes: After inputting several draft words into the gating network of the hybrid expert model, the gating network calculates the weight value of each expert network corresponding to each draft word. The system determines whether each expert network is activated based on the weight values ​​of each draft term corresponding to each expert network, and calculates the activation count and total weight value of each expert network.

3. The multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing according to claim 1, characterized in that, The process of thresholding several expert networks based on the activation count and total weight value of each expert network to obtain dense and sparse expert network subsets includes: The activation frequency of each expert network is calculated based on the number of activations for each expert network, and then compared with a preset frequency threshold. If the activation frequency of the current expert network is greater than or equal to the preset frequency threshold, then the current expert network is classified as a dense expert network. If the activation frequency of the current expert network is less than the preset frequency threshold, the current expert network is classified as a sparse expert network. Ultimately, dense expert network subsets and sparse expert network subsets are obtained.

4. The multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing according to claim 1, characterized in that, The process of optimizing a subset of sparse expert networks includes: Expert pruning is performed on the sparse expert network subset to obtain the pruned sparse expert network subset. Sparse regularization is applied to the pruned sparse expert network subset to obtain structured sparse expert blocks.

5. The multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing according to claim 1, characterized in that, The method further includes: obtaining the calculation results of the Tensor Core and CUDA Core respectively and performing weighted fusion to obtain the final multi-term output result.

6. The multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing according to claim 3, characterized in that, The preset frequency threshold is obtained by setting it based on the throughput ratio of Tensor Cores and CUDA Cores.

7. A multi-word prediction device based on a hybrid expert model using dense-sparse parallel computing, characterized in that, The device, applicable to GPUs including Tensor Cores and CUDA Cores, comprises: The draft model processing module is used to process the input data of the hybrid expert model using a draft model to generate several draft lexical units; wherein, the hybrid expert model includes a gating network and several expert networks; The gated network computation module is used to input several draft lexical units into the gated network of the hybrid expert model for computation, so as to obtain the activation count and total weight value of each expert network. The threshold partitioning module is used to partition several expert networks based on the number of activations and the total weight value corresponding to each expert network, so as to obtain a dense expert network subset and a sparse expert network subset. The optimization processing module is used to optimize the sparse expert network subset to obtain the optimized sparse expert network subset. The parallel processing module is used to send computation tasks of dense expert network subsets to Tensor Cores and computation tasks of optimized sparse expert network subsets to CUDA Cores, so as to perform parallel processing of computation tasks in Tensor Cores and CUDA Cores.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-word prediction method based on a hybrid expert model using dense-sparse parallel computing as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, enables the computer to implement the hybrid expert model multi-word prediction method based on dense-sparse parallel computing as described in any one of claims 1 to 6.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the hybrid expert model multi-word prediction method based on dense-sparse parallel computing as described in any one of claims 1 to 6.

Images