Hybrid expert model expert network processing method, device, equipment and medium thereof

Abstract

The present disclosure provides an expert network processing method of a hybrid expert model and an apparatus, device and medium thereof. The method comprises: firstly, configuring a plurality of thread bundles corresponding to each execution unit according to a plurality of expert networks of the hybrid expert model, so as to distribute expert scores of each expert network to each thread of the thread bundle; then, in response to a reduction operation instruction, performing a cyclic reduction operation on the expert scores held by each thread in each thread bundle through a load storage unit, until a plurality of target expert scores are obtained; and finally, determining a plurality of target expert networks corresponding to the plurality of target expert scores from the plurality of expert networks. Through the reasonable configuration of the thread bundles of the execution unit in the plurality of expert networks of the hybrid expert model, the expert scores are distributed to each thread of the thread bundle, the cyclic reduction operation can be performed through the load storage unit by using the reduction operation instruction, the number of instructions and the register resources required by the Top-K algorithm of the hybrid expert model are reduced, and the calculation efficiency is improved.

Description

Technical Field

[0001] This disclosure relates to the field of chips, and in particular to an expert network processing method, apparatus, device and medium for hybrid expert models. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence and deep learning, the scale and complexity of large language models based on the Transformer architecture are constantly increasing, leading to a surge in computational resource consumption during model inference and training. To further improve model performance, the scale of model parameters is continuously expanding. However, simply increasing the model's depth and width makes training and inference extremely expensive, thus various hardware chips are widely used in model inference and training tasks. Against this backdrop, the Mixture of Experts (MoE) architecture has emerged and become one of the key technologies for building large-scale models. The MoE decomposes the model into multiple smaller expert networks and a gating network. For each input sequence unit (token), the gating network calculates the affinity score between that token and all N expert networks. Then, from these N scores, the top K experts (i.e., Top-K) are selected, and the token's information is routed only to these K selected experts for computation, while the remaining experts remain inactive. This achieves an effective balance between model size and computational efficiency.

[0003] However, in hybrid expert model architectures, the Top-K expert selection problem gradually becomes prominent. This problem occurs frequently in the forward computation of each layer of the model, with an extremely high execution frequency, thus placing high demands on the execution efficiency of the hardware chips used to deploy the model. When the number of expert networks increases significantly, how to quickly and in parallel find the K highest scores from these N expert networks becomes a key bottleneck restricting the overall performance of hybrid expert models. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and medium for processing expert networks using hybrid expert models, which reduces the number of instructions and register resources required for the Top-K algorithm of hybrid expert models and improves computational efficiency.

[0005] According to one aspect of this disclosure, a method for expert network processing using a hybrid expert model is provided, characterized in that it is applied to a computing core, wherein the hybrid expert model is deployed on the computing core, the computing core includes a loading storage unit and a plurality of execution units, the loading storage unit and the plurality of execution units being communicatively connected, and the method includes:

[0006] According to the multiple expert networks of the hybrid expert model, configure multiple thread bundles corresponding to each execution unit to allocate the expert score of each expert network to each thread of the thread bundle;

[0007] In response to the reduction operation instruction, the expert score held by each thread in each thread bundle is cyclically reduced through the loading storage unit until multiple target expert scores are obtained;

[0008] From the plurality of expert networks, determine the plurality of target expert networks corresponding to the plurality of target expert scores.

[0009] In some embodiments, the load storage unit includes a reduction unit and an atomic operation unit, and the cyclic reduction operation performed on each thread bundle through the load storage unit includes:

[0010] The reduction unit performs a first reduction process on the expert score held by each thread in each thread bundle to obtain the first reduction result for each thread bundle.

[0011] The atomic operation unit performs a second reduction process on the first reduction result of each thread bundle to obtain and store the target expert score;

[0012] For target threads holding the aforementioned target expert score, data recording and data removal processes are performed.

[0013] In some embodiments, the reduction unit includes a maximum value reduction tree, and the computational core further includes a register area corresponding to each thread bundle, the register area including a register corresponding to each thread in the thread bundle. The step of performing a first reduction process on the expert score held by each thread in each thread bundle through the reduction unit to obtain a first reduction result for each thread bundle includes:

[0014] The expert score stored in the register corresponding to each thread in each thread bundle is obtained through the maximum value reduction tree;

[0015] The maximum value reduction is performed on all the expert scores of the thread bundle to obtain the local maximum value of the expert score, and the local maximum value of the expert score is used as the first reduction result of the thread bundle.

[0016] In some embodiments, the maximum reduction tree includes a plurality of maximum value comparators, the number of which is determined based on the total number of threads in each thread bundle.

[0017] In some embodiments, the load storage unit further includes shared memory, the atomic operation unit includes an arithmetic logic unit, and the step of performing a second reduction process on the first reduction result of each thread bundle through the atomic operation unit to obtain and store the target expert score includes:

[0018] The arithmetic logic unit obtains the first reduction result of each thread bundle to obtain the maximum value of multiple local expert scores;

[0019] The arithmetic logic unit performs an atomic maximum value operation on all the local expert score maximum values ​​to calculate the global expert score maximum value. The global expert score maximum value is then used as the target expert score and stored in the shared memory.

[0020] In some embodiments, the shared memory includes a plurality of storage addresses, the number of which is determined based on the total number of threads in each thread bundle, and storing the target expert score in the shared memory includes:

[0021] The target expert score is stored in the target storage address by the arithmetic logic unit. The target storage address is either the storage address corresponding to the target thread that holds the target expert score or a storage address in the shared memory specifically used for storing expert scores.

[0022] In some embodiments, the data storage and data culling process for the target thread holding the target expert score includes:

[0023] For the target thread holding the target expert score, the target thread index and target expert index stored in the target register corresponding to the target thread are stored in the shared memory, and the score value of the target expert score stored in the target register is set to a preset minimum value.

[0024] In some embodiments, determining the plurality of target expert networks corresponding to the plurality of target expert scores from the plurality of expert networks includes:

[0025] Obtain the target expert indexes corresponding one-to-one with the target expert scores from the shared memory;

[0026] Based on the multiple target expert indices, determine the multiple target expert networks corresponding to the multiple target expert scores from the multiple expert networks.

[0027] According to one aspect of this disclosure, an expert network processing apparatus based on a hybrid expert model is proposed, the apparatus comprising:

[0028] A configuration unit is configured to configure multiple thread bundles corresponding to each execution unit according to multiple expert networks of the hybrid expert model, so as to allocate the expert score of each expert network to each thread of the thread bundle;

[0029] A reduction operation unit is configured to, in response to a reduction operation instruction, perform a cyclic reduction operation on the expert scores held by each thread in each thread bundle through the loading storage unit until multiple target expert scores are obtained;

[0030] An expert network determination unit is used to determine, from the plurality of expert networks, a plurality of target expert networks corresponding to the plurality of target expert scores.

[0031] According to one aspect of this disclosure, an electronic device is proposed, the electronic device including a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection communication between the processor and the memory, wherein the program is executed by the processor to implement the expert network processing method of the hybrid expert model as described above.

[0032] According to one aspect of this disclosure, a computer-readable storage medium is proposed that stores one or more programs, which can be executed by one or more processors to implement the expert network processing method of the hybrid expert model as described above.

[0033] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program or computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, causing the computer device to perform the expert network processing method of the hybrid expert model as described above.

[0034] This disclosure presents a method, apparatus, device, and medium for processing expert networks in a hybrid expert model. The method includes: first, configuring multiple thread bundles corresponding to each execution unit based on multiple expert networks in the hybrid expert model, so as to allocate the expert score of each expert network to each thread of the thread bundle; then, in response to a reduction operation instruction, performing a cyclic reduction operation on the expert score held by each thread in each thread bundle by loading a storage unit, until multiple target expert scores are obtained; finally, determining multiple target expert networks corresponding to the multiple target expert scores from the multiple expert networks. In this disclosure, by reasonably configuring the thread bundles of the execution unit in the multiple expert networks of the hybrid expert model and allocating the expert score to each thread of the thread bundle, the cyclic reduction operation can be performed by loading a storage unit using a reduction operation instruction, reducing the number of instructions and register resources required for the Top-K algorithm of the hybrid expert model, and improving computational efficiency.

[0035] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0036] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0037] Figure 1 This is a system architecture diagram of the expert network processing method using a hybrid expert model according to embodiments of the present disclosure;

[0038] Figure 2 This is a main flowchart of an expert network processing method for a hybrid expert model according to an embodiment of the present disclosure;

[0039] Figure 3 This is an embodiment of the present disclosure. Figure 2 A flowchart of a sub-step of one embodiment of step S200;

[0040] Figure 4 This is a schematic diagram illustrating the use of a loading memory unit to perform a reduction operation according to an embodiment of this disclosure;

[0041] Figure 5 This is an embodiment of the present disclosure. Figure 4 Schematic diagram of the reduced unit structure;

[0042] Figure 6 This is an embodiment of the present disclosure. Figure 3 A flowchart of a sub-step of one embodiment of step S210;

[0043] Figure 7 This is an embodiment of the present disclosure. Figure 4 A schematic diagram of the structure of a middle atomic operation unit;

[0044] Figure 8 This is an embodiment of the present disclosure. Figure 3 A flowchart of a sub-step of one embodiment of step S220;

[0045] Figure 9 This is a schematic diagram of the structure of an expert network processing device for a hybrid expert model according to an embodiment of the present disclosure;

[0046] Figure 10 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0048] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0049] Chip: A chip typically refers to an integrated circuit, a miniature electronic device that integrates circuitry onto a tiny silicon wafer. Chips are widely used in various electronic devices, such as computers, mobile phones, and automobiles, to perform various functions, such as computing, storage, and communication. A chip is a packaged, bare die that can be used in actual electronic systems. A chip may include a command processor and multiple block processing modules, each of which is connected to the command processor via data communication.

[0050] A Graphics Processing Unit (GPU) is a microprocessor specifically designed for parallel processing of large amounts of graphics and general computing tasks. Its core function is as a processor designed for highly parallel computing tasks, excelling at handling a large number of simple, repetitive computational operations simultaneously. Originally designed to accelerate image rendering, its powerful parallel computing capabilities have led to its widespread application in scientific computing, artificial intelligence, deep learning, physics simulations, and other fields.

[0051] Compute Unit (CU): In a graphics processing unit (GPU) architecture, a compute unit is the basic unit for executing computational tasks. A workgroup running on a GPU actually executes on a compute unit. A compute unit is the basic unit on a GPU used to execute parallel computing tasks; it contains a certain number of processing elements that can execute multiple threads in parallel. A block processing cluster can include multiple compute units. In this embodiment, synchronization at the CU layer refers to synchronization within the same CU, synchronization between different workgroups within a CU, or synchronization between different kernels, and is achieved through data packets and kernels during the synchronization process. A compute unit can include a workgroup cluster data cache, a synchronization engine, a segmentation module, a task scheduling module, and multiple execution units. The workgroup cluster data cache receives information sent by the command computing core; the synchronization engine parses and synchronizes the information in the workgroup cluster data cache to obtain task information; and the segmentation module segments the task information and sends the corresponding task information to the execution units for task execution.

[0052] Streaming Multiprocessors (SMs) are the core computing units of a GPU. They consist of multiple stream processors and related resources, responsible for executing a large number of threads in parallel to achieve efficient graphics rendering and computation tasks. An SM is composed of multiple stream processors (SPs). It also includes shared memory, registers, control units, instruction caches, and other resources. These components work together to support meta-threaded applications. SMs schedule threads in units of warps, with each warp typically containing 32 threads. Threads within the same warp execute the same instructions using Single Instruction Multiple Threading (SIMT) but process different data. This mechanism allows SMs to efficiently handle a large number of parallel tasks, such as rendering multiple pixels in an image. Its core function is parallel computing, primarily used for tasks such as vertex processing and pixel shading in graphics rendering. Simultaneously, with technological advancements, SMs are also widely used in non-graphics fields such as artificial intelligence training and big data analysis, fully leveraging their powerful parallel processing capabilities. SMs are the core modules of a GPU, and multiple SMs constitute the GPU's computing portion. Different GPU models have different numbers of SMs, and the number and performance of SMs directly affect the GPU's computing power and graphics processing speed.

[0053] Execution Unit (EU): The execution unit in a microprocessor is responsible for executing instructions and performing computational tasks such as arithmetic operations, logical operations, and shift operations. In fact, it has the functions of both a controller and an arithmetic unit.

[0054] Load Store Unit (LSU): A key component in processor architecture, primarily responsible for handling data transfer between processor registers and memory. The LSU is designed to efficiently manage data loading and storage operations to support the processor's computational tasks.

[0055] The Arithmetic Logic Unit (ALU) is a core component in the processor architecture that performs basic arithmetic and logical operations. The ALU is responsible for processing integer and floating-point calculation instructions (such as addition, subtraction, multiplication, division, and bitwise operations), and is an indispensable functional unit in the CPU / GPU. Based on differences in data width, the ALU can be divided into scalar ALUs and vector ALUs.

[0056] Shared memory refers to a certain amount of memory in a multiprocessor computer system that can be accessed by different processors. Because multiple processors need to access memory quickly, memory caching is necessary. When any cached data is updated, the shared memory needs to be updated immediately, otherwise different processors might use different data.

[0057] Mixture of Experts (MoE) models are an ensemble learning method that combines multiple expert models. The core idea is to dynamically select or weight the outputs of different expert models based on the characteristics of the input data, thereby improving the overall model's expressive power and generalization performance. In other words, MoE models introduce multiple sub-networks (i.e., expert networks) and dynamically select some expert networks to participate in inference based on the characteristics of the input task, thus achieving on-demand allocation and efficient utilization of computing resources. This dynamic selection mechanism allows the model to flexibly adjust the allocation of computing resources when processing different types of data, thereby improving overall performance. MoE models are widely used in natural language processing, computer vision, and other fields, especially in large-scale models (such as DeepSeek, GPT-4, and Switch Transformer) to improve parameter utilization and inference efficiency.

[0058] Thread Bundle: A thread bundle is the basic execution unit in a Streaming Multiprocessor (SM). When a grid is started (grid startup is equivalent to kernel startup, with each kernel corresponding to its own grid), the grid contains thread blocks. After a thread block is assigned to a particular SM, it is divided into multiple thread bundles. Within a thread bundle, all threads execute in a single-instruction multithreaded manner, executing the same instructions at each step, but processing private data, i.e., the data corresponds to a specific thread.

[0059] A thread is the smallest unit of execution scheduled by the operating system. It is a control flow within a process and has its own stack, program counter, and register state, but shares the resources of the same process (such as memory, file handles, global variables, etc.).

[0060] An atomic operation is an indivisible operation that ensures data integrity in a multithreaded environment. An atomic operation cannot be interrupted by other threads during execution, ensuring the consistency of shared data state. The instruction set that performs an atomic operation can refer to an uninterruptible instruction execution process.

[0061] In the field of parallel computing, multi-threaded concurrent execution technology is widely used in computing devices such as graphics processing units (GPUs) and artificial intelligence accelerators to improve processing efficiency. These computing devices typically require basic computing components (such as arithmetic logic units) to perform core functions such as control flow management, data scheduling, and auxiliary calculations, and distribute and coordinate computing tasks through control thread bundles.

[0062] With the rapid development of technologies such as artificial intelligence and deep learning, the scale and complexity of large language models based on the Transformer architecture are constantly increasing, leading to a surge in computational resource consumption during model inference and training. To further improve model performance, the scale of model parameters is continuously expanding. However, simply increasing the model's depth and width makes training and inference extremely expensive, thus various hardware chips are widely used in model inference and training tasks. Against this backdrop, the Mixture of Experts (MoE) architecture has emerged and become one of the key technologies for building large-scale models. The MoE decomposes the model into multiple smaller expert networks and a gating network. For each input sequence unit (token), the gating network calculates the affinity score between that token and all N expert networks. Then, from these N scores, the top K experts (i.e., Top-K) are selected, and the token's information is routed only to these K selected experts for computation, while the remaining experts remain inactive. This achieves an effective balance between model size and computational efficiency.

[0063] However, in hybrid expert model architectures, the Top-K expert selection problem gradually becomes prominent. This problem occurs frequently in the forward computation of each layer of the model, with an extremely high execution frequency, thus placing high demands on the execution efficiency of the hardware chips used to deploy the model. When the number of expert networks increases significantly, how to quickly and in parallel find the K highest scores from these N expert networks becomes a key bottleneck restricting the overall performance of hybrid expert models.

[0064] Based on this, this disclosure proposes an expert network processing method, apparatus, device, and medium for hybrid expert models, which reduces the number of instructions and register resources required for the Top-K algorithm of hybrid expert models and improves computational efficiency.

[0065] System architecture description applied in the embodiments of this disclosure

[0066] In one embodiment, such as Figure 1 As shown, Figure 1 This is a system architecture diagram of the expert network processing method for executing a hybrid expert model according to an embodiment of the present disclosure. The system architecture mainly consists of a computing core 100, and the hybrid expert model is deployed on the computing core 100. The computing core 100 includes a loading storage unit 110 and multiple execution units 120, which are communicatively connected.

[0067] Specifically, in a parallel computing system, the computing core 100 can invoke one or more related execution units 120 via thread bundles, enabling the execution units 120 to perform operations related to their own functions. The computing core 100 can invoke multiple execution units 120 sequentially or simultaneously via thread bundles; this disclosure does not specifically limit the invocation order or number of execution units 120. Furthermore, a Mixture of Experts (MoE) model is an efficient machine learning model capable of dynamically selecting the most suitable expert network to process input data. To efficiently implement this model on a hardware chip, the computing core 100 in the parallel computing system is responsible for handling the computational tasks of the Mixture of Experts model, including the forward and backward propagation of the expert network.

[0068] In one embodiment, the computing core 100 may be a key component in a graphics processing unit (GPU) or a general-purpose computing on graphics processing unit (GPGPU) responsible for performing parallel computing tasks. It includes multiple execution units 120, loading and storage units 110, register files and other resources, and can process multiple threads simultaneously to achieve efficient parallel computing.

[0069] In one embodiment, the computing core 100 also includes shared memory ( Figure 1 (Not shown), the shared memory can be accessed by different execution units 120 and load storage units 110, and has a large capacity.

[0070] For example, the loading storage unit 110 reads data from an external memory system or I / O device and transfers it to shared memory. The execution unit 120 loads data from shared memory, performs calculations, and then stores the result back to shared memory or external memory via the loading storage unit 110. After receiving instructions, the calculation unit can distribute them to each execution unit 120 via thread bundles. The execution unit 120 performs calculations on the data according to the instructions and stores intermediate or final results in shared memory.

[0071] It is understood that the aforementioned computing core 100 can be a computing unit or a streaming multiprocessor, and this disclosure does not specifically limit it in this regard.

[0072] In one embodiment, the computing core 100 can invoke the loading memory unit 110 via a thread bundle to perform a reduction operation. It is understood that the reduction operation described above can be interpreted as an operation that merges multiple data points into a single result through a specific operation. This specific operation may include, but is not limited to, summation, multiplication, finding the maximum value, and other user-defined functions. This single result can be called the reduction result.

[0073] In one embodiment, the shared memory may be located inside the load storage unit 110. Figure 1 (Not shown), multiple threads within a thread bundle each calculate a value, and the maximum value of the above multiple values ​​is obtained by reduction and stored in the shared memory inside the load storage unit 110.

[0074] In one embodiment, the execution unit 120 is used to execute computational instructions, such as arithmetic operations and logical operations. Specifically, the execution unit 120 can execute various computational tasks in the hybrid expert model, such as matrix multiplication and activation function calculation. Multiple execution units 120 can process different computational tasks simultaneously, realizing parallel computing.

[0075] For example, the computing core 100 can call the execution unit 120 through a thread bundle to perform basic vector arithmetic operations, such as vector addition, vector subtraction, vector multiplication, and vector division; or, the computing core 100 can call the execution unit 120 through a thread bundle to perform vector scalar operations, such as vector addition to a scalar and vector multiplication by a scalar; or, the computing core 100 can call the execution unit 120 through a thread bundle to perform special vector operations, such as vector dot product, extraction of maximum / minimum values ​​of vectors, vector summation / averaging, and vector comparison; or, the computing core 100 can call the execution unit 120 through a thread bundle to perform vector data recombination operations, such as vector shuffling and vector type conversion.

[0076] The expert network processing method for hybrid expert models provided in this disclosure is applied in... Figure 1 The system architecture shown reduces the number of instructions and register resources required for the Top-K algorithm of the hybrid expert model, thereby improving computational efficiency.

[0077] Overall Implementation of the Expert Network Processing Method with Hybrid Expert Model in this Disclosure

[0078] This disclosure proposes an expert network processing method using a hybrid expert model, applied to a computing core. The hybrid expert model is deployed on the computing core, which includes a loading and storage unit and multiple execution units, and the loading and storage unit and the multiple execution units are communicatively connected. (Refer to...) Figure 2 , Figure 2This is a main flowchart of an expert network processing method for a hybrid expert model according to an embodiment of the present disclosure; the expert network processing method for a hybrid expert model includes, but is not limited to, steps S100 to S300.

[0079] Step S100: Configure multiple thread bundles corresponding to each execution unit according to the multiple expert networks of the hybrid expert model, so as to allocate the expert score of each expert network to each thread of the thread bundle;

[0080] Step S200: In response to the reduction operation instruction, the expert scores held by each thread in each thread bundle are cyclically reduced by loading the storage unit until multiple target expert scores are obtained.

[0081] Step S300: Determine multiple target expert networks corresponding to multiple target expert scores from multiple expert networks.

[0082] Specifically, in this disclosure, by rationally configuring the thread bundles of execution units in multiple expert networks in the hybrid expert model, the expert scores are allocated to each thread of the thread bundle. Furthermore, the reduction operation instructions can be used to perform cyclic reduction operations by loading storage units, thereby reducing the number of instructions and register resources required by the Top-K algorithm of the hybrid expert model and improving computational efficiency.

[0083] In one embodiment, step S100 specifically involves the computing core configuring multiple thread bundles corresponding to each execution unit based on the multiple expert networks of the hybrid expert model, and then allocating the expert score of each expert network to each thread of the thread bundle. In one embodiment, the computing core can configure the thread bundles according to the number of expert networks in the hybrid expert model. Taking DeepSeek's 256 expert model as an example, these 256 expert networks can be assigned to a workgroup. A workgroup is typically organized into multiple warps, and a warp typically includes 32 consecutive threads. Each warp within the workgroup processes 32 expert networks, meaning a workgroup contains 8 warps, each warp contains 32 threads, and each thread is assigned the expert score of one expert network. This configuration depends on the number of expert networks in the hybrid expert model. The more expert networks that need to be processed, the larger the number of thread bundles can be, thus ensuring that the expert score of each expert network is allocated to enough threads, maximizing computational efficiency and resource utilization. Within a thread bundle, each thread can be responsible for a specific computational task of an expert network, such as calculating expert scores or forward propagation. It's important to note that the workgroup runs on the computation core, and each thread bundle within the workgroup is assigned to run on each execution unit within the computation core. This allows for flexible configuration of thread bundles and allocation of expert scores based on the number of expert networks and the configuration of the computation cores, making it suitable for hybrid expert models of different scales and avoiding waste of computational resources.

[0084] In one embodiment, step S200 specifically involves the computational core responding to a reduction operation instruction by performing a cyclic reduction operation on the expert scores held by each thread in each thread bundle through a loading storage unit, until multiple target expert scores are obtained. This disclosure enables the expert score reduction operation to be completed quickly through efficient data transfer in the loading storage unit and fast access to shared memory. Furthermore, by utilizing the parallel processing capabilities of thread bundles, multiple expert scores can participate in the reduction operation simultaneously, improving computational efficiency.

[0085] In one embodiment, step S300 specifically involves: the computational core determining multiple target expert networks corresponding to multiple target expert scores from multiple expert networks, and integrating them to obtain the target expert scores of the hybrid expert model and their corresponding target expert networks.

[0086] In one embodiment, such as Figure 3 and Figure 4 As shown, Figure 3 yes Figure 2 A flowchart of a sub-step of one embodiment of step S200, Figure 4 This is a schematic diagram illustrating the reduction operation performed using a loaded memory unit; in Figure 4In the example above, the loading storage unit 110 of the above step embodiment includes a reduction unit 111 and an atomic operation unit 112. The cyclic reduction operation of the expert score held by each thread in each thread bundle through the loading storage unit in step S200 may include, but is not limited to, steps S210 to S230.

[0087] Step S210: Through the reduction unit, the expert score held by each thread in each thread bundle is subjected to the first reduction process to obtain the first reduction result of each thread bundle;

[0088] Step S220: Using atomic operation units, perform a second reduction process on the first reduction result of each thread bundle to obtain and store the target expert score;

[0089] Step S230: Data recording and data removal are performed on the target threads that hold the target expert score.

[0090] In the embodiments of steps S210 to S230, the computing core 100 can perform a cyclic reduction operation on the expert scores held by each thread in each thread bundle through the loading storage unit 110. Further, the internal hardware structure of the loading storage unit 110 may include a reduction unit 111 and an atomic operation unit 112. Specifically, the computing core 100 first performs a first reduction process on the expert scores held by each thread in each thread bundle through the reduction unit 111, obtaining a first reduction result for each thread bundle, ensuring that the initial reduction operation is executed efficiently within each thread bundle. Next, the computing core 100 performs a second reduction process on the first reduction result of each thread bundle through the atomic operation unit 112, obtaining and storing the target expert score. This target expert score is the final score obtained from multiple expert scores through the reduction operation. Finally, for the target thread holding the target expert score, the computing core 100 performs data recording and data removal processing, thereby ensuring the accuracy and efficiency of the results of each cyclic reduction operation.

[0091] In one embodiment, such as Figure 5 As shown, Figure 5 yes Figure 4 A schematic diagram of the structure of the reduction unit 111; the reduction unit 111 in the above-described embodiment includes a maximum value reduction tree, in Figure 5 In the example, the hardware structure of the reduction unit 111 can be a maximum reduction tree; in Figure 4In the example, the computing core also includes a register area 130 corresponding to each thread bundle on each execution unit. The register area 130 includes registers corresponding to each thread in the thread bundle. For example, the computing core includes M register areas, and the M register areas correspond to thread bundle 1, thread bundle 2, thread bundle 3, thread bundle 4, ..., thread bundle M on the execution unit, respectively. In any register area, multiple registers corresponding to each thread in that thread bundle are set up. Each thread is allocated 32 registers. The register area of ​​thread bundle 0 contains registers corresponding to threads 0 to 31, with register numbers from R0 to R31. It should be noted that when a thread bundle performs a computing task, each thread uses its own registers for data storage and computation. For example, the register area of ​​thread 0 contains R0 to R31, used to store the private data of thread 0; the register area of ​​thread 1 contains R0 to R31, used to store the private data of thread 1; in this embodiment of the disclosure, the registers corresponding to each thread can be used to store the expert scores of the expert network allocated to the thread, or the registers can also be used to store the expert index of the expert network or the thread index of the corresponding thread. Further, as Figure 6 As shown, Figure 6 yes Figure 3 The flowchart of a sub-step of step S210 in one embodiment, regarding the reduction unit in step S210, performs a first reduction process on the expert score held by each thread in each thread bundle to obtain the first reduction result of each thread bundle, may include, but is not limited to, steps S211 to S212.

[0092] Step S211: Obtain the expert score stored in the register corresponding to each thread in each thread bundle through the maximum value reduction tree;

[0093] Step S212: Perform maximum reduction on all expert scores of the thread bundle to obtain the maximum local expert score, and use the maximum local expert score as the first reduction result of the thread bundle.

[0094] In the embodiments of steps S211 to S212, for each thread bundle, the expert score held by the thread can be obtained from the register corresponding to each thread in each thread bundle through a maximum value reduction tree. The maximum value reduction tree can quickly and efficiently find the maximum value from multiple expert scores. Then, maximum value reduction is performed on all expert scores of the thread bundle to obtain the local maximum value of the expert score, that is, the first reduction result of the thread bundle. Taking a model with 32 threads in each thread bundle as an example, the maximum value of the 32 expert scores can be selected from the initial input 32 expert scores through multiple pairwise comparisons and layer by layer, so that each thread bundle can instantly calculate the local maximum score value within its own range within one clock cycle.

[0095] In one embodiment, the maximum value reduction tree of the above-described steps includes multiple maximum value comparators, the number of which is determined based on the total number of threads in each thread bundle. It should be noted that, in Figure 5 In the example, the maximum reduction tree of this embodiment consists of multiple maximum value comparators. Specifically, in this embodiment, each thread bundle contains 32 threads, so a five-layer reduction tree can be composed of 31 maximum value comparators. The 16 maximum value comparators of the first layer reduction tree work in parallel, receiving 32 expert scores from the 32 threads in each thread bundle and outputting 16 first expert score maximum values. The 8 maximum value comparators of the second layer reduction tree receive the 16 first expert score maximum values ​​from the first layer reduction tree and output 8 second expert score maximum values. The 4 maximum value comparators of the third layer reduction tree receive the 8 second expert score maximum values ​​from the second layer reduction tree and output 4 third expert score maximum values. The 2 maximum value comparators of the fourth layer reduction tree receive the 4 third expert score maximum values ​​from the third layer reduction tree and output 2 fourth expert score maximum values. The 1 maximum value comparator of the fifth layer reduction tree receives the 2 fourth expert score maximum values ​​from the fourth layer reduction tree and outputs the final 1 local expert score maximum value, thereby realizing parallel reduction of expert scores within a single thread bundle in a single cycle.

[0096] In one embodiment, such as Figure 7 As shown, Figure 7 yes Figure 4 A schematic diagram of the structure of the middle atomic operation unit 112; as shown. Figure 4 and Figure 7 As shown, the loading storage unit 110 in the above-described embodiment further includes shared memory 113. Shared memory 113 can be divided into multiple independent storage units (SRAM Banks), and the number of storage units typically corresponds to the total number of threads within a thread bundle, thereby facilitating the processing of data from one thread bundle within a single cycle. Figure 7 In the example, the atomic operation unit 112 includes an arithmetic logic unit. The hardware structure of the atomic operation unit 112 can be composed of arithmetic logic units, which can then implement atomic operations on the shared memory 113, performing read-modify-write operations in an indivisible manner. It should be noted that atomic operations can include atomic addition, subtraction, maximum value, minimum value, swap, etc., and support data types such as integers and floating-point numbers. Specifically, in this embodiment, the arithmetic logic unit can perform atomic maximum value operations. Further, as... Figure 8 As shown, Figure 8 yes Figure 3The flowchart of a sub-step of one embodiment of step S220, which involves performing a second reduction process on the first reduction result of each thread bundle through atomic operation units to obtain and store the target expert score, may include, but is not limited to, steps S221 to S222.

[0097] Step S221: Obtain the first reduction result of each thread bundle through the arithmetic logic unit to obtain the maximum value of multiple local expert scores;

[0098] Step S222: Perform atomic maximum value operation on all local expert score maximum values ​​through the arithmetic logic unit to calculate the global expert score maximum value, use the global expert score maximum value as the target expert score, and store the target expert score in shared memory.

[0099] In the embodiments of steps S221 to S222, the atomic operation unit and the reduction unit in the loading storage unit are tightly coupled, so the computation core can aggregate and further reduce the maximum local expert scores of all thread bundles through the atomic operation unit. Specifically, the computation core obtains the first reduction result of all thread bundles through the arithmetic logic unit, that is, obtains the maximum local expert score from the maximum value reduction tree of the above embodiment. Further, the arithmetic logic unit performs atomic maximum value operation on all local expert score maximum values ​​to quickly filter out the maximum global expert score, and the final maximum global expert score is used as the target expert score. It can be understood that the atomic maximum value operation ensures the accuracy of the maximum global expert score obtained in a multi-threaded environment, so the maximum global expert score is stored in shared memory as the target expert score.

[0100] It should be noted that, in Figure 7 In the example, atomic operation unit 112 receives an atomic maximum value operation request to be executed. This request includes atomic request data (the maximum local expert score) and an atomic request address (the location in shared memory 113 used to store the maximum local expert score). Further, atomic operation unit 112 issues a read request to the specified address (atomic request address) in shared memory 113, thereby reading the data from shared memory 113 into atomic operation unit 112. Then, the arithmetic logic unit compares the current value read from shared memory 113 with the atomic request data (the maximum local expert score). If the atomic request data is greater than the current value in shared memory 113, the arithmetic logic unit updates the value in shared memory 113 using the atomic request data; otherwise, the value in shared memory 113 remains unchanged. After the arithmetic logic unit updates the value in shared memory 113 using the atomic request data, the maximum global expert score stored in shared memory 113 becomes the target expert score.

[0101] In another embodiment, the shared memory in the above-described step embodiment includes multiple storage addresses, for example, the multiple storage addresses can be... Figure 7 The example uses shared memory partitioning to obtain multiple SRAM Banks, and the number of storage addresses is determined based on the total number of threads in each thread bundle. For example, if a thread bundle contains 32 threads, then the shared memory can be partitioned into 32 SRAM Banks, with each thread corresponding to one SRAM Bank. Furthermore, the step S222 above, which involves storing the target expert score into shared memory, may include, but is not limited to, step S2220.

[0102] Step S2220: The target expert score is stored to the target storage address through the arithmetic logic unit. The target storage address is the storage address corresponding to the target thread that holds the target expert score or a storage address in shared memory dedicated to storing expert scores.

[0103] In the embodiment of step S2220, when storing the target expert score into shared memory, the score can be stored in the target storage address using an arithmetic logic unit. This target storage address can be the storage address corresponding to the target thread, or it can be a dedicated address in shared memory used to store expert scores. By using atomic operation units, the atomicity of the storage operation can be ensured, data conflicts during multi-threaded writes can be avoided, and the accurate expert score reduction result can be obtained.

[0104] In one embodiment, the data recording and data removal process for the target thread holding the target expert score in step S230 may include, but is not limited to, step S231.

[0105] Step S231: For the target thread holding the target expert score, store the target thread index and target expert index stored in the target register corresponding to the target thread in the shared memory, and set the score value of the target expert score stored in the target register to a preset minimum value.

[0106] In the embodiment of step S231, for the target thread holding the target expert score, data recording and data removal processing are required. Specifically, the computational core stores the target thread index and the target expert index in the target register of this target thread into shared memory. The target register is the register in the target thread used to store the target expert score and its related index. The target thread index is used to identify the target thread. The target expert index is the index of the expert network corresponding to the target expert score. The specific storage process can be referred to the above-described embodiments of the relevant steps for storing the target expert score, and will not be repeated here. Further, after the storage of the thread and expert index information is completed, the computational core sets the score value of the target expert score in the target register to a preset minimum value. In this embodiment, the preset minimum value is set to, or can be set to other suitable values ​​according to actual needs. The above steps are equivalent to marking the target expert network corresponding to the target expert score as "selected", making it invalid in subsequent expert score reduction comparisons, thereby ensuring that the target thread will not affect the new cyclic reduction operation in subsequent calculations, thus ensuring the accuracy and efficiency of the next round of reduction operation.

[0107] In one embodiment, the determination of multiple target expert networks corresponding to multiple target expert scores from multiple expert networks in step S300 may include, but is not limited to, steps S310 to S320.

[0108] Step S310: Obtain multiple target expert indices corresponding one-to-one with the multiple target expert scores from shared memory;

[0109] Step S320: Determine multiple target expert networks corresponding to multiple target expert scores from multiple expert networks based on multiple target expert indices.

[0110] In the embodiments of steps S310 to S320, the computing core reads target expert indices corresponding to multiple target expert scores from shared memory. These target expert indices stored in shared memory are determined during the reduction operation in the above-described embodiments. Further, the computing core finds the corresponding target expert network from multiple expert networks based on the read target expert indices. It is understood that each expert network has a unique expert network index, and the corresponding expert network can be directly located through the expert network index.

[0111] Reference Figure 9 , Figure 9 This is a schematic diagram of the structure of an expert network processing device 900 based on a hybrid expert model, as proposed in this disclosure. The device includes:

[0112] Configuration unit 910 is used to configure multiple thread bundles corresponding to each execution unit according to multiple expert networks of the hybrid expert model, so as to allocate the expert score of each expert network to each thread of the thread bundle;

[0113] The reduction operation unit 920 is used to respond to the reduction operation instruction by performing a cyclic reduction operation on the expert scores held by each thread in each thread bundle through the loading storage unit until multiple target expert scores are obtained.

[0114] The expert network determination unit 930 is used to determine multiple target expert networks corresponding to multiple target expert scores from multiple expert networks.

[0115] The expert network processing apparatus 900 for the hybrid expert model disclosed herein is used to execute the expert network processing method for the hybrid expert model as described in the above embodiments. Its specific processing procedure is the same as that of the expert network processing method for the hybrid expert model in the above embodiments, and will not be repeated here.

[0116] Based on the expert network processing method of the hybrid expert model in the above embodiments, various embodiments of the electronic device, computer-readable storage medium and computer program product of this disclosure are proposed below.

[0117] This disclosure also provides an electronic device 1000, comprising:

[0118] At least one processor, and,

[0119] A memory that is communicatively connected to at least one processor; wherein,

[0120] The memory stores instructions that are executed by at least one processor to cause the at least one processor to perform the method as described in any of the above embodiments of the present disclosure when executing the instructions.

[0121] The following is combined with Figure 10 The hardware structure of the electronic device is described in detail. The electronic device 1000 includes: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050.

[0122] The processor 1010 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure.

[0123] The memory 1020 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called by the processor 1010 to execute the expert network processing method of the hybrid expert model of the embodiments of this disclosure.

[0124] The input / output interface 1030 is used to implement information input and output;

[0125] The communication interface 1040 is used to enable communication and interaction between this device and other devices. Communication can be achieved via wired means (e.g., USB, Ethernet cable) or wireless means (e.g., mobile network, Wi-Fi, Bluetooth).

[0126] Bus 1050 transmits information between various components of the device (e.g., processor 1010, memory 1020, input / output interface 1030, and communication interface 1040);

[0127] The processor 1010, memory 1020, input / output interface 1030 and communication interface 1040 are connected to each other within the device via bus 1050.

[0128] It is worth noting that since the electronic device 1000 of this disclosure is capable of executing the expert network processing method of the hybrid expert model of any of the above embodiments, the specific implementation and technical effects of the electronic device 1000 of this disclosure can be referred to the specific implementation and technical effects of the expert network processing method of the hybrid expert model of any of the above embodiments.

[0129] Furthermore, one embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions for executing the aforementioned expert network processing method for hybrid expert models. It is worth noting that since the computer-readable storage medium of this disclosure can execute the expert network processing method for hybrid expert models of any of the above embodiments, the specific implementation methods and technical effects of the computer-readable storage medium of this disclosure can be referenced to the specific implementation methods and technical effects of the expert network processing method for hybrid expert models of any of the above embodiments.

[0130] Furthermore, one embodiment of this application also provides a computer program product, including a computer program or computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium and executes the computer program or computer instructions, causing the computer device to perform the expert network processing method of the hybrid expert model described above.

[0131] It is worth noting that, since the computer program product of this disclosure is capable of executing the expert network processing method of the hybrid expert model in any of the above embodiments, the specific implementation methods and technical effects of the computer program product of this disclosure can be referred to the specific implementation methods and technical effects of the expert network processing method of the hybrid expert model in any of the above embodiments.

[0132] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0133] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0134] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0135] It should be understood that in the description of the embodiments of this disclosure, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0136] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0139] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0140] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for processing expert networks using a hybrid expert model, characterized in that, The method is applied to a computing core, wherein the hybrid expert model is deployed on the computing core, the computing core includes a load storage unit and multiple execution units, the load storage unit and the multiple execution units are communicatively connected, the load storage unit includes a reduction unit and an atomic operation unit, and the method includes: According to the multiple expert networks of the hybrid expert model, configure multiple thread bundles corresponding to each execution unit to allocate the expert score of each expert network to each thread of the thread bundle; In response to the reduction operation instruction, the expert score held by each thread in each thread bundle is cyclically reduced through the loading storage unit until multiple target expert scores are obtained; From the plurality of expert networks, determine the plurality of target expert networks corresponding to the plurality of target expert scores; The cyclic reduction operation includes: The expert score held by each thread in each thread bundle is obtained through the reduction unit, and the maximum value reduction is performed on all the expert scores of the thread bundle to obtain the local expert score maximum value. The local expert score maximum value is used as the first reduction result of the thread bundle. The first reduction result of each thread bundle is obtained through the atomic operation unit to obtain multiple local expert score maximum values. An atomic maximum value operation is performed on all local expert score maximum values ​​to calculate the global expert score maximum value. The global expert score maximum value is used as the target expert score and stored.

2. The method according to claim 1, characterized in that, The cyclic reduction operation also includes: For target threads holding the aforementioned target expert score, data recording and data removal processes are performed.

3. The method according to claim 1, characterized in that, The reduction unit includes a maximum value reduction tree, and the calculation core also includes a register area corresponding to each thread bundle. The register area includes a register corresponding to each thread in the thread bundle. The register is used to store the expert score of the corresponding thread. The maximum value reduction tree is used to obtain the expert score stored in the register corresponding to each thread in each thread bundle, perform maximum value reduction on all the expert scores of the thread bundle to obtain the local expert score maximum value, and use the local expert score maximum value as the first reduction result of the thread bundle.

4. The method according to claim 3, characterized in that, The maximum value reduction tree includes multiple maximum value comparators, the number of which is determined based on the total number of threads in each thread bundle.

5. The method according to claim 2, characterized in that, The loading storage unit further includes shared memory, and the atomic operation unit includes an arithmetic logic unit. The arithmetic logic unit is used to obtain the first reduction result of each thread bundle to obtain multiple local expert score maximum values, perform atomic maximum value operation on all local expert score maximum values ​​to calculate the global expert score maximum value, and use the global expert score maximum value as the target expert score. The cyclic reduction operation further includes storing the target expert score into the shared memory.

6. The method according to claim 5, characterized in that, The shared memory includes multiple storage addresses, the number of which is determined based on the total number of threads in each thread bundle. Storing the target expert score in the shared memory includes: The target expert score is stored in the target storage address by the arithmetic logic unit. The target storage address is either the storage address corresponding to the target thread that holds the target expert score or a storage address in the shared memory specifically used for storing expert scores.

7. The method according to claim 5, characterized in that, The process of recording and removing data for the target thread holding the target expert score includes: For the target thread holding the target expert score, the target thread index and target expert index stored in the target register corresponding to the target thread are stored in the shared memory, and the score value of the target expert score stored in the target register is set to a preset minimum value.

8. The method according to claim 7, characterized in that, The step of determining the multiple target expert networks corresponding to the multiple target expert scores from the multiple expert networks includes: Obtain the target expert indexes corresponding one-to-one with the target expert scores from the shared memory; Based on the multiple target expert indices, determine the multiple target expert networks corresponding to the multiple target expert scores from the multiple expert networks.

9. An expert network processing device with a hybrid expert model, characterized in that, The expert network processing device for the hybrid expert model includes a loading and storage unit and multiple execution units, which are communicatively connected. The loading and storage unit includes a reduction unit and an atomic operation unit. The expert network processing device for the hybrid expert model further includes: A configuration unit is configured to configure multiple thread bundles corresponding to each execution unit according to multiple expert networks of the hybrid expert model, so as to allocate the expert score of each expert network to each thread of the thread bundle; A reduction operation unit is configured to, in response to a reduction operation instruction, perform a cyclic reduction operation on the expert scores held by each thread in each thread bundle through a loading storage unit until multiple target expert scores are obtained. The cyclic reduction operation includes: obtaining the expert scores held by each thread in each thread bundle through the reduction unit; performing maximum value reduction on all expert scores in the thread bundle to obtain local expert score maximum values; using the local expert score maximum values ​​as the first reduction result of the thread bundle; obtaining the first reduction result of each thread bundle through the atomic operation unit to obtain multiple local expert score maximum values; performing an atomic maximum value operation on all local expert score maximum values ​​to calculate a global expert score maximum value; and using the global expert score maximum value as the target expert score. An expert network determination unit is used to determine, from the plurality of expert networks, a plurality of target expert networks corresponding to the plurality of target expert scores.

10. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. The program is executed by the processor to implement the expert network processing method of the hybrid expert model as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the expert network processing method of the hybrid expert model as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program or computer instructions, characterized in that, The computer program or the computer instructions are stored in a computer-readable storage medium, and the processor of the computer device reads the computer program or the computer instructions from the computer-readable storage medium. The processor executes the computer program or the computer instructions, causing the computer device to perform the expert network processing method of the hybrid expert model as described in any one of claims 1 to 8.

Images