A text data reasoning method and device based on hybrid expert model

Abstract

The present invention discloses a method and device for text data reasoning based on a hybrid expert model, which is used to solve the technical problem of low reasoning efficiency caused by the existing reasoning process based on the hybrid expert model. The method includes obtaining a pre-run text dataset; inputting the pre-run text dataset into a preset hybrid expert model in a multi-level heterogeneous memory space for pre-run reasoning, determining the hybrid expert module calculation transmission time and expert correlation table; based on the expert correlation table and the preset number of pre-fetched popular experts, identifying popular experts in the preset hybrid expert model and determining multiple target hybrid expert modules; using a preset constraint-aware planning inequality to calculate the transmission time based on the preset number of pre-fetched popular experts and the hybrid expert module, and determining the number of input text batches; and using each target hybrid expert module in the multi-level heterogeneous memory space to output the model reasoning result based on the subset of text data to be reasoned corresponding to the number of input text batches.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a text data reasoning method and device based on a hybrid expert model. Background Art

[0002] With the rapid development of deep learning in recent years, large models have achieved remarkable results in various fields. To scale models without increasing training and inference costs, recent work has introduced sparsely activated mixture-of-experts (MoE) models. MoE models typically use MoE layers instead of feed-forward neural network (FFN) layers. For each input, only a subset of sparsely activated parameters (i.e., experts) are required for computation, significantly reducing computational cost.

[0003] Extensive experiments have demonstrated the superiority of the MoE architecture. However, the MoE model faces a more severe graphics memory bottleneck during inference than other large models. This is primarily due to the large number of parameters. For example, DeepSeek-V2 has 236B parameters, requiring at least eight state-of-the-art GPUs (such as the A100) for inference. This challenge is particularly severe in resource-constrained environments.

[0004] Most existing reasoning processes based on hybrid expert models directly apply traditional dense models to MoE models, prefetching the next layer while performing calculations on the current layer. This does not consider the differences in I / O resource requirements between different layers and directly prefetches the entire MoE layer, resulting in a large number of "bubbles" in the pipeline and low reasoning efficiency. Summary of the Invention

[0005] The present invention provides a text data reasoning method and device based on a hybrid expert model, which are used to solve the technical problem of low reasoning efficiency caused by the existing reasoning process based on the hybrid expert model.

[0006] The first aspect of the present invention provides a text data reasoning method based on a hybrid expert model, comprising:

[0007] Respond to inference requests, build multi-level heterogeneous memory spaces, and obtain pre-run text datasets;

[0008] Inputting the pre-run text data set into a preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and determining a hybrid expert module calculation transmission time and an expert correlation table;

[0009] Based on the expert correlation table and the preset number of pre-fetched hot experts, performing hot expert identification on multiple hybrid expert modules in the preset hybrid expert model to determine multiple target hybrid expert modules;

[0010] The preset constraint-aware planning inequality is used to calculate the transmission time based on the preset number of pre-fetched popular experts and the hybrid expert module to determine the number of input text batches;

[0011] In the multi-level heterogeneous memory space, each target hybrid expert module is used to output a model reasoning result corresponding to the subset of text data to be inferred according to the number of batches of input texts.

[0012] Optionally, the step of performing hot expert identification on multiple hybrid expert modules in the preset hybrid expert model based on the expert correlation table and the preset number of pre-fetched hot experts to determine multiple target hybrid expert modules includes:

[0013] Determining, based on the expert relevance table, expert selection frequencies of a plurality of pre-operation text data in the pre-operation text dataset for a plurality of experts in each of the hybrid expert modules;

[0014] sorting the expert selection frequencies of the pre-run text data for the experts in the hybrid expert modules in descending order;

[0015] Among the experts in each of the mixed expert modules, experts corresponding to the expert selection frequencies of the pre-set number of popular experts are selected as popular experts, and the remaining experts are selected as non-popular experts;

[0016] The hybrid expert module corresponding to each of the popular experts is used as the target hybrid expert module.

[0017] Optionally, the target hybrid expert module includes a gating layer, an expert layer, and an attention layer; the expert layer includes a preset number of popular experts and a plurality of non-popular experts; the data processing process of the target hybrid expert module is specifically as follows:

[0018] Performing weight calculation on multiple input text unit data in the input text data subset input to the target hybrid expert module through the gating layer, and determining the weight value corresponding to each non-popular expert for each input text unit data;

[0019] Sorting the weight values ​​corresponding to any input text unit data of each of the non-hot experts in descending order, and selecting the non-hot experts corresponding to the preset number of weight values ​​as the non-hot activated experts;

[0020] Collecting statistics on the non-popular activated experts corresponding to each of the input text unit data to determine the activation frequency corresponding to each of the non-popular activated experts;

[0021] Sorting the activation frequencies corresponding to the non-hot activation experts in descending order to generate a non-hot activation expert data processing and sorting result;

[0022] Using a preset number of hot experts to perform calculations on the input text data subset respectively, and outputting the hot expert calculation results corresponding to the preset number of hot experts;

[0023] Using the input text data subsets in sequence as inputs of non-hot activation experts corresponding to the non-hot activation expert data processing and sorting results, and outputting a plurality of non-hot activation expert calculation results;

[0024] Merging the popular expert calculation results and the non-popular activated expert calculation results to generate a merged expert calculation result;

[0025] The combined expert calculation results are calculated by the attention layer using a preset attention mechanism to generate an output model result.

[0026] Optionally, the multi-level heterogeneous memory space includes GPU memory, CPU memory and hard disk; the GPU memory is used to store tensors required for current calculation and pre-fetched tensors; the CPU memory and the hard disk are used to store inactivated tensors.

[0027] Optionally, different types of tensors are placed in the GPU memory, the CPU memory and the hard disk respectively according to preset layer-by-layer allocation conditions.

[0028] Optionally, the hybrid expert module calculates the transmission time including the attention layer calculation time, the gating layer calculation time, the popular expert calculation time, the average calculation time of a single expert, the attention layer transmission time, the gating layer transmission time, and the single expert transmission time; the preset constraint-aware planning inequality is specifically:

[0029] ;

[0030] Where n is the number of input text batches, and n is the smallest integer that satisfies the preset constraint-aware planning inequality; Compute time for the attention layer; Transmission time for the gating layer; Calculate time for the gating layer; Transmit time for a single expert; is the calculation time of popular experts; K is the number of pre-set and pre-fetched popular experts; is the average computation time of a single expert for the i-th expert; is a built-in function; Q is the number of experts; Transfer time for the attention layer.

[0031] A second aspect of the present invention provides a text data reasoning device based on a hybrid expert model, comprising:

[0032] The response module is used to respond to inference requests, build multi-level heterogeneous memory spaces, and obtain pre-run text datasets;

[0033] a pre-run reasoning module, configured to input the pre-run text data set into a preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and determine a hybrid expert module calculation transmission time and an expert correlation table;

[0034] an identification module, configured to perform hot expert identification on a plurality of hybrid expert modules in the preset hybrid expert model based on the expert correlation table and the preset number of hot experts, and determine a plurality of target hybrid expert modules;

[0035] A batch quantity determination module is used to determine the number of input text batches based on the preset number of pre-fetched popular experts and the transmission time calculated by the hybrid expert module using a preset constraint-aware planning inequality;

[0036] The output result module is used to output the model reasoning result corresponding to the text data subset to be inferred according to the number of input text batches using each target hybrid expert module in the multi-level heterogeneous memory space.

[0037] A third aspect of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the text data reasoning method based on the hybrid expert model as described in any one of the above items.

[0038] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of the text data reasoning method based on the hybrid expert model as described in any one of the above.

[0039] A fifth aspect of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer is caused to perform the steps of the text data reasoning method based on the hybrid expert model as described in any one of the above items.

[0040] It can be seen from the above technical solutions that the present invention has the following advantages:

[0041] The above technical solution of the present invention provides a text data reasoning method based on a hybrid expert model. When it is necessary to reason about text data, a multi-level heterogeneous memory space is first constructed, and a pre-run text data set is obtained; then, the pre-run text data set is input into a preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and the hybrid expert module calculation transmission time and the expert correlation table are determined; based on the expert correlation table and the preset pre-fetched hot expert number, hot expert identification is performed on multiple hybrid expert modules in the preset hybrid expert model, and multiple target hybrid expert modules are determined; a preset constraint-aware planning inequality is used to calculate the transmission time according to the preset pre-fetched hot expert number and the hybrid expert module, and the number of input text batches is determined; finally, in the multi-level heterogeneous memory In the space, each target hybrid expert module is used to output the model reasoning result corresponding to the subset of text data to be inferred according to the subset of text data to be inferred corresponding to the number of input text batches; based on the above scheme, according to the pre-run reasoning process, the hardware capabilities in the current environment are adaptively measured, the hybrid expert module calculation transmission time and the expert correlation table are determined, and the preset constraint perception planning inequality is combined to output the number of input text batches, and the subset of text data to be inferred corresponding to the number of input text batches is used as the input of popular experts based on the expert correlation table to identify multiple target hybrid expert modules, and output the model reasoning result. The present invention focuses on the currently popular hybrid expert model, and greatly compresses the "bubbles" existing in the reasoning pipeline, thereby improving the reasoning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A flowchart of a method for text data reasoning based on a hybrid expert model provided in Example 1 of the present invention;

[0044] Figure 2 A schematic diagram of an IO-computing pipeline provided in Example 1 of the present invention;

[0045] Figure 3 A schematic diagram of the computational graph arrangement of popular expert perception provided in the first embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a computational graph traversal provided in the first embodiment of the present invention;

[0047] Figure 5 A flowchart of a text data reasoning method based on a hybrid expert model provided in the second embodiment of the present invention;

[0048] Figure 6 A schematic diagram of the structure of an existing large model provided in the second embodiment of the present invention;

[0049] Figure 7 This is a structural block diagram of a text data reasoning device based on a hybrid expert model provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0050] The embodiments of the present invention provide a text data reasoning method and device based on a hybrid expert model, which are used to solve the technical problem of low reasoning efficiency caused by the existing reasoning process based on the hybrid expert model.

[0051] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0052] Explanation of terms:

[0053] Mixture-of-Experts (MoE): The Mixture-of-Experts model replaces the feedforward neural network layer in the Transformer with an MoE layer. The MoE layer consists of a gating network and multiple experts. The gating function is the key feature of the MoE, which discretely determines the expert selection for each token. The most common practice is to select the top k experts for each token. In this case, the gating function is usually a softmax function, which activates the top k experts according to the calculated probability distribution, with k typically being 1 or 2. The experts are essentially feedforward neural networks (FFNs).

[0054] Hot experts: In each MoE layer, for each input token, the gating network selects the top-k experts from n experts to process the token. When the number of input tokens is large, some experts are selected and used more frequently by the gating mechanism than others. These experts are called hot experts.

[0055] Token: In natural language processing, a token typically refers to a meaningful fragment of text. When processing text, large models break down the input sentence into tokens.

[0056] Throughput: number of tokens generated / time consumed

[0057] Pipeline: A pipeline in a computer breaks down a task into multiple independent stages, each of which can handle different tasks in parallel. The IO-computing pipeline described below primarily describes the order and relationship between computation and data transfer in parallel.

[0058] See also Figure 1 , Figure 1 This is a flowchart of the steps of a text data reasoning method based on a hybrid expert model provided in Example 1 of the present invention.

[0059] The present invention provides a text data reasoning method based on a hybrid expert model, comprising:

[0060] Step 101: In response to an inference request, a multi-level heterogeneous memory space is constructed, and a pre-run text dataset is obtained.

[0061] The multi-level heterogeneous memory space includes GPU memory, CPU memory, and hard disk. GPU memory is used to store tensors required for current computations and pre-fetched tensors, while CPU memory and hard disk are used to store inactive tensors. Tensors of different types are placed in GPU memory, CPU memory, and hard disk according to pre-set layer-by-layer allocation conditions.

[0062] It should be noted that the multi-level heterogeneous memory space constructed by the present invention includes GPU memory, CPU memory and hard disk to meet the storage needs of large models in resource-constrained environments. Through adaptive tensor deployment, various tensors are reasonably placed according to the storage resources in the current environment to improve the utilization rate of existing resources. Specifically, the GPU memory is mainly used to store necessary parts, that is, the tensors required for the current calculation and the pre-fetched tensors. When there are more idle memory resources in the current environment, they can also be further utilized to reduce some IO. Specifically, the present invention can select placement locations for different types of tensors such as experts, gates, attention layers, KV caches, activation values, etc. according to preset layer-by-layer allocation conditions, and supports layer-by-layer placement. For example, the preset layer-by-layer allocation conditions can be set to place the first 3 layers of experts in GPU memory, the next 20 layers of experts in CPU memory, and the rest on disk.

[0063] Furthermore, for the inactivated part, you can choose to offload it to the CPU memory or hard disk. The present invention gives experts a higher priority to occupy the CPU memory. Because the challenge faced by the MoE layer is that the experts requested each time are determined by the gating function and cannot be accurately predicted in advance. Therefore, when faced with large-scale tasks, there is a high probability that experts will need to be transmitted immediately, so it is urgent to quickly transfer the required expert tensors to the video memory. Considering that CPU memory with a faster transmission rate can provide a faster response speed, the expert part is given priority to occupy the CPU memory to deal with unexpected situations.

[0064] It is worth mentioning that when the CPU memory is sufficient, the present invention uses pin_memory to achieve faster CPU-GPU transmission. When the CPU memory is not sufficient and the hard disk needs to be used, in order to reduce the GPU's access to the hard disk resources, the tensors in the CPU are dynamically maintained. Specifically, the relevant tensors of a fixed number of layers L are dynamically maintained in the limited CPU memory. When the calculation reaches the i-th layer, the GPU pre-fetches the relevant tensors of the i+1-th layer, and the CPU also pre-fetches the relevant tensors of the i+L-th layer from the hard disk and moves out the relevant tensors of the i-th layer. This can effectively utilize the idle CPU-hard disk transmission bandwidth to reduce the GPU's access to the hard disk.

[0065] Step 102: Input the pre-run text data set into the preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and determine the hybrid expert module calculation transmission time and expert correlation table.

[0066] The pre-run text dataset includes a plurality of pre-run text data.

[0067] It should be noted that the present invention pre-fetches experts based on the relevance of expert selection. During the reasoning process of the pre-set hybrid expert model, there are popular experts, meaning that a small number of experts can cover most of the calculations. Therefore, the pre-fetch targets of the MoE layer are gating functions and popular experts. Since MoE is data-sensitive, popular experts may change with changes in input. Therefore, in order to identify the popular experts that tokens in the current batch tend to select, the present invention establishes a data-aware expert relevance table.

[0068] Specifically, the pre-run process records the correlation (i.e., frequency relationship) between experts activated at different layers in tokens (pre-run text data), thereby generating an expert correlation table. Furthermore, to perceive the hardware capabilities of the current environment, before using the pre-set hybrid expert model for inference during the pre-run process, the present invention measures the calculation time for batches of different sizes in the current environment, as well as their transmission time, based on information such as the shape and dtype of the model's different layers. This results in the hybrid expert module's calculation and transmission time, and the result (hybrid expert module calculation and transmission time) is recorded locally as a cache.

[0069] Step 103 : Based on the expert correlation table and the preset number of pre-fetched popular experts, perform popular expert identification on multiple hybrid expert modules in the preset hybrid expert model to determine multiple target hybrid expert modules.

[0070] Specifically, step 103 may include the following sub-steps S31-S34:

[0071] Step S31: determining, based on the expert correlation table, expert selection frequencies of a plurality of pre-operation text data in the pre-operation text dataset for a plurality of experts in each hybrid expert module;

[0072] Step S32: sorting the pre-run text data in descending order of the expert selection frequency of each expert in each hybrid expert module;

[0073] Step S33: Among the experts in each mixed expert module, experts whose selection frequencies correspond to the number of pre-set popular experts are selected as popular experts, and the remaining experts are selected as non-popular experts;

[0074] Step S34: Use the hybrid expert module corresponding to each popular expert as the target hybrid expert module.

[0075] It should be noted that during inference, a table lookup is performed based on the expert selections of the first l layers to determine the expert preference of each token in the current layer. For example, assuming that the expert layer in each hybrid expert module has four experts, the gate selects the top-1, l=1, and the larger l is, the more accurate the prefetching is. For the expert activation path of each token in a multi-batch, the present invention performs a table lookup to determine the expert selection preference (expert selection frequency) of each token in the current layer. It then aggregates the preferences of all tokens in the multi-batch and selects the top-K (pre-set number of popular experts to be prefetched) experts in each hybrid expert module for prefetching. That is, the experts with the expert selection frequencies corresponding to the pre-set number of popular experts to be prefetched are selected as popular experts, and the remaining experts are selected as non-popular experts. These K experts will cover the calculation of the majority of tokens.

[0076] For example, assuming that the expert layer in each hybrid expert module includes four experts, and the preset number of pre-fetched popular experts is set to 1, then after sorting in descending order according to the expert selection frequency of each expert in each hybrid expert module, the expert corresponding to the previous expert selection frequency is selected as the popular expert of the corresponding hybrid expert module, and the remaining three experts are selected as non-popular experts. The hybrid expert module corresponding to each popular expert is used as the target hybrid expert module. At this time, the expert layer in the target hybrid expert module includes one popular expert and three non-popular experts.

[0077] In this embodiment, the establishment of the expert correlation table is completed by pre-running. The present invention dynamically updates the correlation table during the inference process, allowing it to better understand the data trends of the current task as the inference progresses. However, updates are not saved locally to prevent data trends in other tasks from affecting the current task. For non-expert tensors, the present invention adopts the same offloading strategy as for dense models, namely pre-fetching the tensor when performing the previous layer of calculation. This is because non-expert tensors only participate in calculations once in a forward pass and are inactive at other times.

[0078] Step 104: Use the preset constraint-aware planning inequality to calculate the transmission time based on the preset number of pre-fetched popular experts and the hybrid expert module to determine the number of input text batches.

[0079] The calculation and transmission time of the hybrid expert module includes the attention layer calculation time, the gating layer calculation time, the popular expert calculation time, the average calculation time of a single expert, the attention layer transmission time, the gating layer transmission time, and the single expert transmission time.

[0080] It should be noted that in order to ensure that multiple calculations in the current layer can completely overlap the transmission of the next layer, it is necessary to consider the batch size and the number n contained in the batch group (the number of input text batches), the hardware capabilities in the environment, that is, the transmission bandwidth between heterogeneous memories and the speed of GPU calculation. However, the hardware capabilities in the environment are fixed, and the batch size is generally a multiple of 4, so the space for selection is not large. If n is too large, a large amount of KV cache will be introduced; if n is too small, the total time of n batch calculations will hardly overlap the time of the next layer transmission.

[0081] Further, see Figure 2 In order to explore the value of n, the present invention Figure 2Some arrows are inserted in the figure, indicating the position where a tensor needs to start calculation. Among them, (1) means that the gated calculation will start. (2) means that the calculation of the popular expert will start. (3) represents that the calculation of the non-popular expert will start. (4) indicates that the calculation of the next attention layer will be carried out. This means that the transmission of the corresponding tensor needs to be completed before this position, indicating that the communication and calculation have been completely overlapped. From this, we can get the inequalities that need to be satisfied at the four key positions, and get the preset constraint-aware planning inequality, which can be expressed as:

[0082] ;

[0083] Where n is the number of input text batches, and n is the smallest integer that satisfies the preset constraint-aware planning inequality; Compute time for the attention layer; Transmission time for the gating layer; Calculate time for the gating layer; Transmit time for a single expert; is the calculation time of popular experts; K is the number of pre-set and pre-fetched popular experts; is the average computation time of a single expert for the i-th expert; is a built-in function; Q is the number of experts; Transfer time for the attention layer.

[0084] It is worth mentioning that the transmission time and computation time will vary with the hardware environment, model, and batch size. The length of the expert queue Q activated at each layer will also not be fixed.

[0085] Furthermore, the hybrid expert module obtained based on the above steps calculates the transmission time, solves the preset constraint-aware planning inequality, and obtains the value of the number of input text batches n. Assuming that the final result is n>=x, then n= [x]. Because n at this time is sufficient to obtain a bubble-free pipeline. If n is further increased, the throughput will be improved but the magnitude will be low. Because there are no bubbles in the pipeline, the improvement at this time mainly comes from the reduction in the number of IO times. This means that increasing n can only bring a slight marginal benefit, and the resulting large amount of KV cache will add a greater burden to storage. Therefore, n only needs to be the smallest integer that satisfies the inequality. In addition, if the value of n is too large, it may be necessary to manually adjust the strategy. Since n is a positive integer, this process is not difficult.

[0086] Step 105: In the multi-level heterogeneous memory space, each target hybrid expert module is used to output the model reasoning result corresponding to the subset of text data to be inferred according to the number of input text batches.

[0087] It should be noted that after completing the deployment of the model (construction of multi-level heterogeneous memory space) and the generation of the expert correlation table, in order to provide efficient services, the present invention designs a tensor scheduling algorithm based on popular experts, and realizes efficient reasoning in the current environment by perceiving and solving hardware constraints.

[0088] Further, see Figure 3 , the multi-batch calculation diagram of the MoE model is as follows Figure 3 As shown, each row is a request batch. Traditional computation methods complete the computation of each batch row by row. However, in resource-constrained environments, where most tensors are offloaded, row-by-row computation makes it difficult to ensure full overlap of communication and computation (i.e., completing the transmission of tensors for the next layer within the timeframe of the current layer's computation).

[0089] In multi-batch scenarios, after the model loads a layer, it performs multiple vertical calculations while simultaneously loading the weights for the next layer. This allows the total time for multiple calculations to cover the time required to load one layer, reducing average latency and I / O times. However, the MoE layer, which includes gating / routing functions and multiple experts, is large and slow to transmit, potentially leading to long wait times. Furthermore, not all experts participate in the calculations, resulting in wasted communication.

[0090] Therefore, the present invention proposes to perform column-by-column calculations, that is, to consider calculations of multiple batches (a batch group) simultaneously to achieve weight sharing.

[0091] Optionally, the target hybrid expert module includes a gating layer, an expert layer, and an attention layer. The expert layer includes a pre-set number of popular experts and a plurality of non-popular experts. The data processing process of the target hybrid expert module can be implemented by executing the following steps S01 to S08:

[0092] Step S01: weighting multiple input text unit data in the input text data subset input to the target hybrid expert module through the gating layer to determine the weight value corresponding to each non-popular expert for each input text unit data;

[0093] Step S02: sort the weight values ​​corresponding to any input text unit data of each non-hot expert in descending order, and select non-hot experts corresponding to a preset number of weight values ​​as non-hot activated experts;

[0094] Step S03: Count the non-popular activated experts corresponding to each input text unit data to determine the activation frequency corresponding to each non-popular activated expert;

[0095] Step S04: sorting the activation frequencies corresponding to the non-hot activation experts in descending order to generate a non-hot activation expert data processing and sorting result;

[0096] Step S05: using a preset number of hot experts to perform calculations on the input text data subsets respectively, and outputting the hot expert calculation results corresponding to the preset number of hot experts;

[0097] Step S06: using the input text data subsets as inputs of non-hot activation experts corresponding to the non-hot activation expert data processing and sorting results, and outputting multiple non-hot activation expert calculation results;

[0098] Step S07: Merge the popular expert calculation results and the calculation results of each non-popular activated expert to generate a merged expert calculation result;

[0099] Step S08: The preset attention mechanism is used to calculate the combined expert calculation results through the attention layer to generate the output model results.

[0100] It should be noted that the present invention splits the MoE layer into the gating / routing function and the experts. When calculating the previous layer, the weights of the gating / routing function and the popular experts are pre-fetched. When calculating the gating / routing function, it is determined whether the selected expert is a popular expert or a selected expert. If not, the weight of the expert is transmitted. However, if Figure 3 As shown, since experts 2 and 4 are popular experts and have already completed pre-fetching, while experts 5 and 3 are still in the process of transferring, the calculation order of [2523424…] will cause the GPU to be blocked at positions 5 and 3, because they are likely not yet completed the transfer. However, the subsequent calculations of 2 and 4 could have been performed directly. Therefore, in order to reduce this unnecessary delay, the present invention further adjusts the order of expert calculations involved in multiple batches, merges the calculations of the same experts, and prioritizes the calculations of popular experts. This is because popular experts are not only transferred to the GPU memory first, but also participate in the largest number of calculations. This can provide more time for subsequent experts who have not completed the transfer. After the calculation of the popular expert is completed, the remaining experts perform calculations in the order they were transferred. In addition, experts that have completed all calculations will be unloaded in advance, rather than waiting until the entire layer of calculations is completed, to reduce the peak value of the video memory.

[0101] Specifically, when performing calculations at the attention layer and the gating layer, the present invention asynchronously transmits popular experts to achieve overlap between layers. When the gating results (activated popular experts, non-popular activated experts) are obtained, when entering the expert layer calculation, the calculation of the popular experts is prioritized (the transmission has been completed, no waiting is required), and the remaining experts (non-popular activated experts) are asynchronously transmitted in descending order, achieving overlap between experts (i.e., overlap within the layer). After the popular experts complete the calculation, the popular expert results are obtained. Then, according to the sorting results of the non-popular activated expert data processing, the non-popular activated experts process the input data in the order in which the input data is processed in the result, and obtain the corresponding non-popular activated expert calculation results. For example, assuming the preset number is 2, if the expert layer in the target hybrid expert module includes one popular expert and three non-popular experts (non-popular expert A, non-popular expert B, non-popular expert C), based on the gating layer for each input text in the input text data subset, After calculation of this data, the weight value corresponding to each non-popular expert for each input text unit data is obtained, and the weight values ​​corresponding to each non-popular expert for any input text unit data are sorted in descending order, and the non-popular experts corresponding to the first two weight values ​​are selected as non-popular activation experts (non-popular activation expert A, non-popular activation expert B), and corresponding statistics are made to obtain the activation frequency corresponding to each non-popular activation expert. After sorting, the non-popular activation expert data processing sorting result for non-popular activation expert A and non-popular activation expert B is: non-popular activation expert B-non-popular activation expert A (that is, non-popular activation expert B completes the calculation of the input data first, and then calculates on non-popular activation expert A); among them, the number of preset pre-fetched popular experts can be set to 1 or 2, and the preset number can be set as needed, and the present invention is not limited to this.

[0102] Further, see Figure 4 , the final calculation of the target hybrid expert module is as follows Figure 4 The computation graph shown in the figure performs calculations and shares the loaded weights among multiple batches. This not only reduces the number of I / O operations to approximately 1 / n of the original number, but also overlaps the time of each I / O operation to achieve a pipeline with almost no bubbles, significantly improving the throughput of model inference.

[0103] It is worth mentioning that, assuming that the number of layers of the preset hybrid expert model is 24, that is, the model contains 24 MoEBlocks (hybrid expert modules), it means that the preset hybrid expert model will include 24 target hybrid expert modules. Each target hybrid expert module calculates and processes the input data based on the step principle of the above target hybrid expert module. That is, after the whole process is cycled multiple times (that is, the number of model layers), the output result is the model inference result. The number of output model inference results corresponds to the number of text data subsets to be inferred corresponding to the number of input text batches. Assuming that the number of input text batches is 3, it means that the number of input text data subsets to be inferred is 3, and the number of output model inference results is also 3.

[0104] For comparison purposes, existing technologies can be used as a reference. With the rapid development of deep learning in recent years, large models have achieved remarkable results in various fields. To scale models without increasing training and inference costs, recent work has introduced sparsely activated mixture-of-experts (MoE) models. MoE models typically use MoE layers instead of feed-forward neural network (FFN) layers. For each input, only a subset of parameters (i.e., experts) are sparsely activated, significantly reducing computational cost. Extensive experiments have demonstrated the superiority of the MoE architecture. However, MoE models face a more severe graphics memory bottleneck during inference than other large models, primarily due to their large parameter size. For example, DeepSeek-V2 has 236B parameters, requiring at least eight state-of-the-art GPUs (such as the A100) for inference. This challenge is particularly acute in resource-constrained environments.

[0105] Offloading is a current mainstream solution for optimizing video memory during large-model inference. This approach can offload tensors not currently involved in the calculation to CPU memory or hard disk, significantly reducing the demand for video memory. However, due to the physical limitations of computation speeds generally exceeding transmission speeds, existing technologies struggle to achieve complete overlap between computation and communication. This often results in numerous "bubbles" in the inference pipeline, leading to low inference efficiency.

[0106] Among them, see Figure 6 ,In recent years, with the rapid development of deep learning, large models have gradually ,penetrated into various fields. Existing large models are mainly based on ,Transformer structure, such as Figure 6(a) Since GShard introduced the novel Mixture of Experts architecture, researchers have continuously explored its enormous potential in improving the performance of large language models. It has gradually become one of the mainstream architectures for large language models. Popular models such as GPT-4, Gemini 1.5, and Mixtral 8x7B all use the MoE architecture. Figure 6 Middle: Transformer block structure and MoE block structure. The Transformer-based large model and the MoE-based large model are respectively composed of multiple Transformer blocks and MoE blocks connected together. E represents expert.

[0107] Different from the large model based on Transformer structure, the hybrid expert model uses MoE layer to replace the feedforward neural network layer, such as Figure 6 (b) shows the MoE layer, which consists of a gating network and multiple experts. The gating function is a key feature of the MoE, discretely selecting participating experts for each input. The most common approach is to select the top k experts for each token. In this case, the gating function is typically a softmax function, activating the top k experts based on the calculated probability distribution, where k is typically 1 or 2. Existing research shows that the expert selection path for each token can indicate the characteristics of the token and can be used to predict expert selection.

[0108] Experts are essentially feedforward neural networks that are sparsely activated based on the results of the gating function. This sparse activation property makes it the most feasible approach for achieving larger models. For each input, the final output of the MoE layer is a weighted sum of the outputs from the selected experts.

[0109] Furthermore, optimizing video memory during large model inference is currently one of the mainstream solutions. During model inference, not all tensors participate in the calculation. By offloading tensors that are not currently involved in the calculation to the CPU memory or hard disk, a large amount of video memory space is freed up. Therefore, in theory, as long as there is enough video memory to accommodate computing resources, and the CPU memory and hard disk are sufficient to offload other resources, model inference can be performed. However, in contrast, offloading strategies are always accompanied by latency issues. Because the relevant tensors must be present in the GPU video memory during calculation, due to transmission bandwidth limitations, the speed of tensor transmission is often slower than the speed of GPU calculation. The GPU can only wait for the tensor transmission to complete before performing calculations, resulting in a waste of computing power.

[0110] In summary, existing model inference processes are primarily targeted at traditional dense models. Directly applying them to MoE models results in performance losses and reduced inference efficiency. This is primarily because they indiscriminately prefetch the next layer while the current layer is being computed, without considering the varying I / O resource requirements of different layers. This approach is efficient for dense models because each Transformer block contains only one FFN. However, MoE blocks contain multiple FFNs (i.e., multiple experts). Directly prefetching the entire MoE layer results in a large number of "bubbles" in the pipeline. Furthermore, because computation speeds currently outstrip physical transmission speeds, the I / O cost of a single expert is significantly greater than the computation time. Existing work focuses on improving the accuracy of prefetching experts. However, even with 100% accurate prefetching, significant bubbles still exist in the pipeline due to the long I / O time required by the expert. When applied to models with a large number of expert choices, the number of "bubbles" increases exponentially, making latency even more significant.

[0111] To address these issues, this paper proposes a text data reasoning method based on a hybrid expert model to achieve high-throughput MoE reasoning in resource-constrained scenarios. This paper first analyzes the inefficiencies of current MoE reasoning pipelines and then proposes an expert-aware multi-batch processing pipeline. This method leverages the high-computation, low-IO characteristics of popular experts to orchestrate multiple batches of computations, minimizing both inter- and intra-layer bubbles in the pipeline. Furthermore, this paper adaptively measures the hardware capabilities of the current environment to generate an optimal execution plan for that environment.

[0112] In an embodiment of the present invention, a method for reasoning about text data based on a hybrid expert model is provided. When reasoning about text data is required, a multi-level heterogeneous memory space is first constructed, and a pre-run text data set is obtained. Then, the pre-run text data set is input into a preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and the hybrid expert module calculation transmission time and the expert correlation table are determined. Based on the expert correlation table and the preset number of pre-fetched hot experts, hot expert identification is performed on multiple hybrid expert modules in the preset hybrid expert model to determine multiple target hybrid expert modules. A preset constraint-aware planning inequality is used to calculate the transmission time according to the preset number of pre-fetched hot experts and the hybrid expert module, and the number of input text batches is determined. Finally, in the multi-level heterogeneous memory space, the pre-run text data set is input into a preset hybrid expert model for pre-run reasoning, and the hybrid expert module calculation transmission time and the expert correlation table are determined. In the storage space, each target hybrid expert module is used to output the model reasoning result corresponding to the subset of text data to be inferred according to the number of input text batches; based on the above scheme, according to the pre-run reasoning process, the hardware capabilities in the current environment are adaptively measured, the hybrid expert module calculation transmission time and the expert correlation table are determined, and the preset constraint perception planning inequality is combined to output the number of input text batches, and the subset of text data to be inferred corresponding to the number of input text batches is used as the input of popular experts based on the expert correlation table to identify multiple target hybrid expert modules, and output the model reasoning result. The present invention focuses on the currently popular hybrid expert model, and greatly compresses the "bubbles" existing in the reasoning pipeline, thereby improving the reasoning efficiency.

[0113] For better explanation, refer to Figure 5 , which shows a schematic flow chart of a method for text data reasoning based on a hybrid expert model, provided in Example 2 of the present invention. It should be noted that this embodiment only briefly describes the general flow of the method for text data reasoning based on a hybrid expert model. The specific implementation of each step can be understood by referring to the relevant content in the aforementioned embodiments and will not be elaborated here. It should be understood that the present invention is not limited to this.

[0114] The general process of the text data reasoning method based on the hybrid expert model proposed in the present invention is divided into two parts: a planning stage and an execution pipeline stage; the planning stage specifically includes: an adaptive model deployment module: constructing a multi-level heterogeneous memory space, including GPU video memory, CPU memory and hard disk; an expert correlation table generation module: inputting the pre-run text data set into the preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and determining the hybrid expert module calculation transmission time and expert correlation table; an IO-computation pipeline planning module: combining hardware constraint perception and a tensor scheduling algorithm based on hot experts, that is, using preset constraint perception planning inequalities to calculate the transmission time according to the preset pre-fetched number of hot experts and the hybrid expert module, and determining the number of input text batches; performing hot expert identification based on the expert correlation table and the preset pre-fetched number of hot experts, and determining multiple target hybrid expert modules; in the execution pipeline stage, the pipeline is executed according to the calculation sequence arranged by the present invention and the input text batch number n obtained by solving the solution, and the model reasoning result is output. Specifically, in a multi-level heterogeneous memory space, each target hybrid expert module is used to output the model reasoning result corresponding to the subset of text data to be inferred according to the number of input text batches.

[0115] The present invention orchestrates multiple batches of computational processes based on popular experts, leveraging their high-computation, low-IO characteristics to orchestrate multiple batches of computations, minimizing "bubbles" in the pipeline and improving inference efficiency. Furthermore, the present invention designs a constraint-aware IO-computation planner (with pre-set constraint-aware planning inequalities) to formulate inference pipelines for different environments. Furthermore, the present invention focuses on the inference efficiency of MoEs in resource-constrained scenarios, achieving high-throughput MoE inference in resource-constrained scenarios by designing an MoE-oriented inference engine. Furthermore, the present invention focuses on the inference efficiency of MoE models in resource-constrained scenarios. Compared to existing technical solutions, the present invention not only focuses on currently popular MoE models, but also significantly compresses the bubbles in the inference pipeline, significantly improving multi-batch processing efficiency while achieving model offloading, and enabling high-throughput inference of MoE models in resource-constrained environments.

[0116] See also Figure 7 , Figure 7 This is a structural block diagram of a text data reasoning device based on a hybrid expert model provided in Example 3 of the present invention.

[0117] The present invention provides a text data reasoning device based on a hybrid expert model, comprising:

[0118] Response module 701, used to respond to inference requests, build multi-level heterogeneous memory spaces, and obtain pre-run text datasets;

[0119] A pre-run reasoning module 702 is configured to input a pre-run text dataset into a preset hybrid expert model in a multi-level heterogeneous memory space for pre-run reasoning, and determine a hybrid expert module calculation transmission time and an expert correlation table;

[0120] Identification module 703, configured to perform hot expert identification on multiple hybrid expert modules in the preset hybrid expert model based on the expert correlation table and the preset number of pre-fetched hot experts, and determine multiple target hybrid expert modules;

[0121] The batch number determination module 704 is used to determine the number of input text batches based on the preset number of pre-fetched popular experts and the transmission time calculated by the hybrid expert module using the preset constraint-aware planning inequality;

[0122] The output result module 705 is used to output the model reasoning result corresponding to the text data subset to be inferred according to the number of input text batches using each target hybrid expert module in the multi-level heterogeneous memory space.

[0123] Furthermore, the identification module 703 is specifically configured to:

[0124] determining, based on the expert relevance table, expert selection frequencies of a plurality of pre-operation text data in the pre-operation text dataset for a plurality of experts in each hybrid expert module;

[0125] sorting the expert selection frequencies of each expert in each hybrid expert module for each pre-run text data in descending order;

[0126] Among the experts in each hybrid expert module, the experts whose selection frequencies correspond to the number of pre-set popular experts are selected as popular experts, and the remaining experts are selected as non-popular experts;

[0127] The hybrid expert module corresponding to each popular expert is used as the target hybrid expert module.

[0128] Furthermore, the target hybrid expert module includes a gating layer, an expert layer, and an attention layer; the expert layer includes a pre-set number of popular experts and a plurality of non-popular experts; the data processing process of the target hybrid expert module is as follows:

[0129] The weight of multiple input text unit data in the input text data subset input to the target hybrid expert module is calculated through the gating layer to determine the weight value corresponding to each non-popular expert for each input text unit data;

[0130] Sort the weight values ​​corresponding to any input text unit data of each non-popular expert in descending order, and select the non-popular experts corresponding to the preset number of weight values ​​as the non-popular activated experts;

[0131] Counting the non-popular activated experts corresponding to each input text unit data to determine the activation frequency corresponding to each non-popular activated expert;

[0132] Sort the activation frequencies corresponding to the non-popular activated experts in descending order to generate the non-popular activated experts data processing and sorting results;

[0133] Use a preset number of popular experts to calculate the input text data subsets respectively, and output the popular expert calculation results corresponding to the preset number of popular experts;

[0134] Using the input text data subsets as the input of the non-hot activation experts corresponding to the non-hot activation expert data processing and sorting results, and outputting multiple non-hot activation expert calculation results;

[0135] Merge the popular expert calculation results and the non-popular activated expert calculation results to generate a merged expert calculation result;

[0136] The preset attention mechanism is used through the attention layer to calculate the combined expert calculation results and generate the output model results.

[0137] Furthermore, the multi-level heterogeneous memory space includes GPU memory, CPU memory and hard disk; GPU memory is used to store tensors required for current calculations and pre-fetched tensors; CPU memory and hard disk are used to store inactive tensors.

[0138] Furthermore, different types of tensors are placed in GPU memory, CPU memory, and hard disk respectively according to preset layer-by-layer allocation conditions.

[0139] Furthermore, the hybrid expert module calculates the transmission time, including the attention layer calculation time, the gating layer calculation time, the popular expert calculation time, the average calculation time of a single expert, the attention layer transmission time, the gating layer transmission time, and the single expert transmission time; the preset constraint-aware planning inequality is:

[0140] ;

[0141] Where n is the number of input text batches, and n is the smallest integer that satisfies the preset constraint-aware planning inequality; Compute time for the attention layer; Transmission time for the gating layer; Calculate time for the gating layer; Transmit time for a single expert; is the calculation time of popular experts; K is the number of pre-set and pre-fetched popular experts; is the average computation time of a single expert for the i-th expert; is a built-in function; Q is the number of experts; Transfer time for the attention layer.

[0142] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0143] An embodiment of the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory; when the computer program is executed by the processor, the processor executes the steps of the text data reasoning method based on the hybrid expert model as described in the first embodiment above.

[0144] An embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the text data reasoning method based on the hybrid expert model as described in the first embodiment are implemented.

[0145] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the text data reasoning method based on the hybrid expert model as described in the first embodiment above.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0147] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0148] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text data reasoning method based on a hybrid expert model, characterized in that: include: Respond to inference requests, build multi-level heterogeneous memory spaces, and obtain pre-run text datasets; Inputting the pre-run text data set into a preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and determining a hybrid expert module calculation transmission time and an expert correlation table; Based on the expert correlation table and the preset number of pre-fetched hot experts, performing hot expert identification on multiple hybrid expert modules in the preset hybrid expert model to determine multiple target hybrid expert modules; The preset constraint-aware planning inequality is used to calculate the transmission time based on the preset number of pre-fetched popular experts and the hybrid expert module to determine the number of input text batches; In the multi-level heterogeneous memory space, each target hybrid expert module is used to output a model reasoning result corresponding to the subset of text data to be inferred according to the number of batches of input texts.

2. The text data reasoning method based on the hybrid expert model according to claim 1 is characterized in that: The method of performing hot expert identification on multiple hybrid expert modules in the preset hybrid expert model based on the expert correlation table and the preset number of hot experts to determine multiple target hybrid expert modules includes: Determining, based on the expert relevance table, expert selection frequencies of a plurality of pre-operation text data in the pre-operation text dataset for a plurality of experts in each of the hybrid expert modules; sorting the expert selection frequencies of the pre-run text data for the experts in the hybrid expert modules in descending order; Among the experts in each of the mixed expert modules, experts corresponding to the expert selection frequencies of the pre-set number of popular experts are selected as popular experts, and the remaining experts are selected as non-popular experts; The hybrid expert module corresponding to each of the popular experts is used as the target hybrid expert module.

3. The text data reasoning method based on the hybrid expert model according to claim 1 is characterized in that: The target hybrid expert module includes a gating layer, an expert layer, and an attention layer; the expert layer includes a preset number of popular experts and a plurality of non-popular experts; the data processing process of the target hybrid expert module is specifically as follows: Performing weight calculation on multiple input text unit data in the input text data subset input to the target hybrid expert module through the gating layer, and determining the weight value corresponding to each non-popular expert for each input text unit data; Sorting the weight values ​​corresponding to any input text unit data of each of the non-hot experts in descending order, and selecting the non-hot experts corresponding to the preset number of weight values ​​as the non-hot activated experts; Collecting statistics on the non-popular activated experts corresponding to each of the input text unit data to determine the activation frequency corresponding to each of the non-popular activated experts; Sorting the activation frequencies corresponding to the non-popular activation experts in descending order to generate a non-popular activation expert data processing and sorting result; Using a preset number of hot experts to perform calculations on the input text data subset respectively, and outputting the hot expert calculation results corresponding to the preset number of hot experts; Using the input text data subsets in sequence as inputs of non-hot activation experts corresponding to the non-hot activation expert data processing and sorting results, and outputting a plurality of non-hot activation expert calculation results; Merging the popular expert calculation results and the non-popular activated expert calculation results to generate a merged expert calculation result; The combined expert calculation results are calculated by the attention layer using a preset attention mechanism to generate an output model result.

4. The text data reasoning method based on hybrid expert model according to claim 1 is characterized in that: The multi-level heterogeneous memory space includes GPU memory, CPU memory and hard disk; the GPU memory is used to store tensors required for current calculation and pre-fetched tensors; the CPU memory and the hard disk are used to store inactivated tensors.

5. The text data reasoning method based on the hybrid expert model according to claim 4 is characterized in that: According to the preset layer-by-layer allocation conditions, different types of tensors are placed in the GPU memory, the CPU memory, and the hard disk respectively.

6. The text data reasoning method based on hybrid expert model according to claim 1 is characterized in that: The hybrid expert module calculates the transmission time including the attention layer calculation time, the gate layer calculation time, the popular expert calculation time, the average calculation time of a single expert, the attention layer transmission time, the gate layer transmission time, and the single expert transmission time; the preset constraint-aware planning inequality is specifically: ； Where n is the number of input text batches, and n is the smallest integer that satisfies the preset constraint-aware planning inequality; Compute time for the attention layer; Transmission time for the gating layer; Calculate time for the gating layer; Transmit time for a single expert; is the calculation time of popular experts; K is the number of pre-set and pre-fetched popular experts; is the average computation time of a single expert for the i-th expert; is a built-in function; Q is the number of experts; Transfer time for the attention layer.

7. A text data reasoning device based on a hybrid expert model, characterized in that: include: The response module is used to respond to inference requests, build multi-level heterogeneous memory spaces, and obtain pre-run text datasets; a pre-run reasoning module, configured to input the pre-run text data set into a preset hybrid expert model in the multi-level heterogeneous memory space for pre-run reasoning, and determine a hybrid expert module calculation transmission time and an expert correlation table; an identification module, configured to perform hot expert identification on a plurality of hybrid expert modules in the preset hybrid expert model based on the expert correlation table and the preset number of hot experts, and determine a plurality of target hybrid expert modules; A batch quantity determination module is used to determine the number of input text batches based on the preset number of pre-fetched popular experts and the transmission time calculated by the hybrid expert module using a preset constraint-aware planning inequality; The output result module is used to output the model reasoning result corresponding to the text data subset to be inferred according to the number of input text batches using each target hybrid expert module in the multi-level heterogeneous memory space.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the text data reasoning method based on the hybrid expert model as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the text data reasoning method based on the hybrid expert model as described in any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer is caused to execute the text data reasoning method based on the hybrid expert model as described in any one of claims 1 to 6.

Images