Multi-element computing power reasoning apparatus, method, electronic device, program, and storage medium

Abstract

The application discloses a kind of multi-element computing power reasoning device, method, electronic equipment, program and storage medium, applied to model inference technical field, comprising: control unit is used to read the model parameter of inference model and is loaded to first resource group and second resource group, receives inference request and sends to first resource group;The computing power of first acceleration card is greater than the computing power of second acceleration card, and the bandwidth of second acceleration card is greater than the bandwidth of first acceleration card;First resource group is used to execute prefilling operation by each first acceleration card of itself, and the operation result is sent to second resource group by accelerator interconnection switch;Second resource group is used to execute decoding operation for inference request by each second acceleration card of itself, obtain inference result for inference request, solve the technical problem that the utilization rate of resource is not high, achieve efficient inference, improve the technical effect of resource utilization.

Description

Technical Field

[0001] The present application relates to the field of model reasoning technology, and in particular to a multi-component computing power reasoning device, method, electronic device, program and storage medium. Background Art

[0002] Model inference technology is a core technology used in the implementation of generative AI (artificial intelligence). Based on the computational characteristics of generative AI model inference, it can be divided into two phases: prefill and decode. These two phases have significantly different resource requirements.

[0003] The pre-population phase is compute-bound, while the decoding phase is memory-bound. When a generative task is actually executed, the pre-population phase begins, caching the calculated keys and values ​​to form a key-value cache. The decoding phase then proceeds to generate the next generated word based on the context cache. The pre-population phase involves a large number of matrix multiplications and attention value calculations, resulting in high compute utilization but low bandwidth pressure. The decoding phase, on the other hand, relies on the context information obtained in the pre-population phase and previously generated content to predict the next generated word. This relies on the previous content, resulting in low computational load but high bandwidth pressure. This shows that when using the same GPU (Graphics Processing Unit) to perform inference on generative tasks, bandwidth utilization is underutilized during the pre-population phase, while compute utilization is underutilized during the decoding phase, resulting in wasted resources.

[0004] In summary, how to improve resource utilization when implementing model reasoning is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0005] The present application provides a multi-computing power reasoning device, method, electronic device, program and storage medium to at least solve the problem of low resource utilization in related technologies.

[0006] To solve the above technical problems, this application provides the following technical solutions:

[0007] This application provides a multi-computing power reasoning device, comprising:

[0008] a control unit, configured to read model parameters of the inference model and load them into the first resource group and the second resource group, receive an inference request and send the inference request to the first resource group;

[0009] The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator cards is greater than the computing power of the second accelerator cards, and the bandwidth of the second accelerator cards is greater than the bandwidth of the first accelerator cards;

[0010] The first resource group is configured to perform a pre-fill operation for the inference request based on the model parameters through each of its first accelerator cards, and send the operation result to the second resource group through the accelerator interconnect switch;

[0011] The second resource group is configured to perform a decoding operation for the inference request based on the model parameters and the operation result, through each of the second accelerator cards of the second resource group, to obtain an inference result for the inference request;

[0012] An accelerator interconnect switch is communicatively connected to each first accelerator card and each second accelerator card to implement data transmission.

[0013] The present application provides a multi-component computing power reasoning method, which is applied to a first resource group in a multi-component computing power reasoning device, including:

[0014] After the control unit reads the model parameters of the inference model and loads them into the first resource group and the second resource group, receiving the inference request sent by the control unit;

[0015] The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator cards is greater than the computing power of the second accelerator cards, and the bandwidth of the second accelerator cards is greater than the bandwidth of the first accelerator cards;

[0016] Based on the model parameters, the first accelerator cards of the second resource group execute a pre-fill operation for the inference request, and send the operation result to the second resource group through the accelerator interconnect switch, so that the second resource group executes a decoding operation for the inference request based on the model parameters and the operation result through the second accelerator cards of the second resource group, thereby obtaining an inference result for the inference request;

[0017] The accelerator interconnect switch is in communication with each of the first acceleration cards and each of the second acceleration cards to achieve data transmission.

[0018] The present application provides a multi-component computing power reasoning method, which is applied to a second resource group in a multi-component computing power reasoning device, including:

[0019] receiving a computation result sent by the first resource group through the accelerator interconnect switch;

[0020] The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator card is greater than the computing power of the second accelerator card, and the bandwidth of the second accelerator card is greater than the bandwidth of the first accelerator card; the operation result is the operation result obtained by the control unit reading the model parameters of the inference model and loading them into the first resource group and the second resource group, sending the inference request to the first resource group, and then the first resource group performing the pre-fill operation for the inference request based on the model parameters through each of the first accelerator cards in the control unit;

[0021] Based on the model parameters and the calculation result, the second accelerator card performs a decoding operation for the inference request to obtain an inference result for the inference request;

[0022] The accelerator interconnect switch is in communication with each of the first acceleration cards and each of the second acceleration cards to achieve data transmission.

[0023] The present application provides an electronic device, comprising:

[0024] memory for storing computer programs;

[0025] A processor is used to implement the steps of the multi-computing power reasoning method as described above when executing the computer program.

[0026] The present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-computing force reasoning method as described above are implemented.

[0027] The present application provides a computer program product, which, when executed by a processor, implements the steps of the multi-component computing power reasoning method as described above.

[0028] This application solution takes into account that the pre-filling stage requires higher computing power and lower bandwidth pressure, while the decoding stage requires less computing power but places greater bandwidth pressure. Therefore, two different types of accelerator cards can be set up to respectively handle the pre-filling stage and the decoding stage. Specifically, the first resource group includes multiple first accelerator cards, and the second resource group includes multiple second accelerator cards. The computing power of the first accelerator cards is greater than that of the second accelerator cards, and the bandwidth of the second accelerator cards is greater than that of the first accelerator cards. After reading the model parameters of the inference model, the control unit can load them into the first and second resource groups, receive inference requests, and then send the inference requests to the first resource group. The first resource group can then perform pre-filling operations for the inference request using its own first accelerator cards. After the obtained operation results are sent to the second resource group, the second resource group can then perform decoding operations for the inference request using its own second accelerator cards based on the model parameters and operation results, obtain the inference results for the inference request, and complete the entire inference process. It can be seen that in the above inference process, the high-computing-power first accelerator card and the high-bandwidth second accelerator card are responsible for the pre-filling stage and the decoding stage, respectively, thereby effectively improving the utilization of hardware resources in each stage.

[0029] In addition, the present application further takes into account that in actual applications, the number of model parameters is usually very large and cannot be accommodated by the video memory of a single accelerator card. Therefore, it is usually necessary to adopt a model parallel approach and place different layers of the model in different accelerator cards. This means that multiple first accelerator cards are required to perform related operations in the pre-filling stage, and then transmit them to the second accelerator card in the second resource group. During this process, the CPU can be used for transit and transmission. However, since the aggregation, transmission, and distribution of data at different layers all take time, such a solution will cause waiting for device resources, resulting in the inability to fully release the utilization of overall resources.

[0030] In this application, an accelerator interconnection switch is provided that is communicatively connected to each first accelerator card and each second accelerator card, so that the application realizes data transmission through the accelerator interconnection switch without the need for a control unit to transfer and send, that is, the calculation result obtained by the first resource group is directly sent to the second resource group through the accelerator interconnection switch, which is conducive to further improving the overall resource utilization. In addition, it should be pointed out that since each first accelerator card performs calculations in the pre-filling stage, and the accelerator interconnection switch is communicatively connected to each first accelerator card and each second accelerator card, after any first accelerator card completes the pre-filling calculation of the first accelerator card, the corresponding result can be immediately transmitted to the second resource group through the accelerator interconnection switch, that is, the application does not require the control unit to aggregate, transmit, and distribute data at different layers, so it will not cause waiting for equipment resources and will not waste time, which is conducive to further improving the overall resource utilization.

[0031] In summary, the present application solution can achieve efficient reasoning and effectively improve resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1 A schematic diagram of the structure of a multi-computing power inference device provided in a specific embodiment of the present application;

[0034] Figure 2 A schematic diagram of the structure of a multi-computing power reasoning device provided in another specific embodiment of the present application;

[0035] Figure 3 This is a schematic diagram of the transmission principle of cache data obtained by the first accelerator card in a specific embodiment of the present application;

[0036] Figure 4 This is a schematic diagram of a program framework in a specific implementation of this application;

[0037] Figure 5 A flowchart of the implementation of the multi-computing power reasoning method provided in a specific embodiment of the present application applied to the first resource group;

[0038] Figure 6 A flowchart of the implementation of the multi-computing power reasoning method provided in one specific embodiment of the present application applied to the second resource group;

[0039] Figure 7 A schematic diagram of the structure of an electronic device provided in a specific embodiment of the present application;

[0040] Figure 8 This is a schematic structural diagram of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0043] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0044] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a multi-component computing power reasoning device provided in a specific embodiment of the present application. The multi-component computing power reasoning device may include:

[0045] The control unit 30 is configured to read the model parameters of the inference model and load them into the first resource group 10 and the second resource group 20 , receive an inference request and send the inference request to the first resource group 10 ;

[0046] The first resource group 10 includes multiple first accelerator cards, and the second resource group 20 includes multiple second accelerator cards. The computing power of the first accelerator card is greater than the computing power of the second accelerator card, and the bandwidth of the second accelerator card is greater than the bandwidth of the first accelerator card.

[0047] The first resource group 10 is configured to perform pre-fill operations for inference requests based on model parameters using its own first accelerator cards, and send the operation results to the second resource group 20 via the accelerator interconnect switch 40;

[0048] The second resource group 20 is configured to perform a decoding operation for the inference request based on the model parameters and the operation result through each of its second accelerator cards to obtain an inference result for the inference request;

[0049] An accelerator interconnect switch 40 is communicatively connected to each first accelerator card and each second accelerator card to implement data transmission.

[0050] Specifically, in the present application scheme, a PD separation architecture (separation of the Prefill stage and the Decode stage) is set up for the inference model. Specifically, the first resource group 10 includes multiple first acceleration cards, and the first resource group 10 is used to implement the operations of the prefill stage. The second resource group 20 includes multiple second acceleration cards, and the second resource group 20 is used to implement the operations of the decoding stage.

[0051] The reasoning model described in this application can be a generative AI reasoning model, and is usually a large language model (LLM). Of course, the specific model structure can be set and adjusted according to actual needs as long as it can effectively meet user needs.

[0052] The model parameters of the inference model can be stored in the hard disk. Before performing inference, the control unit 30 needs to read the model parameters of the inference model stored in the hard disk and then load them into the first resource group 10 and the second resource group 20. It can be understood that since the first resource group 10 implements the operation of the pre-filling stage and the second resource group 20 implements the operation of the decoding stage, when loading the model parameters into the first resource group 10 and the second resource group 20, the corresponding parts of the model parameters can be loaded into the first resource group 10 and the second resource group 20 according to actual conditions. That is, for the complete model parameters, the part required by the first resource group 10 is loaded into the first resource group 10, and the part required by the second resource group 20 is loaded into the second resource group 20.

[0053] In addition, it should be noted that the control unit 30 needs to load the model parameters into the first resource group 10, and in the present application, the first resource group 10 includes multiple first accelerator cards, so the control unit 30 specifically loads the model parameters into each first accelerator card in the first resource group 10, and because different first accelerator cards can be responsible for the calculation of different layers of the model, when loading the model parameters into each first accelerator card in the first resource group 10, different parts of the model parameters can be loaded into the corresponding first accelerator card, that is, for any first accelerator card, the control unit 30 will load the model parameters required by the first accelerator card into the first accelerator card. Similarly, in the present application, the second resource group 20 includes multiple second accelerator cards, so the control unit 30 specifically loads the model parameters into each second accelerator card in the second resource group 20, and for any second accelerator card, the control unit 30 will load the part of the model parameters required by the second accelerator card into the second accelerator card.

[0054] After the user inputs the inference request, the control unit 30 can obtain the inference request, and then send the inference request to the first resource group 10, so that the first resource group 10 can perform the pre-filling stage operation accordingly. And the first resource group 10 can use its own first acceleration cards to perform the pre-filling operation for the inference request in a pipelined parallel execution manner. The pipelined parallel execution method can effectively improve the execution efficiency of the pre-filling stage. The first acceleration card is a high-computing-power acceleration card, and the calculation count per second is usually used to measure the computing power of the acceleration card. Furthermore, in actual applications, the first acceleration cards in the first resource group 10 can all be acceleration cards of the same model for easy implementation. Similarly, the second acceleration cards in the second resource group 20 can all be acceleration cards of the same model for easy implementation.

[0055] The result obtained during the pre-population phase is the computation result obtained by first resource group 10. It is understood that this result includes the pre-population computation results of each first accelerator card. The computation results obtained during the pre-population phase are sent from first resource group 10 to second resource group 20. Based on the model parameters and computation results, second resource group 20 uses its own second accelerator cards to perform decoding operations for the inference request and obtain an inference result for the inference request.

[0056] In the present application, the calculation results obtained in the pre-filling stage are transmitted through a specially set accelerator interconnection switch 40, so that the process does not require the control unit 30 to transfer and send. In addition, since each first accelerator card is used to perform the pre-filling operation, and the accelerator interconnection switch 40 is in communication with each first accelerator card and each second accelerator card, after any first accelerator card completes its own pre-filling operation, it does not need the control unit 30, but directly transmits the corresponding result to the second resource group 20 through the accelerator interconnection switch 40. That is, the present application does not require the control unit 30 to aggregate, transmit, and distribute data at different layers, so it does not cause waiting for equipment resources and does not waste time, so that the present application can effectively improve the overall resource utilization.

[0057] In a specific embodiment of the present application, the accelerator interconnect switch 40 may be specifically configured to implement data transmission by providing the first accelerator card and the second accelerator card with access to video memory resources between the first accelerator card and the second accelerator card.

[0058] The specific type of the accelerator interconnect switch 40 can be set and adjusted according to actual needs. For example, it can be an NVLink Switch, a UALink Switch, etc. In this embodiment, to ensure data transmission efficiency, the accelerator interconnect switch 40 can support memory resource access between the first accelerator card and the second accelerator card. For example, in a specific scenario, the accelerator interconnect switch 40 can be specifically a UALink Switch. By connecting each first accelerator card and each second accelerator card to the UALink Switch via a data cable, data intercommunication can be achieved between all accelerator cards. Accelerator cards of different models and different protocol types can access each other's memory resources via UALink (Ultra Accelerator Link).

[0059] The specific implementation of the control unit 30 can be set and adjusted according to actual needs. For example, in a specific embodiment of the present application, the control unit 30 includes multiple central processing units that are interconnected for communication, so that multiple central processing units read the model parameters of the inference model from the storage device in a parallel reading manner. The storage device described here can generally be a hard disk.

[0060] This embodiment takes into account that the control unit 30 can be implemented based on a single CPU or based on multiple CPUs. If the control unit 30 is implemented based on multiple CPUs that are connected to each other for communication, the model parameters of the inference model can be read from the storage device in a parallel reading manner, which effectively improves the reading efficiency of the model parameters. In particular, in some occasions, since the number of model parameters is very large, the parallel reading of model parameters by multiple CPUs can effectively reduce the reading time.

[0061] In addition, it can be understood that since the control unit 30 includes multiple central processing units that are interconnected, when the inference model is loaded into the first resource group 10 and the second resource group 20, it can also be executed in parallel by each CPU in the control unit 30, effectively reducing the loading time.

[0062] Further, in a specific embodiment of the present application, please refer to Figure 2 , the control unit 30 includes a first central processing unit and a second central processing unit communicatively connected to the first central processing unit;

[0063] The first central processing unit is communicatively connected to each first acceleration card via a first universal interconnect bus switch, and the second central processing unit is communicatively connected to each second acceleration card via a second universal interconnect bus switch.

[0064] This embodiment takes into account that if the number of CPUs provided in the control unit 30 is too large, the cost will increase. Therefore, a more suitable embodiment is as follows: Figure 2 As shown, the control unit 30 includes a first central processing unit and a second central processing unit in communication with the first central processing unit, for example, a UPI bus is used.

[0065] Since a first central processor and a second central processor are provided, when communicating with a resource group, it can be specifically configured that the first central processor is connected to the first resource group 10 and the second central processor is connected to the second resource group 20, that is, the first central processor and the second central processor are each responsible for the management of the corresponding resource group. Figure 2 In an embodiment, the first central processing unit is specifically connected to each first accelerator card via a first universal interconnect bus switch, and the second central processing unit is specifically connected to each first accelerator card via a second universal interconnect bus switch. Both the first universal interconnect bus switch and the second universal interconnect bus switch can typically be selected as PCIe switches, which are highly versatile and can effectively achieve communication connections between the first central processing unit and each first accelerator card, as well as communication connections between the second central processing unit and each second accelerator card. Furthermore, the provision of the first universal interconnect bus switch and the second universal interconnect bus switch can effectively reduce the pin usage of the first and second central processing units.

[0066] In a specific implementation of the present application, it may further include: a first memory connected to the first central processing unit; and a second memory connected to the second central processing unit.

[0067] This embodiment takes into account that, since a first central processing unit and a second central processing unit are provided, the host memory can be divided into a first memory and a second memory, the first memory being connected to the first central processing unit, and the second memory being connected to the second central processing unit, so that the first central processing unit and the second central processing unit can read the model parameters of the inference model from the storage device in parallel and then store the model parameters. For example, in one scenario, it is necessary to store the parameters of the inference model through multiple hard disks. For the hard disk mounted under the first central processing unit, the model parameters of the inference model therein can be read through the first central processing unit and then stored in the first memory. Similarly, for the hard disk mounted under the second central processing unit, the model parameters of the inference model therein can be read through the second central processing unit and then stored in the second memory.

[0068] In addition, the first central processing unit and the second central processing unit can also realize the transmission of relevant model parameters through communication. For example, in a specific example, for the model parameters read from the hard disk and stored in the first memory, some of them need to be loaded into the second resource group 20, so the first central processing unit will pass the part of the model parameters to the second central processing unit to store them in the second memory, so that the second central processing unit will subsequently load them into the second resource group 20. Similarly, for example, in the above example, for the model parameters read from the hard disk and stored in the second memory, some of them need to be loaded into the first resource group 10, so the second central processing unit will pass the part of the model parameters to the first central processing unit to store them in the first memory, so that the first central processing unit will subsequently load them into the first resource group 10. In short, through the collaboration of the first central processing unit and the second central processing unit, the model parameters can be efficiently read from the hard disk, and then the model parameters of the inference model can be loaded into the first resource group 10 and the second resource group 20.

[0069] In a specific embodiment of the present application, the inference model is a K-layer inference model, the first resource group 10 includes N first accelerator cards, the second resource group 20 includes N second accelerator cards, and the i-th first accelerator card among the N first accelerator cards is used to execute the K-th i Layer to K i+1 The pre-fill operation of the layer, the i-th second accelerator card among the N second accelerator cards is used to perform the K-th i Layer to K i+1 layer decoding operations, so that each first accelerator card sends the cache data obtained during the pre-filling operation to the corresponding second accelerator card through the accelerator interconnection switch;

[0070] Wherein, N is a positive integer not less than 2, and the operation result includes the cache data obtained by each first accelerator card; i is a positive integer ranging from 1 to N, and K is the total number of layers of the inference model and is a positive integer; the Kth i Layer is the starting layer number of the i-th first accelerator card, and is the starting layer number of the i-th second accelerator card; i+1 Layer is the cutoff layer number that the i-th first accelerator card is responsible for, and is the cutoff layer number that the i-th second accelerator card is responsible for.

[0071] This implementation takes into account that each prompt word can be obtained based on the inference request. The first resource group 10 performs parallel computations on all tokens (embedded word vectors of linguistic units such as characters, words, and symbols) of the prompt word in a pipelined manner. This process is called prefill. During the prefill phase, the context of the user input needs to be read and "understood." Each layer of the inference model caches the K (key) and V (value) calculated for each token, forming a KV cache and sending it to the second resource group 20. This second resource group 20 then performs the decoding phase.

[0072] In this embodiment, each first accelerator card can be responsible for the pre-filling operation of the corresponding number of layers to obtain KV cache data. Therefore, the KV cache data obtained by each first accelerator card constitutes the operation result of the entire first resource group 10. In addition, the i-th first accelerator card among the N first accelerator cards is used to perform the K-th i Layer to K i+1 The pre-fill operation of the layer, the i-th second accelerator card among the N second accelerator cards is used to perform the K-th i Layer to K i+1 Layer decoding operations. That is, when a first accelerator card in the first resource group 10 is responsible for the pre-fill operations of a corresponding layer of the inference model, a corresponding second accelerator card in the second resource group 20 is responsible for the decoding operations of these layers. This is equivalent to establishing a correspondence between each first accelerator card in the first resource group 10 and each second accelerator card in the second resource group 20 based on the number of layers of the inference model. For example, in one scenario, if the first first accelerator card is responsible for the pre-fill operations of layers 1 to 15, then the first second accelerator card is responsible for the decoding operations of layers 1 to 15. If the second first accelerator card is responsible for the pre-fill operations of layers 16 to 32, then the second second accelerator card is responsible for the decoding operations of layers 16 to 32.

[0073] It can be seen that in this embodiment, the correspondence between each first accelerator card in the first resource group 10 and each second accelerator card in the second resource group 20 is established by the number of layers of the inference model. Therefore, the KV cache data obtained by any first accelerator card can be conveniently sent to the corresponding second accelerator card. For example Figure 3 In the example, the KV cache data obtained by the four first accelerator cards can be sent to the corresponding second accelerator cards respectively.

[0074] In a specific embodiment of the present invention, sending the calculation result to the second resource group 20 through the accelerator interconnect switch includes:

[0075] For any first accelerator card in the first resource group 10, after completing the pre-fill operation of the corresponding number of layers of the inference model for which it is responsible, the first accelerator card sends the obtained cache data to the second resource group 20 through the accelerator interconnect switch;

[0076] The calculation result includes cache data obtained by each first acceleration card.

[0077] This embodiment takes into account that pre-filling operations can be implemented in a pipelined parallel execution manner through each first accelerator card, and the KV cache obtained by each first accelerator card does not need to be aggregated and distributed by the CPU. In other words, the cache data obtained by any first accelerator card can be immediately sent to the corresponding second accelerator card through the accelerator interconnect switch 40 without waiting.

[0078] In a specific embodiment of the present application, the inference model is a K-layer inference model; when K / a is a positive integer, each first accelerator card is used to perform a pre-fill operation of the K / a layer, so that a first accelerator cards together complete the pre-fill operation of the K layers of the inference model; when K / a is not a positive integer, b first accelerator cards are used to perform Layer pre-fill operation, ab first accelerator cards are used to perform Layer pre-filling operation, so that a first accelerator cards can jointly complete the pre-filling operation of K layers of the inference model;

[0079] Where K is the total number of layers of the inference model and is a positive integer, a is the total number of first accelerator cards in the first resource group, and b is the remainder when K / a is not a positive integer. Indicates rounding down.

[0080] This implementation further takes into account that the first accelerator cards of the present application are generally first accelerator cards of the same model. Therefore, in order to ensure that the time taken by each accelerator card to complete its own pre-filling operation is roughly the same, in this implementation, the K-layer inference model can be evenly divided into layers, so that when K / a is a positive integer, each first accelerator card is used to perform the pre-filling operation of K / a layers, and when K / a is not a positive integer, the difference in the number of layers responsible for any two first accelerator cards is at most 1.

[0081] For example Figure 3In the example, K / a is a positive integer, the first accelerator card is responsible for the pre-filling operations of the 1st to 15th layers, the second first accelerator card is responsible for the pre-filling operations of the 16th to 32nd layers, the third first accelerator card is responsible for the pre-filling operations of the 33rd to 48th layers, and the fourth first accelerator card is responsible for the pre-filling operations of the 49th to 64th layers. In the pre-filling stage, after each first accelerator card completes the calculation and obtains the KV cache data, it can immediately send the KV cache data to the corresponding second accelerator card through the communication channel provided by the accelerator interconnection switch 40. Since the time taken by each accelerator card to complete its own pre-filling operation is roughly the same, when each accelerator card executes the operation of the pre-filling stage in a pipelined parallel manner, there is no need for the first accelerator cards to wait for each other. Of course, the present application solution does not need to aggregate the KV cache data of different first accelerator cards through the CPU and then distribute it to the corresponding second accelerator card. Therefore, it is very efficient and can effectively save time.

[0082] Of course, in this implementation, each first accelerator card is used to perform the pre-fill operation of K / a layers. In some implementations, when K cannot be divided by a, this implementation can also ensure that each first accelerator card is responsible for a certain number of layers. Still taking a=4 as an example, and for example, when the inference model has 61 layers, the remainder of K / a is 1. , that is, b=1 in this example, then one of the four first accelerator cards is responsible for performing a total of 16 layers of pre-fill operations, and each of the remaining three first accelerator cards is responsible for performing a total of 15 layers of pre-fill operations. In this way, these four first accelerator cards together complete a total of 61 layers of pre-fill operations for the inference model.

[0083] also Figure 3 This is a schematic diagram of the transmission principle of the cache data obtained by each first acceleration card, and therefore other components such as the accelerator interconnection switch 40 are not shown.

[0084] In one specific embodiment of the present application, the second resource group 20 may be specifically configured to, upon receiving the complete computation result, execute a decoding operation for the inference request using each of its own second accelerator cards based on the model parameters and the computation result, thereby obtaining an inference result for the inference request. For any second accelerator card, the condition for initiating the decoding operation may be that the second accelerator card has obtained the data required for the decoding operation.

[0085] Still Figure 3 For example, each second accelerator card needs to perform decoding operations for inference requests. Therefore, for Figure 3 In the example, the first second accelerator card can start its decoding operation after receiving the KV cache data of layers 1 to 15. Figure 3The second second accelerator card in the example needs to start its decoding operation after receiving the KV cache data of the 16th to 32nd layers and the decoding result of the first second accelerator card. Therefore, for any second accelerator card, the condition for the second accelerator card to start the decoding operation can be: the second accelerator card obtains the data required for itself to perform this decoding operation. In other words, the second accelerator card needs to obtain the cache data from the corresponding first accelerator card, and needs to obtain the output data of the second accelerator card at the previous level. To this end, in this implementation, the second resource group 20 will not start the decoding stage until it receives the complete operation result, otherwise it can continue to wait to avoid errors. After the second resource group 20 starts the decoding stage, according to the structure of the inference model, each second accelerator card can perform the decoding operation for the inference request in sequence, ensuring the orderly execution of the decoding stage and not prone to errors. In actual applications, after one or more rounds of decoding, the generated data that needs to be fed back to the user can be obtained.

[0086] In a specific embodiment of the present application, performing a pre-fill operation for an inference request and sending the operation result to the second resource group through the accelerator interconnect switch includes: performing a pre-fill operation for the inference request based on a program required for running a first accelerator card deployed in a first container, and sending the operation result to the second resource group through the accelerator interconnect switch.

[0087] Executing a decoding operation for the inference request to obtain an inference result for the inference request, including: executing a decoding operation for the inference request based on a program required for running the second accelerator card deployed in the second container to obtain an inference result for the inference request.

[0088] In this embodiment, a first container and a second container are provided in the operating system of the multi-computing power inference device, so that the programs required for the operation of each first accelerator card are deployed through the first container, and the programs required for the operation of each second accelerator card are deployed through the second container.

[0089] See Figure 4 , is a schematic diagram of the program framework in a specific implementation. The operating system of the multi-computing force reasoning device can provide basic functions such as resource management of general equipment and human-computer interaction, and can provide a running environment for other software. Since the first accelerator card and the second accelerator card are two accelerator cards of different models, two different driver software need to be installed, that is, Figure 4 The first accelerator card driver and the second accelerator card driver shown in the figure can effectively manage these dedicated accelerator cards and provide interface functions for upper-level application software through the first accelerator card driver and the second accelerator card driver.

[0090] In this embodiment, two independent containers, namely the first container and the second container, are set up. The first container is deployed with the programs required for the operation of each first accelerator card, and the second container is deployed with the programs required for the operation of each second accelerator card. Through the containerized design, the programs required for the operation of the first accelerator card and the programs required for the operation of the second accelerator card are respectively deployed in independent containers, which can effectively avoid resource conflicts and interference, and improve the stability, reliability and flexibility of the system. For example, in one scenario, the programs required for the operation of the first accelerator card include program A, and the first accelerator card of this model specifically requires program A to be version 1.0, while the programs required for the operation of the second accelerator card also include program A, and the second accelerator card of this model specifically requires program A to be version 2.0. Since the present application scheme is provided with two independent containers, it can support this requirement, that is, through two independent containers, the programs in the two are independent of each other and do not affect each other, and both reliability and flexibility are improved.

[0091] The program required for the operation of the first accelerator card is deployed in the first container. Therefore, when the first resource group 10 performs operations in the pre-filling stage, it specifically executes pre-filling operations for inference requests based on the program required for the operation of the first accelerator card deployed in the first container, and sends the operation results to the second resource group 20 through the accelerator interconnect switch.

[0092] Similarly, the program required for the operation of the second accelerator card is deployed in the second container. Therefore, when the second resource group 10 performs operations in the decoding stage, it specifically executes decoding operations for the inference request based on the program required for the operation of the second accelerator card deployed in the second container to obtain the inference result for the inference request.

[0093] In a specific embodiment of the present application, a multi-computing power reasoning framework program is also deployed in both the first container and the second container.

[0094] Based on a program required for running the first accelerator card deployed in the first container, a pre-filling operation for the inference request is executed, and the operation result is sent to the second resource group through the accelerator interconnect switch, including: obtaining an execution process code of the pre-filling operation based on a unified upper-layer interface provided by the multi-computing power inference framework program, and executing the pre-filling operation for the inference request by scheduling the program required for running the first accelerator card deployed in the first container according to the execution process code of the pre-filling operation, and sending the operation result to the second resource group through the accelerator interconnect switch;

[0095] Execute a decoding operation for the inference request to obtain an inference result for the inference request, including: obtaining an execution flow code of the decoding operation based on a unified upper-level interface provided by the multi-computing power inference framework program, and executing a decoding operation for the inference request according to the execution flow code of the decoding operation by scheduling a program required for running the second accelerator card deployed in the second container to obtain an inference result for the inference request.

[0096] In this implementation, a multi-computing power reasoning framework program is deployed in the first container. This program schedules the programs required for the operation of each first accelerator card, enabling the first resource group 10 to complete the pre-population operation. The multi-computing power reasoning framework program is also deployed in the second container. This program schedules the programs required for the operation of each second accelerator card, enabling the second resource group 20 to complete the decoding operation. The multi-computing power reasoning framework program provides a unified upper-layer interface.

[0097] See Figure 4 In this example, a multi-computing power reasoning framework program is deployed in both the first container and the second container. This multi-computing power reasoning framework program is an inference framework that can simultaneously support different types of accelerator cards. It is divided into two layers: an upper-layer interface and a lower-layer implementation. Users can call a unified upper-layer interface, while the lower-layer implementation is responsible for implementing various operator functions based on the programming languages ​​and programming models of different accelerator cards. This means that it can schedule the programs required to run the first accelerator card and the second accelerator card, and then load and run the inference model.

[0098] This two-layer design can provide users with a unified upper-layer interface, that is, a unified calling interface. Therefore, when developing upper-layer application code, users do not need to pay attention to the specific underlying accelerator card model. For example, for the "sum" operation, when developing upper-layer application code, there is no need to pay attention to the specific underlying accelerator card model. When the first accelerator card performs the summation operation, the multi-computing power reasoning framework program schedules the program required for the operation of the first accelerator card to complete the summation operation. Similarly, when the second accelerator card performs the summation operation, the multi-computing power reasoning framework program schedules the program required for the operation of the second accelerator card to complete the summation operation. Such differences will not be reflected to users, which improves the convenience of developing upper-layer application code.

[0099] That is to say, in this implementation, the two-layer design of the upper-level interface and the underlying implementation of the multi-computing power inference framework program provides users with a unified calling interface, so that when developing the upper-level application code, there is no need to pay attention to the specific underlying accelerator card model, which simplifies development and maintenance work.

[0100] Therefore, in this embodiment, the execution flow code for the pre-fill operation can be obtained based on the unified upper-level interface provided by the multi-computing power reasoning framework program, and then, according to the execution flow code of the pre-fill operation, the program required for the operation of the first accelerator card deployed in the first container is scheduled to complete the pre-filling stage. Similarly, the execution flow code for the decoding operation can be obtained based on the unified upper-level interface provided by the multi-computing power reasoning framework program, and according to the execution flow code of the decoding operation, the program required for the operation of the second accelerator card deployed in the second container is scheduled to complete the decoding stage.

[0101] In the present application scheme, the programs required for the operation of the first accelerator card and the second accelerator card are scheduled by the multi-computing power reasoning framework program, which can effectively complete the relevant operations of the pre-filling stage and the decoding stage, and can also complete the memory resource monitoring, allocation, recovery, and data flow management.

[0102] In a specific embodiment of the present application, a distributed communication framework program is provided in the operating system of the multi-computing power inference device to complete the communication management between the first accelerator card in the first resource group 10 and the second accelerator card in the second resource group 20 based on the distributed communication framework program.

[0103] Through the distributed communication framework program, the communication functions of different container nodes can be managed. That is, through the distributed communication framework program, the communication between the first accelerator card and the second accelerator card can be managed, for example, including the management of the reception and transmission of KV cache data and the synchronization of status information.

[0104] The present application solution takes into account that the pre-filling stage requires higher computing power and has less bandwidth pressure, while the decoding stage has less computing power but greater bandwidth pressure. Therefore, two different types of accelerator cards can be set to be responsible for the pre-filling stage and the decoding stage respectively. Specifically, the first resource group 10 includes multiple first accelerator cards, and the second resource group 20 includes multiple second accelerator cards. The computing power of the first accelerator card is greater than that of the second accelerator card, and the bandwidth of the second accelerator card is greater than that of the first accelerator card. After the control unit 30 reads the model parameters of the inference model, it can be loaded into the first resource group 10 and the second resource group 20, and can receive the inference request and then send the inference request to the first resource group 10. The first resource group 10 can then perform the pre-filling operation for the inference request through its own first accelerator cards. After the obtained operation results are sent to the second resource group 20, the second resource group 20 can then perform the decoding operation for the inference request based on the model parameters and the operation results through its own second accelerator cards, obtain the inference result for the inference request, and complete the entire inference process. It can be seen that in the above inference process, the high-computing-power first accelerator card and the high-bandwidth second accelerator card are responsible for the pre-filling stage and the decoding stage, respectively, thus effectively improving the utilization of hardware resources in each stage.

[0105] In addition, the present application further takes into account that in actual applications, the number of model parameters is usually very large and cannot be accommodated in the video memory of a single accelerator card. Therefore, it is usually necessary to adopt a model parallel approach and place different layers of the model in different accelerator cards. In other words, multiple first accelerator cards are required to perform related operations in the pre-filling stage, and then transmit them to the second accelerator card in the second resource group 20. During this process, the CPU can be used for transit and transmission. However, since the aggregation, transmission, and distribution of data at different layers all take time, such a solution will cause waiting for device resources, resulting in the inability to fully release the utilization of overall resources.

[0106] In this application, an accelerator interconnection switch 40 is provided that is communicatively connected to each first accelerator card and each second accelerator card, so that the application realizes data transmission through the accelerator interconnection switch 40 without using the control unit 30 for transit transmission, that is, the calculation result obtained by the first resource group 10 is directly sent to the second resource group 20 through the accelerator interconnection switch 40, which is conducive to further improving the overall resource utilization. In addition, it should be pointed out that since each first accelerator card performs calculations in the pre-filling stage, and the accelerator interconnection switch 40 is communicatively connected to each first accelerator card and each second accelerator card, after any first accelerator card completes the pre-filling calculation of the first accelerator card, the corresponding result can be immediately transmitted to the second resource group 20 through the accelerator interconnection switch 40, that is, the application does not require the control unit 30 to aggregate, transmit, and distribute data at different layers, so it will not cause waiting for device resources and will not waste time, which is conducive to further improving the overall resource utilization.

[0107] In summary, the present application solution can achieve efficient reasoning and effectively improve resource utilization.

[0108] Corresponding to the above embodiment of the multi-component computing power reasoning device, the embodiment of the present invention also provides a multi-component computing power reasoning method, which can be referenced in conjunction with the above. Figure 5 The multi-component computing power reasoning method can be applied to the first resource group in the multi-component computing power reasoning device, including the following steps:

[0109] Step S501: After the control unit reads the model parameters of the inference model and loads them into the first resource group and the second resource group, an inference request sent by the control unit is received;

[0110] The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator card is greater than the computing power of the second accelerator card, and the bandwidth of the second accelerator card is greater than the bandwidth of the first accelerator card;

[0111] Step S502: Based on the model parameters, the first accelerator cards of the second resource group execute a pre-fill operation for the inference request, and send the operation result to the second resource group through the accelerator interconnect switch. The second resource group then executes a decoding operation for the inference request based on the model parameters and the operation result through the second accelerator cards of the second resource group, thereby obtaining an inference result for the inference request.

[0112] The accelerator interconnect switch is in communication with each first acceleration card and each second acceleration card to achieve data transmission.

[0113] See Figure 6The multivariate computing power reasoning method can be applied to the second resource group in the multivariate computing power reasoning device, and includes the following steps:

[0114] Step S601: receiving a calculation result sent by a first resource group via an accelerator interconnect switch;

[0115] The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator card is greater than the computing power of the second accelerator card, and the bandwidth of the second accelerator card is greater than the bandwidth of the first accelerator card; the operation result is the control unit reading the model parameters of the inference model and loading them into the first resource group and the second resource group, and after sending the inference request to the first resource group, the first resource group performs the pre-filled operation for the inference request based on the model parameters through its own first accelerator cards.

[0116] Step S602: Based on the model parameters and the calculation results, the second accelerator cards of the computer execute a decoding operation for the inference request to obtain an inference result for the inference request.

[0117] The accelerator interconnect switch is in communication with each first acceleration card and each second acceleration card to achieve data transmission.

[0118] Corresponding to the above embodiments of the multi-computing force reasoning method and the multi-computing force reasoning device, the embodiments of the present invention further provide an electronic device, a computer-readable storage medium and a computer program product, which can be referenced in correspondence with the above.

[0119] See also Figure 7 As shown, the electronic device may include:

[0120] Memory 701, used for storing computer programs;

[0121] The processor 702 is configured to execute a computer program to implement the steps of the multivariate computing power reasoning method in any of the above embodiments.

[0122] The computer program product includes a computer program, which, when executed by a processor, implements the steps of the multivariate computing power reasoning method in any of the above embodiments.

[0123] See Figure 8The computer-readable storage medium 80 stores a computer program 81. When executed by a processor, the computer program 81 implements the steps of the multivariate computational power inference method described in any of the above embodiments. The computer-readable storage medium 50 herein includes random access memory (RAM), internal memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable disk, or any other form of storage medium known in the art.

[0124] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] The above is a detailed introduction to a multi-computing force reasoning device, method, electronic device, program and storage medium provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A multi-element computing power reasoning device, characterized in that: include: a control unit, configured to read model parameters of the inference model and load them into the first resource group and the second resource group, receive an inference request and send the inference request to the first resource group; The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator cards is greater than the computing power of the second accelerator cards, and the bandwidth of the second accelerator cards is greater than the bandwidth of the first accelerator cards; The first resource group is configured to perform a pre-fill operation for the inference request based on the model parameters through each of its first accelerator cards, and send the operation result to the second resource group through the accelerator interconnect switch; The second resource group is configured to perform a decoding operation for the inference request based on the model parameters and the operation result, through each of the second accelerator cards of the second resource group, to obtain an inference result for the inference request; An accelerator interconnect switch that is in communication with each first accelerator card and each second accelerator card and is used to implement data transmission; The accelerator interconnection switch is specifically used for: Data transmission is achieved by providing the first accelerator card and the second accelerator card with access to the video memory resources between the first accelerator card and the second accelerator card.

2. The multivariate arithmetic inference device according to claim 1, characterized in that: The inference model is a K-layer inference model, the first resource group includes N first accelerator cards, the second resource group includes N second accelerator cards, and the i-th first accelerator card among the N first accelerator cards is used to execute the K-th i Layer to K i+1 The pre-fill operation of the layer, the i-th second accelerator card among the N second accelerator cards is used to perform the K-th i Layer to K i+1 layer decoding operation, so that each of the first accelerator cards sends the cache data obtained when performing the pre-fill operation to the corresponding second accelerator card through the accelerator interconnect switch; Wherein, N is a positive integer not less than 2, the operation result includes the cache data obtained by each of the first accelerator cards; i is a positive integer ranging from 1 to N, K is the total number of layers of the inference model and is a positive integer; the Kth i Layer is the starting layer number of the i-th first accelerator card, and is the starting layer number of the i-th second accelerator card; i+1 Layer is the cutoff layer number that the i-th first accelerator card is responsible for, and is the cutoff layer number that the i-th second accelerator card is responsible for.

3. The multivariate arithmetic inference device according to claim 1, characterized in that: The inference model is a K-layer inference model; when K / a is a positive integer, each of the first accelerator cards is used to perform the pre-filling operation of the K / a layers, so that a pieces of the first accelerator cards together complete the pre-filling operation of the K layers of the inference model; when K / a is not a positive integer, b pieces of the first accelerator cards are used to perform The first accelerator card is used to perform the pre-fill operation of the layer. Layer pre-filling operation, so that a pieces of the first accelerator cards can jointly complete the pre-filling operation of a total of K layers of the inference model; Wherein, K is the total number of layers of the inference model and is a positive integer, a is the total number of the first accelerator cards in the first resource group, and b is the remainder when K / a is not a positive integer. Indicates rounding down.

4. The multivariate arithmetic inference device according to claim 1, characterized in that: The control unit includes a plurality of central processing units that are communicatively connected to each other, so that the plurality of central processing units read the model parameters of the inference model from the storage device in a parallel reading manner.

5. The multivariate arithmetic inference device according to claim 4, characterized in that: The control unit includes a first central processing unit and a second central processing unit communicatively connected to the first central processing unit; The first central processing unit is communicatively connected to each of the first accelerator cards via a first universal interconnect bus switch, and the second central processing unit is communicatively connected to each of the second accelerator cards via a second universal interconnect bus switch.

6. The multivariate arithmetic inference device according to claim 1, characterized in that: Sending the calculation result to the second resource group through the accelerator interconnect switch includes: For any first accelerator card in the first resource group, after completing the pre-fill operation of the corresponding number of layers of the inference model for which the first accelerator card is responsible, the first accelerator card sends the obtained cache data to the second resource group through the accelerator interconnect switch; The calculation result includes the cache data obtained by each of the first acceleration cards.

7. The multivariate arithmetic inference device according to claim 1, characterized in that: The first accelerator cards in the first resource group are all accelerator cards of the same model, and the second accelerator cards in the second resource group are all accelerator cards of the same model.

8. The multivariate arithmetic inference device according to any one of claims 1 to 7, characterized in that: Performing a pre-fill operation for the inference request and sending the operation result to the second resource group through the accelerator interconnect switch includes: Execute a pre-filled operation for the inference request based on a program required for running the first accelerator card deployed in the first container, and send the operation result to the second resource group through the accelerator interconnect switch; Performing a decoding operation for the inference request to obtain an inference result for the inference request includes: Performing a decoding operation on the inference request based on a program required for running the second accelerator card deployed in the second container to obtain an inference result on the inference request; Among them, the first container and the second container are set in the operating system of the multi-computing power reasoning device.

9. The multivariate arithmetic inference device according to claim 8, characterized in that: A multi-computing power reasoning framework program is also deployed in both the first container and the second container; The method includes executing a pre-filled operation for the inference request based on a program required for running the first accelerator card deployed in the first container, and sending the operation result to the second resource group through the accelerator interconnect switch, including: Obtaining, based on a unified upper-layer interface provided by the multi-computing power inference framework program, an execution flow code for a pre-filled operation, and executing, according to the execution flow code for the pre-filled operation, the pre-filled operation for the inference request by scheduling a program required for running the first accelerator card deployed in the first container, and sending the operation result to the second resource group through the accelerator interconnect switch; Performing a decoding operation for the inference request to obtain an inference result for the inference request includes: Based on the unified upper-layer interface provided by the multi-computing power reasoning framework program, the execution process code of the decoding operation is obtained, and according to the execution process code of the decoding operation, the program required for the operation of the second accelerator card deployed in the second container is scheduled to execute the decoding operation for the reasoning request, and the reasoning result for the reasoning request is obtained.

10. A multivariate computing power reasoning method, characterized in that: The first resource group used in the multi-computing power inference device according to any one of claims 1 to 9 comprises: After the control unit reads the model parameters of the inference model and loads them into the first resource group and the second resource group, receiving the inference request sent by the control unit; The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator cards is greater than the computing power of the second accelerator cards, and the bandwidth of the second accelerator cards is greater than the bandwidth of the first accelerator cards; Based on the model parameters, the first accelerator cards of the second resource group execute a pre-fill operation for the inference request, and send the operation result to the second resource group through the accelerator interconnect switch, so that the second resource group executes a decoding operation for the inference request based on the model parameters and the operation result through the second accelerator cards of the second resource group, thereby obtaining an inference result for the inference request; The accelerator interconnect switch is in communication with each of the first acceleration cards and each of the second acceleration cards to achieve data transmission.

11. A multivariate computing power reasoning method, characterized in that: The second resource group used in the multi-computing power reasoning device according to any one of claims 1 to 9 comprises: receiving a computation result sent by the first resource group through the accelerator interconnect switch; The first resource group includes multiple first accelerator cards, the second resource group includes multiple second accelerator cards, the computing power of the first accelerator card is greater than the computing power of the second accelerator card, and the bandwidth of the second accelerator card is greater than the bandwidth of the first accelerator card; the operation result is the operation result obtained by the control unit reading the model parameters of the inference model and loading them into the first resource group and the second resource group, sending the inference request to the first resource group, and then the first resource group performing the pre-fill operation for the inference request based on the model parameters through each of the first accelerator cards in the control unit; Based on the model parameters and the calculation result, the second accelerator card performs a decoding operation for the inference request to obtain an inference result for the inference request; The accelerator interconnect switch is in communication with each of the first acceleration cards and each of the second acceleration cards to achieve data transmission.

12. An electronic device, characterized in that: include: memory for storing computer programs; A processor is used to implement the steps of the multivariate computing power reasoning method according to claim 10 or claim 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the multivariate computing power reasoning method according to claim 10 or claim 11 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the multivariate computing power reasoning method according to claim 10 or claim 11 are implemented.

Images