Model operation method and device
By dividing data into batches in the artificial intelligence model and computed in parallel on each chip, the bandwidth competition problem between computing operators and data exchange specification operators is solved, and the model inference efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202411246468.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-26
AI Technical Summary
In the process of artificial intelligence model inference, the competition in the intermediate-level cache bandwidth between computing operators and data exchange regulations class operators leads to performance degradation, affecting the efficiency of model inference.
By dividing the data to be computed into multiple batches and performing parallel calculations on each chip, avoiding the preemption of L2 bandwidth by batch data loading and data exchange processes, the target chip is used to exchange data to obtain exchanged data, combined with batches The calculation results of data and exchange data are modeled.
It effectively avoids calculation delay, improves the speed and efficiency of model operations, and ensures the accuracy of inference results.
Smart Images

Figure CN119025810A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model calculation method and device. Background Art
[0002] Artificial intelligence models usually include computing operators and data exchange protocol operators. Computing operators refer to operators or instructions that perform actual calculations or processing. Computing operators usually involve mathematical operations, logical operations, or function calculations on data. Data exchange protocol operators mainly involve the movement, exchange, and protocol operations of data. Data exchange protocol operators usually handle the transmission, aggregation, or merging of data between different storage locations (such as different chips).
[0003] At present, when performing artificial intelligence model reasoning, computing operators and data exchange protocol operators are mostly executed in parallel. However, this method will cause the performance of computing operators to decline, thereby affecting the efficiency of model reasoning. Summary of the invention
[0004] The present invention provides a model calculation method and device to solve the defects existing in the related technology.
[0005] The present invention provides a model calculation method, comprising the following steps: Divide the data to be calculated into multiple batches of data, and load each batch of data onto a corresponding chip; On each chip, a data exchange step is performed, wherein each chip exchanges data with a target chip when performing calculations based on batch data on each chip, so that each chip obtains exchange data of each chip from the target chip, and the target chip refers to the remaining chips except each chip; Based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip, model calculation is performed to obtain the model calculation results.
[0006] According to a model operation method provided by the present invention, each chip exchanges data with a target chip, including: Determining target data required by the target chip; Each chip sends the target data to the target chip, and each chip receives the exchange data sent by the target chip.
[0007] According to a model operation method provided by the present invention, the exchange data of each chip is determined based on the operation logic of each chip.
[0008] According to a model operation method provided by the present invention, when performing operations based on batch data on each chip, each chip exchanges data with a target chip, including: Based on the operation logic of each chip, determine the current operation data of each chip from the batch data of each chip; In the process of loading the current operation data into the corresponding chip, counting the amount of data loaded into the corresponding chip; When the amount of data loaded in the corresponding chip reaches the amount of data of the current operation data, operations are performed based on the current operation data loaded by each chip, and data is exchanged with the target chip of the corresponding chip.
[0009] According to a model calculation method provided by the present invention, the model calculation is performed based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip to obtain the model calculation results, including: The calculation results of the batch data on each chip and the calculation results of the exchange data of each chip are added together to obtain the model calculation result.
[0010] According to a model operation method provided by the present invention, each chip is used for text generation model training or reasoning; When each chip is used to perform the text generation model training, the data to be calculated is determined based on the sample input text, and the model calculation result is used to optimize the model parameters; When each chip is used to perform the text generation model inference, the data to be calculated is determined based on the input text, and the model calculation result is used to generate the output text corresponding to the input text.
[0011] The present invention also provides a model computing device, comprising the following modules: A distribution unit, used to divide the data to be calculated into multiple batches of data, and load each batch of data onto a corresponding chip; The exchange unit is used to perform a data exchange step on each chip, wherein the data exchange step includes, when performing operations based on batch data on each chip, each chip exchanges data with a target chip, so that each chip obtains exchange data of each chip from the target chip; the target chip refers to the remaining chips except each chip; The computing unit is used to perform model computing based on the computing results of the batch data on each chip and the computing results of the exchange data of each chip to obtain the model computing results.
[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-mentioned model operation methods is implemented.
[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the model operation method described in any one of the above is implemented.
[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned model operation methods.
[0015] The model calculation method and device provided by the present invention first load each batch of data to the corresponding chip, and then perform calculations on each chip based on the batch data of each chip, and at the same time exchange data with the target chip to obtain exchange data, thereby avoiding the process of loading each batch of data to the corresponding chip and the process of exchanging data between each chip and the target chip to occupy the L2 bandwidth, thereby avoiding calculation delays and improving the model calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1 It is a flow chart of the model operation method provided by the present invention.
[0018] Figure 2 It is a schematic diagram of the execution of the matrix multiplication operator and the data exchange reduction operator provided by the present invention.
[0019] Figure 3 It is a schematic diagram of execution synchronization of the matrix multiplication operator and the data exchange reduction operator provided by the present invention.
[0020] Figure 4 It is a structural schematic diagram of the model operation device provided by the present invention.
[0021] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] At present, when performing artificial intelligence model reasoning, most of the calculation operators and data exchange protocol operators are executed in parallel, specifically: The task data of the computing operators are divided into multiple batches of data, and each chip loads the corresponding batch of data for parallel computing. During the calculation process, each chip performs the task of the data exchange protocol operator, that is, data is exchanged between chips to determine the correctness and completeness of the calculation results. Among them, the data exchange protocol operators are mostly executed through direct memory access (DMA), and the computing operators mostly use single precision computing (SPC).
[0024] However, the main problem facing the current technology is that computing operators and data exchange protocol operators compete for resources on the shared intermediate level cache (Level 2 Cache, L2 cache) bandwidth. This bandwidth competition significantly affects the overall performance, especially in the scenario of large-scale models and data sets. DMA is used for efficient data transmission, while SPC performs floating-point operations. The parallel execution of the two may lead to a bottleneck in the L2 cache bandwidth. Specifically, data exchange protocol operators use DMA to reduce the chip burden, while computing operators frequently access the L2 cache for data processing. When DMA and SPC operations are performed simultaneously, their competition for L2 cache bandwidth will lead to bandwidth saturation, resulting in a decrease in data transmission speed and an increase in computing latency, which ultimately affects the overall performance of the model.
[0025] Figure 1 It is a flow chart of the model operation method provided by the present invention, such as Figure 1 As shown, the method includes step 110 , step 120 and step 130 .
[0026] Step 110: divide the data to be calculated into multiple batches of data, and load each batch of data onto a corresponding chip.
[0027] Here, the data to be calculated can be understood as the task data of the computing operator, that is, the data to be calculated is the inference data of the model. Considering that each chip has its fixed memory capacity, loading all the data to be calculated into a single chip at one time may exceed the memory limit, resulting in the inability to perform the calculation or extremely low efficiency. Therefore, the data to be calculated is divided into multiple batches of data, and each batch of data is allocated and loaded onto the corresponding chip, so as to ensure that the amount of data processed by each chip is within the memory capacity and avoid memory overflow problems. In addition, each batch of data is loaded onto the corresponding chip, so that each chip can execute the computing tasks corresponding to each batch of data in parallel, thereby improving computing efficiency.
[0028] Optionally, if the specifications of the chips are the same, it means that the memory capacity of the chips is the same, and the data to be calculated can be evenly distributed to the chips. For example, if there are n chips, the data to be calculated can be evenly divided into n batches of data, and each batch of data can be allocated to a corresponding chip.
[0029] Among them, the chip here can be an artificial intelligence chip, and the artificial intelligence chip can include a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), etc.
[0030] Step 120, executing a data exchange step on each chip, wherein each chip exchanges data with a target chip when performing operations based on batch data on each chip, so that each chip obtains its exchange data from the target chip, and the target chip refers to the remaining chips except each chip.
[0031] Specifically, after the batch data of each chip is loaded into each chip, each chip performs calculations based on the batch data. In addition, in order to obtain complete calculation results, when each chip performs calculations, it is also necessary to synchronously exchange data with the target chip to obtain exchange data. The exchange data can be the batch data on the target chip, and the target chip refers to the remaining chips except each chip.
[0032] Among them, when the batch data of each chip is calculated, since the batch data has been loaded into the local cache of the chip, there is no need to frequently access the L2 cache, so the batch data of each chip will not occupy the L2 bandwidth during the calculation. When data needs to be exchanged between chips, the data needs to be transferred from the L2 cache of one chip to the L2 cache of another chip. This process involves cross-chip data transmission and requires data exchange through the L2 bandwidth. Therefore, the data exchange between each chip and the target chip will occupy the L2 bandwidth.
[0033] It should be noted that the embodiment of the present invention first executes step 110 and then executes step 120. The consideration is that the process of loading each batch of data to the corresponding chip in step 110 will occupy the bandwidth of the L2 cache. The process of calculating the batch data of each chip in step 120 will not occupy the L2 bandwidth, and the data exchange between each chip and the target chip in step 120 will occupy the L2 bandwidth. If step 110 and step 120 are performed in parallel, since the process of loading each batch of data to the corresponding chip and the process of exchanging data between each chip and the target chip will occupy the L2 bandwidth, there will be a phenomenon of occupying the L2 bandwidth in the two processes, which will lead to a decrease in data transmission speed and an increase in calculation delay, which ultimately affects the calculation speed of the model.
[0034] In this regard, an embodiment of the present invention first executes step 110 and then executes step 120, thereby avoiding the process of loading each batch of data into the corresponding chip and the process of exchanging data between each chip and the target chip from occupying the L2 bandwidth, thereby avoiding computing delays and improving the model inference speed.
[0035] Step 130 : Based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip, perform model calculation to obtain the model calculation results.
[0036] Specifically, since the batch data on each chip is a part of the computation data, that is, the batch data on each chip is not the complete computation data, if the model computation is performed only based on the computation result of the batch data on each chip, the obtained computation result may be biased.
[0037] Considering that the calculation results of the exchange data of each chip are calculated based on the exchange data, and the exchange data is the batch data obtained from the target chip, the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip are combined to obtain a complete model calculation result. Among them, the model calculation result can be applied to model reasoning or model training, which is not specifically limited in the embodiment of the present invention.
[0038] The model calculation method provided by the embodiment of the present invention first loads each batch of data to the corresponding chip, and then performs calculations on each chip based on the batch data of each chip, and at the same time exchanges data with the target chip to obtain exchange data, thereby avoiding the process of loading each batch of data to the corresponding chip and the process of exchanging data between each chip and the target chip to occupy the L2 bandwidth, thereby avoiding calculation delays and improving the model calculation speed.
[0039] Based on the above embodiment, each chip exchanges data with the target chip, including: Determine the target data required by the target chip; Each chip sends target data to the target chip, and each chip receives exchange data sent by the target chip.
[0040] Specifically, the calculation results of the batch data of each chip can be understood as local results. In order to obtain the global results, it is necessary to obtain the data required by each chip from the target chip (that is, the exchange data of each chip). The exchange data of each chip is usually the batch data on the target chip.
[0041] In addition, when each chip obtains its exchange data from the target chip, it also needs to send the target data required by the target chip to the target chip through the data exchange operator, so that each chip can obtain the global result based on its own batch data and the batch data on the target chip, thereby ensuring the accuracy of the inference results.
[0042] Based on any of the above embodiments, the exchange data of each chip is determined based on the operation logic of each chip.
[0043] Specifically, the operation logic of each chip refers to the specific operation rules and calculation methods that the chip follows when performing computing tasks. Based on the operation logic of each chip, the order of computing tasks and what data is needed for each computing task can be determined, that is, the data required by each chip (i.e., the exchange data of each chip) can be determined.
[0044] Based on any of the above embodiments, when performing calculations based on batch data on each chip, each chip exchanges data with the target chip, including: Based on the operation logic of each chip, determine the current operation data of each chip from the batch data of each chip; In the process of loading the current operation data into the corresponding chip, the amount of data loaded into the corresponding chip is counted; When the amount of data loaded in the corresponding chip reaches the amount of data of the current operation data, operations are performed based on the current operation data loaded by each chip, and data is exchanged with the target chip of the corresponding chip.
[0045] Specifically, the current computing data of each chip can be understood as the data required by each chip when executing the current computing task. In the process of loading the current computing data into each chip, the amount of data loaded in the chip is recorded. When the amount of data loaded in the chip reaches the amount of data of the current computing data, it indicates that all the current computing data has been loaded on the chip, and the computing task corresponding to the current computing data can be executed, that is, the computing can be performed based on the current computing data loaded by each chip, and data can be exchanged with the target chip to obtain exchange data.
[0046] It should be noted that the embodiment of the present invention takes into account that the L2 bandwidth will be occupied in the process of loading the current operation data to the corresponding chip. Therefore, when the amount of data loaded in the corresponding chip reaches the amount of data of the current operation data, it can be confirmed that the chip has completed loading the current operation data, and no longer needs to occupy the L2 bandwidth. At this time, the current operation data loaded by each chip is used for calculation, and data is exchanged with the target chip to obtain exchange data. Since the L2 bandwidth will not be occupied in the process of calculation based on the current operation data loaded by each chip, when each chip exchanges data with the target chip, there will be no phenomenon of occupying the L2 bandwidth, thereby improving the data exchange efficiency.
[0047] Based on any of the above embodiments, based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip, the model calculation is performed to obtain the model calculation results, including: The calculation results of the batch data on each chip and the calculation results of the exchange data of each chip are added together to obtain the model calculation result.
[0048] Specifically, the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip are added, and the result of the addition is the global result, which is the model calculation result. The model calculation result can be applied to model training to improve model performance. The model calculation result can also be applied to model reasoning to ensure the accuracy of the reasoning result.
[0049] Based on any of the above embodiments, each chip is used to perform text generation model inference, the data to be calculated is determined based on the input text, and the calculation result is used to generate output text corresponding to the input text.
[0050] Optionally, the input text can be input into the text generation model to extract the data to be operated, and the model operation can be performed on each chip based on the data to be operated using the above-mentioned embodiment method to obtain the model operation result, and the output text corresponding to the input text is generated based on the model operation result.
[0051] Based on any of the above embodiments, each chip is used to perform text generation model training, the data to be calculated is determined based on the sample input text, and the model calculation results are used to optimize the model parameters.
[0052] Optionally, the sample input text can be input into the text generation model to extract the data to be operated, and the model operation can be performed on each chip based on the data to be operated using the above-mentioned embodiment method to obtain the model operation result, and the predicted output text corresponding to the sample input text is generated based on the model operation result. Based on the difference between the predicted output text and the actual output text corresponding to the sample input text, the loss value of the model is determined, and the model parameters are optimized based on the loss value.
[0053] In addition, the above chip can also be applied to the training or reasoning of other neural network models, which may include text graph models, image classification models, target detection models, semantic segmentation models, etc.
[0054] Figure 2 Schematic diagram of the execution of the matrix multiplication operator and the data exchange reduction operator provided by the present invention ( Figure 2 The calculation operator is a matrix multiplication operator for illustration. The calculation operator can also be other operators in the artificial intelligence neural network, such as convolution operator, addition operator, multiplication operator, etc.), such as Figure 2 As shown, the data to be calculated is A 00 , A 10 , A 20 , A 30 , A 01 , A 11 , A 21 , A 31 , B 00 , B 10 , B 01 and B 11 GPU0 and GPU1 are responsible for storing half of the data to be calculated, that is, GPU0 is responsible for storing A 00 , A 10 , A 20 , A 30 , B 00 and B 10 , GPU1 is responsible for saving A 01 , A 11 , A 21 , A 31 , B 01 and B 11 , GPU0 and GPU1 execute matrix multiplication operators. In addition, the operation logic corresponding to GPU0 is to first calculate C' 00 , C' 10 , C' 20 , C' 30 , and then calculate C 00 , C 10 , C 20 , C 30 The operation logic corresponding to GUP1 is to calculate C' first 01 , C' 11 , C' 21 , C' 31 , and then calculate C 01 , C 11 , C 21 , C 31 .
[0055] According to the operation logic of GPU0, when calculating C' 00, C' 10 , C' 20 , C' 30 The required data include A 00 , A 10 , A 20 , A 30 , B 00 , these data are stored in the memory of GPU0, and then C' can be directly calculated 00 , C' 10 , C' 20 , C' 30 Next, calculate C 00 , C 10 , C 20 , C 30 The required data include C' 00 , C' 10 , C' 20 , C' 30 , A 01 , A 11 , A 21 , A 31 , B 10 .
[0056] According to the operation logic of GPU1, when calculating C' 01 , C' 11 , C' 21 , C' 31 The required data include A 01 , A 11 , A 21 , A 31 , B 11 , these data are stored in the memory of GPU1, and then C' can be directly calculated 01 , C' 11 , C' 21 , C' 31 Next, calculate C 01 , C 11 , C 21 , C 31 The required data include C' 01 , C' 11 , C' 21 , C' 31 , A 00 , A 10 , A 20 , A 30 , B 01 .
[0057] Among them, A 01 , A 11 , A 21 , A31 Stored in the memory of GPU1, A 00 , A 10 , A 20 , A 30 The data is stored in the memory of GPU0, so GPU1 and GPU2 need to exchange data through the data exchange protocol operator, that is, GPU0 receives the A sent by GPU1. 01 , A 11 , A 21 , A 31 , and A 00 , A 10 , A 20 , A 30 Sent to GPU1, while GPU1 receives A sent by GPU0 00 , A 10 , A 20 , A 30 , and A 01 , A 11 , A 21 , A 31 Sent to GPU0.
[0058] After completing the data exchange, GPU0 uses the exchanged data (A 01 , A 11 , A 21 , A 31 ) to perform matrix multiplication and combine it with the matrix multiplication result (C') completed in the previous step 01 , C' 11 , C' 21 , C' 31 ) to accumulate and get the final result (C 00 , C 10 , C 20 , C 30 GPU1 is based on the exchanged data (A 00 , A 10 , A 20 , A 30 ) to perform matrix multiplication and combine it with the matrix multiplication result (C') completed in the previous step 01 , C' 11 , C' 21 , C' 31 ) to accumulate and get the final result (C 01 , C 11 , C 21 , C 31 ).
[0059] Figure 3 Schematic diagram of the execution synchronization of the matrix multiplication operator and the data exchange reduction operator provided by the present invention, such as Figure 3As shown, when executing the matrix multiplication operator, the left matrix and the right matrix are loaded first. After the data loading is completed, the matrix multiplication operator sends a synchronization signal to the communication operator, and the communication operator starts to execute data exchange. When loading the left matrix and the right matrix, the communication operator waits for the synchronization signal of the matrix multiplication operator.
[0060] by Figure 2 For example, the operation logic corresponding to GPU0 is to first calculate C' 00 , C' 10 , C' 20 , C' 30 , and then calculate C 00 , C 10 , C 20 , C 30 The operation logic corresponding to GUP1 is to calculate C' first 01 , C' 11 , C' 21 , C' 31 , and then calculate C 01 , C 11 , C 21 , C 31 .
[0061] Under the above operation logic, GPU0 loads the left matrix A 00 and the right matrix B 00 After loading, the left matrix A is processed on GPU0 00 and the right matrix B 00 Perform matrix multiplication operation, and at the same time, the matrix multiplication operator sends a synchronization signal to the communication operator, and the communication operator starts to execute data exchange. GPU1 calculates C 00 Required exchange data A 01 Sent to GPU0 so that GPU0 can exchange data A 01 and B 10 Perform matrix multiplication and add the matrix multiplication result to A 00 and B 00 The matrix multiplication results are accumulated to obtain C 00 Similarly, GPU0 calculates C using a similar method. 10 , C 20 , C 30 .
[0062] Under the above operation logic, GPU1 loads the left matrix A 01 and the right matrix B 11 After loading, the left matrix A is processed on GPU1 01 and the right matrix B 11Perform matrix multiplication operation. At the same time, the matrix multiplication operator sends a synchronization signal to the communication operator. The communication operator starts to execute data exchange. GPU0 calculates C 01 Required exchange data A 00 Sent to GPU1, so that GPU1 can exchange data A 00 and B 01 Perform matrix multiplication and add the matrix multiplication result to A 01 and B 11 The matrix multiplication results are accumulated to obtain C 01 Similarly, GPU1 calculates C using a similar method. 11 , C 21 , C 31 .
[0063] The model operation device provided by the present invention is described below. The model operation device described below and the model operation method described above can be referenced to each other.
[0064] Based on any of the above embodiments, Figure 4 is a schematic diagram of the structure of the model operation device provided by the present invention, such as Figure 4 As shown, the device comprises: The allocation unit 410 is used to divide the data to be calculated into multiple batches of data and load each batch of data onto a corresponding chip; The exchange unit 420 is used to perform a data exchange step on each chip, wherein the data exchange step includes, when performing operations based on batch data on each chip, each chip exchanges data with a target chip, so that each chip obtains exchange data of each chip from the target chip, and the target chip refers to the remaining chips except each chip; The operation unit 430 is used to perform model operation based on the operation results of the batch data on each chip and the operation results of the exchange data of each chip to obtain the model operation results.
[0065] The model computing device provided by the embodiment of the present invention first loads each batch of data to the corresponding chip, and then performs calculations on each chip based on the batch data of each chip, and at the same time exchanges data with the target chip to obtain exchange data, thereby avoiding the process of loading each batch of data to the corresponding chip and the process of exchanging data between each chip and the target chip to occupy the L2 bandwidth, thereby avoiding calculation delays and improving the model calculation speed.
[0066] Based on any of the above embodiments, each chip exchanges data with the target chip, including: Determine the target data required by the target chip; Each chip sends target data to a target chip, and each chip receives exchange data sent by the target chip.
[0067] Based on any of the above embodiments, the exchange data of each chip is determined based on the operation logic of each chip.
[0068] Based on any of the above embodiments, when performing calculations based on batch data on each chip, each chip exchanges data with the target chip, including: Based on the operation logic of each chip, determine the current operation data of each chip from the batch data of each chip; In the process of loading the current operation data into the corresponding chip, the amount of data loaded into the corresponding chip is counted; When the amount of data loaded in the corresponding chip reaches the amount of data of the current operation data, operations are performed based on the current operation data loaded by each chip, and data is exchanged with the target chip of the corresponding chip.
[0069] Based on any of the above embodiments, based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip, the model calculation is performed to obtain the model calculation results, including: The calculation results of the batch data on each chip and the calculation results of the exchange data of each chip are added together to obtain the model calculation result.
[0070] Based on any of the above embodiments, each chip is used to perform text generation model training or reasoning; When each chip is used to perform the text generation model training, the data to be calculated is determined based on the sample input text, and the model calculation result is used to optimize the model parameters; When each chip is used to perform the text generation model inference, the data to be calculated is determined based on the input text, and the model calculation result is used to generate the output text corresponding to the input text.
[0071] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the model operation method, which includes: dividing the data to be operated into multiple batches of data, and loading each batch of data onto a corresponding chip; executing a data exchange step on each chip, wherein the data exchange step includes when performing operations on each chip based on the batch data, each chip exchanges data with a target chip, so that each chip obtains the exchange data of each chip from the target chip, and the target chip refers to the remaining chips except each chip; based on the operation results of the batch data on each chip and the operation results of the exchange data of each chip, the model operation is performed to obtain the model operation result.
[0072] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0073] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the model calculation method provided by the above methods, which method includes: dividing the data to be calculated into multiple batches of data, and loading each batch of data onto a corresponding chip; on each chip, executing a data exchange step, the data exchange step includes when performing calculations based on the batch data on each chip, each chip exchanges data with a target chip, so that each chip obtains the exchange data of each chip from the target chip, and the target chip refers to the remaining chips except each chip; based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip, model calculation is performed to obtain the model calculation results.
[0074] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the model calculation method provided by the above-mentioned methods, the method comprising: dividing the data to be calculated into multiple batches of data, and loading each batch of data onto a corresponding chip; on each chip, performing a data exchange step, the data exchange step comprising, when performing calculations based on the batch data on each chip, each chip exchanging data with a target chip, so that each chip obtains the exchange data of each chip from the target chip, the target chip refers to the remaining chips except each chip; based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip, model calculation is performed to obtain the model calculation results.
[0075] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0076] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A model operation method, characterized in that: include: Divide the data to be calculated into multiple batches of data, and load each batch of data onto a corresponding chip; On each chip, a data exchange step is performed, wherein each chip exchanges data with a target chip when performing calculations based on batch data on each chip, so that each chip obtains exchange data of each chip from the target chip, and the target chip refers to the remaining chips except each chip; Based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip, model calculation is performed to obtain the model calculation results.
2. The model calculation method according to claim 1, characterized in that: Each chip exchanges data with the target chip, including: Determining target data required by the target chip; Each chip sends the target data to the target chip, and each chip receives the exchange data sent by the target chip.
3. The model calculation method according to claim 2, characterized in that: The exchange data of each chip is determined based on the operation logic of each chip.
4. The model calculation method according to any one of claims 1 to 3, characterized in that: When performing calculations based on batch data on each chip, each chip exchanges data with the target chip, including: Based on the operation logic of each chip, determine the current operation data of each chip from the batch data of each chip; In the process of loading the current operation data into the corresponding chip, counting the amount of data loaded into the corresponding chip; When the amount of data loaded in the corresponding chip reaches the amount of data of the current operation data, operations are performed based on the current operation data loaded by each chip, and data is exchanged with the target chip of the corresponding chip.
5. The model calculation method according to any one of claims 1 to 3, characterized in that: The method of performing model calculation based on the calculation results of the batch data on each chip and the calculation results of the exchange data of each chip to obtain the model calculation results includes: The calculation results of the batch data on each chip and the calculation results of the exchange data of each chip are added together to obtain the model calculation result.
6. The model calculation method according to any one of claims 1 to 3, characterized in that: Each chip is used for text generation model training or inference; When each chip is used to perform the text generation model training, the data to be calculated is determined based on the sample input text, and the model calculation result is used to optimize the model parameters; When each chip is used to perform the text generation model inference, the data to be calculated is determined based on the input text, and the model calculation result is used to generate the output text corresponding to the input text.
7. A model computing device, characterized in that: include: A distribution unit, used to divide the data to be calculated into multiple batches of data, and load each batch of data onto a corresponding chip; The exchange unit is used to perform a data exchange step on each chip, wherein the data exchange step includes, when performing operations based on batch data on each chip, each chip exchanges data with a target chip, so that each chip obtains exchange data of each chip from the target chip; the target chip refers to the remaining chips except each chip; The computing unit is used to perform model computing based on the computing results of the batch data on each chip and the computing results of the exchange data of each chip to obtain the model computing results.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the model calculation method according to any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the model operation method according to any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the model operation method according to any one of claims 1 to 6 is implemented.
