Method and computing device for artificial intelligence chip computing

Abstract

The present invention provides a method and computing device for artificial intelligence chip calculation, the method comprising: extracting a convolution block with data overflow from a convolution network model; obtaining the index range of each convolution output of the convolution block by reverse splitting; actually splitting the convolution input according to the index range obtained by reverse splitting, thereby forming n branches; sending the actually split model to an artificial intelligence platform, thereby calculating the data of the n branches through each convolution layer of the optimized model, splicing the calculated data in the image width or height dimension, and the splicing result is the final output result of the convolution block. According to the technical solution of the present invention, it is possible to solve the data overflow problem of artificial intelligence chips during convolution calculation, speed up the calculation speed of artificial intelligence chips, and improve the utilization rate of artificial intelligence chips.

Description

Technical Field

[0001] The present invention relates to the field of software development technology, and in particular to a method and computing device for artificial intelligence chip computing. Background Art

[0002] With the rapid development of deep learning technology, the inference speed of AI models with large parameter amounts on edge devices has gradually become an important development bottleneck of this technology. In order to solve this problem, AI chips with massive parallel computing capabilities that can accelerate AI computing have emerged.

[0003] In order to accelerate the inference speed of convolutional networks on AI chips, the more mature technologies currently include: parameter quantization, model pruning, operator fusion and splitting methods. Among them, quantization technology converts floating-point model parameters (weights and activations) into low-bit integer values, which not only reduces the size of the model but also speeds up the inference speed of the model. However, the quantization process will inevitably introduce truncation errors, thereby affecting the accuracy of model predictions; pruning technology reduces the number of model parameters by trimming unnecessary channel weights in the convolution layer. However, the pruned model usually needs fine-tuning training, which is time-consuming and difficult to restore to the prediction effect of the original model.

[0004] When AI chip devices perform parallel calculations on vectors or matrices, they usually temporarily store input and output data in the internal cache to reduce the time it takes to read and write data. However, due to the size limit of the internal cache, when the amount of input and output data exceeds the rated internal cache size, the data will overflow to the external storage device, causing the output data of the current calculation and the input data of the next calculation to need to be moved back and forth between the chip and the external storage device, seriously affecting the computing efficiency of the AI ​​chip.

[0005] To this end, a technical solution is needed that can solve the data overflow problem of artificial intelligence chips during convolution calculations, speed up the calculation speed of artificial intelligence chips, and improve the utilization rate of artificial intelligence chips. Summary of the invention

[0006] The present invention aims to provide a method and computing device for preventing convolution data from overflowing from an AI chip, which can increase the computing efficiency of the AI ​​chip and improve the utilization rate of the AI ​​chip.

[0007] According to one aspect of the present invention, a method for preventing convolution data from overflowing from an AI chip is provided, the method comprising:

[0008] Extract the convolutional blocks with data overflow from the convolutional network model;

[0009] According to the index range obtained by reverse splitting, the convolution input is actually split to form n branches, where n is an integer;

[0010] The actual split model is sent to the artificial intelligence platform, so that the data of the n branches are calculated through each convolution layer of the optimized model, and the calculated data are spliced ​​in the image width or height dimension. The splicing result is the final output result of the convolution block.

[0011] According to some embodiments, obtaining the index range of each layer of convolution output of the convolution block by reverse splitting includes:

[0012] The last layer of convolution output in the convolution block is pre-split into n parts in the image width or height dimension to obtain the Nth index range after the last layer of convolution output is split, where N and n are integers.

[0013] According to some embodiments, obtaining the index range of each layer of convolution output of the convolution block by reverse splitting further includes:

[0014] According to the Nth index range after the last layer of convolution output is split, reverse deduction is performed in sequence until the 0th index range required for the first layer of convolution input split is derived.

[0015] According to some embodiments, the number of splits is gradually increased from 1 until the optimal number of splits n is determined, and the optimal number of splits satisfies the requirement of preventing data overflow and the number of splits is the smallest.

[0016] According to some embodiments, when the sum of the input and output data sizes of each convolutional layer in the convolutional block is less than or equal to the cache size of the artificial intelligence chip, the number of splits is determined to be optimal.

[0017] According to some embodiments, the data size difference between the portion with the largest data size and the portion with the smallest data size is minimized during the splitting.

[0018] According to some embodiments, all convolutional layers in the convolutional block are replicated n times, and the input data in the n branches are calculated respectively.

[0019] According to some embodiments, the output results of each branch after the split are concatenated in the split dimension, thereby ensuring that the output results of the convolution blocks before and after the split are consistent in shape and value.

[0020] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method as described above is implemented.

[0021] According to another aspect of the present invention, there is provided a computing device, comprising:

[0022] Processor; and

[0023] A memory stores a computer program, and when the computer program is executed by the processor, the method described in any one of the above items is implemented.

[0024] According to an embodiment of the present invention, the last convolution output in the convolution block with data overflow is pre-split into n parts in the image height or width dimension, and after obtaining the index range after the last convolution output is split, reverse deduction is performed in sequence until the index range of the first convolution input split is derived, and the input data of the first convolution is truly split, the branch data is calculated through each convolution layer, and the calculated data is spliced ​​in the selected dimension, and the splicing result is the final output result. The present invention can solve the data overflow problem during chip convolution calculation, and improve the utilization rate and computing efficiency of the chip.

[0025] According to some embodiments, by pre-splitting the output of the convolutional layer in a specified dimension, the parallel processing capabilities of modern computing hardware can be effectively utilized, and each split part can be continuously calculated inside the chip, thereby significantly reducing the computing time and accelerating the model training or inference process.

[0026] According to some embodiments, for large-scale data sets, a single device may not be able to accommodate the entire model or data. By splitting the data and computing tasks, the amount of memory required at any time can be reduced, allowing the model to run in an environment with limited resources.

[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0029] Figure 1 An overall flow chart of a method for preventing convolution data from overflowing an AI chip according to an example embodiment is shown.

[0030] Figure 2 A detailed schematic diagram illustrating a method of preventing convolution data from overflowing an AI chip according to example embodiments.

[0031] Figure 3 A schematic diagram showing a comparison of forward splitting and backward splitting of a convolution block according to an example embodiment.

[0032] Figure 4 A schematic diagram of a method for traversing the optimal number of splits according to an example embodiment is shown.

[0033] Figure 5A block diagram of a computing device is shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0035] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0036] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0037] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0038] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0039] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0040] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0041] AI chips are specially designed to accelerate artificial intelligence algorithms. At present, these artificial intelligence algorithms are generally based on deep learning algorithms. Whether in the field of computer vision or natural language processing, these deep learning algorithms are currently based on convolutional computing. Therefore, how to accelerate the inference speed of convolutional networks on AI chips is a key issue in realizing the actual deployment of models.

[0042] In order to accelerate the reasoning speed of convolutional networks on AI chips, the more mature technologies currently include parameter quantization, model pruning, operator fusion and splitting. Among them, quantization technology converts floating-point model parameters (weights and activations) into low-bit integer values, which not only reduces the size of the model but also speeds up the reasoning speed of the model. However, the quantization process will inevitably introduce truncation errors, which will affect the accuracy of model predictions; pruning technology is to reduce the number of model parameters by cutting unnecessary channel weights in the convolution layer. However, the pruned model usually needs to be fine-tuned and trained, which is time-consuming and difficult to restore to the original model's prediction effect; operator fusion and splitting methods are to merge and split some operators with serious time-consuming problems based on the characteristics of hardware device calculations without affecting the model's calculation logic. Operator fusion and splitting methods usually do not affect the calculation results of the model, but optimize the corresponding operators according to the calculation characteristics of the hardware, thereby achieving the effect of accelerating model reasoning.

[0043] When AI chip devices perform parallel calculations of vectors or matrices, they temporarily store input and output data in the internal cache to reduce the time it takes to read and write data. However, due to the size limit of its internal cache, when the amount of input and output data is greater than the rated internal cache size, the data will overflow into the external memory device, causing both the output data of the current calculation and the input data of the next calculation to be moved back and forth between the chip and the external storage device. In actual board testing, it was found that the time taken to move this part of the data was significantly greater than the time taken to calculate inside the chip, that is, the chip was in a process of waiting for data for a long time, and this problem seriously affected the computing efficiency of the AI ​​chip. Therefore, the input of individual operators with larger data volumes can be split to ensure that the amount of data for each branch split out can be placed in the internal cache of the chip, thereby effectively solving the problem of data overflow inside the chip.

[0044] When forward splitting continuous convolution operators without 1x1 kernel size, the output data of each convolution operator needs to be re-aligned, that is, slicing and splicing operations need to be performed between the outputs of each branch split out, so as to ensure that the data entering the next convolution operator after the split is consistent with that before the split. The board test results show that these alignment operations will not only introduce additional data calculation and transportation time, but also interrupt the continuous matrix multiplication calculation, resulting in the chip computing efficiency failing to achieve the expected optimization effect; in addition, when splitting the input of the convolution layer with a non-1x1 kernel size, there will be overlapping parts. These overlapping parts are likely to cause a sudden increase in data volume and computing volume after the split. The transportation time of the newly added data may cover the benefits brought by the split, resulting in a decrease in the computing efficiency of the chip.

[0045] To this end, the present invention proposes a method for preventing convolution data from overflowing from an AI chip, which can solve the data overflow problem during chip convolution calculation and improve the chip utilization and computing efficiency.

[0046] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0047] Figure 1 An overall flow chart of a method for preventing convolution data from overflowing from an AI chip according to an example embodiment is shown.

[0048] See also Figure 1 ,In S101, the convolutional blocks with data overflow are extracted from the convolutional network model.

[0049] Extract the convolution blocks with data overflow from the original model, aiming to identify and separately process those network parts that may cause memory overflow or computing bottlenecks under specific hardware resource constraints. Assume that the convolution block contains N layers of sequentially connected convolution layers, and the number of splits to be split is n. The specific number of splits will be determined in the subsequent screening process.

[0050] According to some embodiments, the convolution block includes multiple layers of sequentially connected convolution layers, and the specific number of splits of the convolution block is determined by a screening process.

[0051] According to some embodiments, a performance analysis tool is used to perform performance analysis, and based on the results of the performance analysis, those convolutional layers or convolutional blocks that cause data overflow due to high computational complexity or large-size feature maps are identified. Once the data overflow is determined, it is traced back to the input end and traced down to the output end, and a convolutional block containing the overflow data and its directly connected layers before and after is defined, and the defined convolutional block is extracted from the original model.

[0052] In S103, the index range of each layer of convolution output of the convolution block is obtained by reverse splitting.

[0053] According to some embodiments, the last layer of convolution output in the convolution block is pre-split into n parts in the image width or height dimension to obtain the Nth index range after the last layer of convolution output is split, where N and n are integers. According to the Nth index range after the last layer of convolution output is split, reverse deduction is performed in sequence until the 0th index range required for splitting the first layer of convolution input is derived.

[0054] According to some embodiments, the selected dimension ensures that the amount of data in each of the splits is equivalent. The split number is gradually increased from 1, and when the sum of the size of the input data and the output data of each convolution layer in the convolution block is less than or equal to the cache size of the chip, the split number is the optimal split number.

[0055] The last convolution output in the convolution block is split into n parts in the selected dimension. It is not actually split at this time, but is only used to derive the index. Get the index range S after the last convolution output is split N .

[0056] For example, the convolution output shape format of the Nth layer, that is, the last layer, is NHWC, where N is the number of layers, H is the height, W is the width, and C is the channel. The output data of the Nth layer is 1x10x20x64, which is split into three parts in the H dimension. The data volume of each part is 3, 3, and 4, and the corresponding index range S NThey are: [0,2], [3,5], [6,9] respectively. It should be noted that when splitting, the data volume of each portion should be equal, so that the difference in data volume between the portion with the largest data volume and the portion with the smallest data volume is minimized, as shown in 3, 3, and 4 above, so as to avoid different overflow situations of different data blocks.

[0057] According to some embodiments, based on the Nth index range after the last layer of convolution output is split, the third index range of the last layer of convolution input split is reversely derived, and the third index range is the index range after the last layer of convolution output is split.

[0058] According to the Nth index range SN after the last layer of convolution output is split, the index range SN-1 after the previous layer of output is split is reversely derived.

[0059] For example, the Nth convolution layer is a convolution layer with a convolution kernel size of 3x3, its step size and expansion rate are both 1, and no padding is required. Then, according to the Nth index range SN in S103, it can be deduced that the index range after the Nth convolution input is split is: [0,4], [3,7], [6,11], which is also the index range after the N-1th convolution output is split, recorded as SN-1. In this way, reverse deduction is performed in sequence until the third index range S1 after the first layer output is split is derived. At this time, reverse deduction is performed again to obtain the 0th index range S0 required for the first layer convolution input split.

[0060] In S105, the convolution input is actually split according to the index range obtained by reverse splitting, thereby forming n branches.

[0061] According to some embodiments, any one of the n branches passes through all convolutional layers in the convolutional block individually. All convolutional layers in the convolutional block are replicated n times to calculate data in the n branches respectively.

[0062] According to the 0th index range S0, the input data of the first layer of convolution, that is, the input data of the entire convolution block, is truly split to form n branches. Each split branch must pass through the N convolution layers in the convolution block separately, that is, the N convolution layers in the convolution block need to be copied n times to calculate the data in the n branches respectively.

[0063] In S107, the model after actual splitting is sent to the artificial intelligence platform, so that the data of the n branches are calculated through each convolution layer of the optimized model, and the calculated data are spliced ​​in the width or height dimension of the image, and the splicing result is the final output result of the convolution block.

[0064] According to some embodiments, the output data of the n branches are concatenated in a selected dimension, and the concatenated result is the final output result of the convolution block.

[0065] In order to avoid the problem of introducing additional alignment operations when the convolution block is forward split, the method of the present invention performs a reverse splitting operation on the continuous convolution layers, so that the calculation data on each branch will not overflow to the external device, and the problem of the output data of each layer of convolution needs to be aligned, which significantly speeds up the calculation speed of the chip; for the problem that convolution splitting may introduce more additional calculations in the overlapping area, this patent traverses the number of splits incrementally and screens out the minimum number of splits that will not overflow the data, thereby reducing the amount of additional calculations in the overlapping area to a minimum.

[0066] Figure 2 A schematic diagram showing an overall flow of a method for preventing convolution data from overflowing from an AI chip according to an example embodiment.

[0067] In order to verify the effectiveness of the patented method, the Caisa4 chip was used as an example to test several common deep learning models before and after optimization, and compared with the forward splitting of the convolution block. The Caisa 4 chip has a vector engine and a matrix engine specifically used to accelerate the calculation of AI operators. The calculation of the convolution operation is completed by the matrix engine, and the vector engine and the matrix engine each have their own dedicated cache device, but the capacity is limited, about 2048KB.

[0068] Forward splitting of convolution blocks means that the input of each convolution layer is split only according to the calculation requirements of the current convolution layer, without considering subsequent convolutions. The specific process is as follows: Figure 3 As shown (taking the convolution layer with a kernel size of 3x3 as an example). It can be seen that after the input is split according to the forward splitting method, the output of the first layer of convolution cannot be directly used as the input of the second layer of convolution. Each output result of the first layer must be re-filled (such as: the first output needs to fill the number 3 data on the far right), that is, the output data needs to be aligned, otherwise it will cause data loss problems. This alignment operation will not only increase the time consumption of data slicing, but most importantly, it will interrupt the continuous matrix calculation, causing the matrix engine of the artificial intelligence chip to be in a waiting state, reducing the computing efficiency of the artificial intelligence chip. The reverse splitting process relies on the output of the last layer for reverse deduction, which avoids the alignment operation well, so that the matrix engine can always maintain a continuous calculation state. It is worth noting that although the amount of data at the input end of the reverse splitting is increased compared to the forward splitting (such as: Figure 3The input of the forward split has 16 nodes and the input of the reverse split has 20 nodes), but the increased matrix calculation time is much lower than the engine waiting time introduced by the alignment operation, which is reflected in the test results below. In addition, this patent also provides a method for selecting the minimum number of splits, thereby ensuring that the additional convolution calculation is minimized.

[0069] When the amount of data in the convolution is larger than the cache device capacity of the matrix engine, the data will be moved to the memory, causing the matrix engine to be idle and waiting during the calculation process. In order to verify the optimization effect of this technology, the Faster-RCNN, Enet, ConvNext, and CenterNet models were tested as examples. The input was unified as a single 640x640x3 image, and a single-threaded test was performed. During optimization, all convolution blocks in each model with a total input and output data volume greater than 2048KB are split in the H dimension according to the above process, and the specific number of splits is automatically determined according to the above method for solving the optimal number of splits. The specific test results are as follows (unit: milliseconds):

[0070] Table 1

[0071]

[0072] From the above test results, it can be seen that the forward splitting of the convolution block has certain time-consuming benefits, but it is not obvious, and it is quite different from the benefits brought by the reverse splitting. This is mainly because the alignment operation will be introduced between each layer of convolution during the forward splitting. The alignment operation will bring additional data handling process and interrupt the continuous calculation process of the matrix, affecting the computing efficiency of the chip. The reverse splitting technology of the convolution block avoids this problem well, ensuring that the output of the current convolution layer can be directly used as the input of the next layer of convolution. In addition, it can be seen from the test results that the larger the model parameters, the higher the time-consuming benefits after the reverse splitting. This is because the model with a larger number of parameters often has more convolution layers, and the larger the input and output channels of the convolution layer, the more data handling time can be saved after splitting.

[0073] It is worth noting that for convolutional networks with continuous residual structures, after splitting the convolutional blocks, it is not necessary to splice them immediately. Instead, they can be added to the branches from the short-circuit layer first, and then the output results can be spliced. This can avoid data overflow problems that may be caused by addition calculations.

[0074] Figure 4 A schematic diagram of a method for traversing the optimal number of splits according to an example embodiment is shown.

[0075] The more data is split, the better. The more data is split, the more overlapping areas there are. Figure 3The larger the area [3,4] and [6,7] in the Nth layer convolution input, the more additional computation is required on the chip. When this additional computation is too large, it may even mask the benefits of splitting.

[0076] Therefore, in order to determine the optimal number of splits n, the number of splits is gradually increased from the initial value 1. When all the convolutional layer data in the convolution block will not overflow to the external device, that is, the sum of the input and output data size of each convolutional layer is less than or equal to the chip cache size, the number of splits is the optimal number of splits. This ensures that while preventing data overflow, the number of splits is minimized, so that the additional overlap area introduced is minimized, and the additional amount of calculation is also minimized.

[0077] Figure 5 A block diagram of a computing device is shown according to an exemplary embodiment.

[0078] like Figure 5 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.

[0079] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions. According to some embodiments, the computing device 30 may also include a high-performance graphics card (GPU) 20 for accelerating the processor 12.

[0080] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 may read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.

[0081] The computing device 30 may also communicate with one or more networks via the network interface 16. The network interface 16 may be a wireless network interface.

[0082] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.

[0083] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.

[0084] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the program is executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0085] An embodiment of the present invention further provides a computer program product, which includes a computer program. The computer program is operable to enable a computer to execute part or all of the steps of any one of the methods described in the above method embodiments.

[0086] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0087] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0088] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0089] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0093] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.

Claims

1. A method for artificial intelligence chip computing, characterized in that: include: Extract the convolutional blocks with data overflow from the convolutional network model; Obtaining the index range of each layer of convolution output of the convolution block by reverse splitting; According to the index range obtained by reverse splitting, the convolution input is actually split to form n branches, where n is an integer; The actual split model is sent to the artificial intelligence platform, so that the data of the n branches are calculated through each convolution layer of the optimized model, and the calculated data are spliced ​​in the image width or height dimension. The splicing result is the final output result of the convolution block.

2. The method according to claim 1, characterized in that The index range of each convolution output of the convolution block is obtained by reverse splitting, including: The last layer of convolution output in the convolution block is pre-split into n parts in the image width or height dimension to obtain the Nth index range after the last layer of convolution output is split, where N is an integer.

3. The method according to claim 2, characterized in that The index range of each convolution output of the convolution block is obtained by reverse splitting, and also includes: According to the Nth index range after the last layer of convolution output is split, reverse deduction is performed in sequence until the 0th index range required for the first layer of convolution input split is derived.

4. The method according to claim 1, characterized in that: Also includes: The number of splits is gradually increased from 1 until the optimal number of splits n is determined, which satisfies the requirement of preventing data overflow and has the smallest number of splits.

5. The method according to claim 4, characterized in that When the sum of the input and output data sizes of each convolutional layer in the convolutional block is less than or equal to the cache size of the artificial intelligence chip, it is determined that the number of splits is optimal.

6. The method according to claim 1, characterized in that When splitting, the difference in data volume between the portion with the largest data volume and the portion with the smallest data volume is minimized.

7. The method according to claim 6, characterized in that The data of the n branches are calculated through each convolution layer of the optimized model, including: All convolutional layers in the convolutional block are copied into n copies according to the split n branches, and the input data in the n branches are calculated respectively.

8. The method according to claim 1, characterized in that The output results of each branch after splitting are concatenated along the splitting dimension to ensure that the output results of the convolution blocks before and after splitting are consistent in shape and value.

9. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed by a processor.

10. A computing device, characterized in that: include: processor; as well as A memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images