Method for optimizing data access in parallel computing of artificial intelligence chip and artificial intelligence chip

Abstract

The present invention provides a method for optimizing data access in parallel computing of an artificial intelligence chip and an artificial intelligence chip. The method includes: determining the computational parallelism of an artificial intelligence chip; obtaining first two-dimensional data for parallel computing of the artificial intelligence chip, wherein the first two-dimensional data meets the requirement that the second dimension multiplied by a first integer is the computational parallelism, and the first dimension divided by the first integer is a second integer multiple of the computational parallelism, and the first integer is greater than or equal to 1; performing a preprocessing operation on the first two-dimensional data inside the artificial intelligence chip, wherein the preprocessing operation includes a combination of a deformation operation and a transposition operation, and transposing the first two-dimensional data into data that meets the parallelism requirement. The technical solution according to the present invention can reduce the inference time of the model on the chip and increase the utilization rate of the artificial intelligence chip.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence chip technology, and in particular to a method for optimizing data access in parallel computing of an artificial intelligence chip, an artificial intelligence chip, and a computer program product. Background Art

[0002] As deep learning technology develops rapidly, artificial intelligence chips specifically used to accelerate deep learning models are gradually gaining more and more attention. Broadly speaking, any chip that can run artificial intelligence algorithms is called an artificial intelligence chip (AI chip). However, AI chips in the general sense refer to chips that have been specially designed to accelerate artificial intelligence algorithms, and these artificial intelligence algorithms are generally based on deep learning algorithms.

[0003] In deep learning models, common data access operations include data transposition and data slicing. Data transposition refers to converting multidimensional data between different dimensions, while data slicing refers to extracting data in a certain dimension. Therefore, both operations involve data access operations, and as the amount of data increases, the access time will increase significantly.

[0004] At present, the core of mainstream AI chips mainly uses multiplication and addition computing acceleration arrays to accelerate the most important convolution operations in convolutional neural networks. Such data processing methods usually have parallel requirements for the data to be calculated. The so-called computational parallelism refers to the smallest unit of data accessed by the hardware each time. The parallelism requirement is met only when the size of the last dimension of the data is an integer multiple of the computational parallelism. When the amount of data to be accessed meets the parallelism requirement, fast access can be achieved inside the chip. Otherwise, the data needs to be output to an external storage device for access. Therefore, when the artificial intelligence chip performs data access operations such as data transposition and data slicing, if the data to be accessed does not meet the parallelism requirements, it cannot be accessed inside it, but needs to store the data on an external storage device for access, which will significantly increase the inference time of the model on the chip and ultimately reduce the utilization rate of the artificial intelligence chip.

[0005] To this end, a technical solution is needed that can implement data access operations such as data transposition and data slicing within the chip, reduce the time spent on model inference on the chip, and increase the utilization rate of artificial intelligence chips. Summary of the invention

[0006] The present invention aims to provide a method for optimizing data access in parallel computing of artificial intelligence chips, an artificial intelligence chip and a computer program product, which can realize data access operations such as data transposition and data slicing inside the chip, reduce the reasoning time of the model on the chip, and increase the utilization rate of the artificial intelligence chip.

[0007] According to one aspect of the present invention, a method for optimizing data access in parallel computing of an artificial intelligence chip is provided, comprising:

[0008] Determine the computational parallelism of AI chips;

[0009] Acquire first two-dimensional data for parallel computing of the artificial intelligence chip, wherein the first two-dimensional data satisfies the requirement that a second dimension multiplied by a first integer is the computing parallelism, and a first dimension divided by the first integer is a second integer multiple of the computing parallelism, and the first integer is greater than or equal to 1;

[0010] A preprocessing operation is performed on the first two-dimensional data inside the artificial intelligence chip, wherein the preprocessing operation includes a combination of a deformation operation and a transposition operation, so as to transpose the first two-dimensional data into data that meets the parallelism requirement.

[0011] According to some embodiments, performing a preprocessing operation on the first two-dimensional data within the artificial intelligence chip includes:

[0012] Continuously perform deformation operations and transposition operations on the first two-dimensional data, so that each transposition operation meets the parallelism requirement executed within the artificial intelligence chip.

[0013] According to some embodiments, performing a preprocessing operation on the first two-dimensional data within the artificial intelligence chip includes:

[0014] Performing a first deformation operation on the first two-dimensional data to obtain first deformed data;

[0015] performing a transposition operation on the first deformed data to obtain first transposed data;

[0016] Performing a second deformation operation on the first transposed data to obtain second deformed data;

[0017] performing a transposition operation on the second deformed data to obtain second transposed data;

[0018] A third deformation operation is performed on the second transposed data to obtain third deformed data, wherein the third deformed data is two-dimensional data, a first dimension of the third deformed data is equal to a second dimension of the first two-dimensional data, and a second dimension of the third deformed data is equal to a first dimension of the first two-dimensional data.

[0019] According to some embodiments, performing a first deformation operation on the first two-dimensional data to obtain first deformed data includes:

[0020] The first dimension of the first two-dimensional data is divided by the first integer as the first dimension of the first deformed data, and the second dimension of the first two-dimensional data is multiplied by the first integer as the second dimension of the first deformed data. The first deformed data meets the computational parallelism requirement, so that a transposition operation is performed on the first deformed data inside the artificial intelligence chip.

[0021] According to some embodiments, performing a second deformation operation on the first transposed data to obtain second deformed data includes: deforming the second dimension data of the first transposed data into second two-dimensional data, the second dimension of the second two-dimensional data is the same as the second dimension of the first two-dimensional data, the second deformed data meets the computational parallelism requirement, and performing the transposition operation on the second deformed data inside the artificial intelligence chip;

[0022] The second deformed data is transposed, including: transposing the second two-dimensional data to form transposed second two-dimensional data.

[0023] According to some embodiments, the second two-dimensional data is transposed to form transposed second two-dimensional data.

[0024] According to some embodiments, the method further comprises:

[0025] Performing a data slicing operation on the third deformed data to obtain a first data slice;

[0026] The first data slice is transposed to obtain a second data slice.

[0027] According to some embodiments, obtaining first two-dimensional data for parallel computing of the artificial intelligence chip includes:

[0028] Obtain multi-dimensional data to be processed;

[0029] The multi-dimensional data to be processed is transformed into the first two-dimensional data through a deformation operation.

[0030] According to some embodiments, the method further comprises:

[0031] The third deformed data is transformed into second multi-dimensional data corresponding to the multi-dimensional data to be processed through a deformation operation.

[0032] According to one aspect of the present invention, there is provided an artificial intelligence chip, comprising: a data preprocessing module, wherein the data preprocessing module implements the method described in any one of the above items when executed.

[0033] According to one aspect of the present invention, a computer program product is provided, comprising: a computer program, wherein when the computer program is executed by a processor, the method described in any one of the above items is implemented.

[0034] According to another aspect of the present invention, there is provided a computing device, comprising:

[0035] Processor; and

[0036] A memory storing a computer program, which, when executed by the processor, enables the processor to execute any of the methods described above.

[0037] According to another aspect of the present invention, there is provided a non-transitory computer-readable storage medium having computer-readable instructions stored thereon, and when the instructions are executed by a processor, the processor is caused to execute any of the above methods.

[0038] According to an embodiment of the present invention, by performing a series of deformation and transposition operations on the data to be processed that does not meet the parallel computing requirements, the access to the data to be processed is optimized in advance, so that it meets the parallel computing requirements and the parallel computing rate is improved. The deformation and transposition operations used in the method described in the scheme of the present invention enable the entire reasoning and calculation process to be always performed inside the chip, greatly reducing the reasoning time of the AI ​​chip and increasing the utilization rate and computing efficiency of the AI ​​chip.

[0039] According to some embodiments, by applying the method described in the solution of the present invention, it is possible to achieve accelerated calculation of access operations for data of any dimensionality. The method is simple and convenient and has a wide range of applications.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0042] Figure 1 A flow chart of a method for optimizing data access in parallel computing of an artificial intelligence chip according to an example embodiment is shown.

[0043] Figure 2 A schematic diagram of a method for optimizing data access in parallel computing of an artificial intelligence chip according to another example embodiment is shown.

[0044] Figure 3 A block diagram of a computing device is shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0045] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0046] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0047] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0048] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0049] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0050] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0051] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0052] In deep learning models, common data access operations include data transposition and data slicing. Data transposition refers to converting multidimensional data between different dimensions, while data slicing refers to extracting data in a certain dimension. Therefore, both operations involve data access operations, and as the amount of data increases, the access time will increase significantly.

[0053] At present, the core of mainstream AI chips mainly uses multiplication and addition calculation acceleration arrays to accelerate the most important convolution operations in convolutional neural networks. This data processing method usually has parallel requirements for the data to be calculated. The so-called computational parallelism refers to the smallest unit of data accessed by the hardware each time. The parallelism requirement is met only when the size of the last dimension of the data is an integer multiple of the computational parallelism. When the amount of data to be accessed meets the parallelism requirement, fast access can be achieved inside the chip. Otherwise, the data needs to be output to an external storage device for access. Therefore, when the AI ​​chip performs data access operations such as data transposition and data slicing, if the data to be accessed does not meet the parallelism requirements, the access operation cannot be performed inside it. The parallelism requirements are met only when the size of the last dimension of the data is an integer multiple of the calculation parallelism. Otherwise, the parallelism requirements are not met. For example, the parallelism is 64, but the data shape is: [1, 3, 32, 32], which means that the data has four dimensions, and the sizes of the four dimensions are 1, 3, 32, 32, that is, there are a total of 1*3*32*32 values, and the size of the last dimension of the data is 32, which is not an integer multiple of 64. At this time, the parallelism calculation requirements are not met. When the data to be accessed does not meet the parallelism requirements, these data need to be stored on external storage devices for access, which will significantly increase the inference time of the model on the chip and ultimately reduce the utilization of the AI ​​chip.

[0054] To this end, the present invention proposes a method for optimizing data access in parallel computing of artificial intelligence chips, an artificial intelligence chip, and a computer program product, which can implement data access operations such as data transposition and data slicing inside the chip, reduce the inference time of the model on the chip, and increase the utilization rate of the artificial intelligence chip. According to the embodiment, by performing a series of deformation and transposition operations on the data to be processed that does not meet the parallel computing requirements, the access of the data to be processed is optimized in advance to meet the parallel computing requirements, the parallel computing rate is improved, and the entire inference calculation process is always performed inside the chip, which greatly reduces the inference time of the AI ​​chip and increases the utilization rate and computing efficiency of the AI ​​chip.

[0055] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0056] Figure 1 A flow chart for optimizing data access in parallel computing of an artificial intelligence chip according to an example embodiment is shown.

[0057] See also Figure 1 , the figure shows a method for optimizing data access in parallel computing of artificial intelligence chips, including:

[0058] In S101, the computational parallelism of the artificial intelligence chip is determined.

[0059] According to some embodiments, determining the computational parallelism is one of the key steps to optimize computing performance. The computational parallelism can usually be determined by the architectural characteristics of the hardware and a specific parallel computing model, or by running a specialized benchmark tool to evaluate the parallel processing capabilities of the hardware.

[0060] In S103, first two-dimensional data for parallel computing of the artificial intelligence chip is obtained, wherein the first two-dimensional data meets the requirement that the second dimension multiplied by a first integer is the computing parallelism, and the first dimension divided by the first integer is a second integer multiple of the computing parallelism, and the first integer is greater than or equal to 1.

[0061] According to some embodiments, first two-dimensional data are obtained for parallel computing of an artificial intelligence chip. The first two-dimensional data need to meet certain conditions. The first two-dimensional data conforms to the condition that the second dimension multiplied by the first integer is the computing parallelism, and the first dimension divided by the first integer is a second integer multiple of the computing parallelism. The first integer is greater than or equal to 1, so that subsequent deformation and transposition operations can be carried out smoothly and the data can efficiently utilize the parallel computing capabilities of the chip.

[0062] Thereafter, a preprocessing operation may be performed on the first two-dimensional data within the artificial intelligence chip, wherein the preprocessing operation includes a combination of a deformation operation and a transposition operation to transpose the first two-dimensional data into data that meets the parallelism requirement. For example, the deformation operation and the transposition operation are performed continuously on the first two-dimensional data, so that each transposition operation meets the parallelism requirement performed within the artificial intelligence chip, as described in detail below.

[0063] In S105, a first deformation operation is performed on the first two-dimensional data to obtain first deformed data, where the first deformed data meets the computational parallelism requirement.

[0064] According to some embodiments, a first deformation operation is performed on the acquired first two-dimensional data to ensure that the deformed data can meet the requirement of computing parallelism. Specifically, it is necessary to ensure that the number of columns of the first deformed data after deformation can be divided by the parallelism, so that the data can efficiently utilize the parallel computing capability of the chip.

[0065] According to some embodiments, the first dimension of the first deformed data is the first dimension of the first two-dimensional data divided by the first integer, and the second dimension of the first deformed data is the second dimension of the first two-dimensional data multiplied by the first integer.

[0066] In S107, a transposition operation is performed on the first deformed data to obtain first transposed data.

[0067] According to some embodiments, a transposition operation is performed on the first deformed data to change the shape of the data so as to make it more suitable for subsequent parallel computing.

[0068] In S109, a second deformation operation is performed on the first transposed data to obtain second deformed data, where the second deformed data meets the computational parallelism requirement.

[0069] According to some embodiments, a second deformation operation is performed on the first transposed data to ensure that the obtained second deformed data can meet the requirements of computational parallelism. The second dimension of the first transposed data is deformed into a two-dimensional array, and then appropriate transposition and deformation operations are performed to ensure that the shape of the data meets the requirements of parallel computing.

[0070] According to some embodiments, the second deformation operation deforms the second dimension data of the first transposed data into second two-dimensional data, and the second dimension of the second two-dimensional data is the same as the second dimension of the first two-dimensional data. By performing the second deformation operation on the first transposed data, the first device data is further split and deformed, wherein the first dimension data of the first transposed data is deformed into second two-dimensional data, and the second dimension of the second two-dimensional data is the same as the second dimension of the first two-dimensional data.

[0071] In S111, a transposition operation is performed on the second deformed data to obtain second transposed data.

[0072] According to some embodiments, a transposition operation is performed on the second deformed data to change the shape of the data in preparation for the next operation.

[0073] According to some embodiments, the second two-dimensional data is transposed to form transposed second two-dimensional data. After performing a second deformation operation on the first transposed data, the deformed second two-dimensional data is transposed to obtain transposed second two-dimensional data, and the transposed second two-dimensional data is used to perform subsequent deformation operations.

[0074] In S113, a third deformation operation is performed on the second transposed data to obtain third deformed data, wherein the third deformed data is two-dimensional data, wherein a first dimension of the third deformed data is equal to a second dimension of the first two-dimensional data, and a second dimension of the third deformed data is equal to a first dimension of the first two-dimensional data.

[0075] According to some embodiments, a third deformation operation is performed on the second transposed data to restore the two-dimensional form of the data, and it is ensured that the first dimension of the third deformed data is equal to the second dimension of the first two-dimensional data, and the second dimension of the third deformed data is equal to the first dimension of the first two-dimensional data, and the transposition operation is completed inside the artificial intelligence chip.

[0076] According to some embodiments, the method further includes: performing a data slicing operation on the third deformed data to obtain a first data slice; and performing a transposition operation on the first data slice to obtain a second data slice. The method described above is applied to perform a data slicing operation inside the artificial intelligence chip, and a slicing operation is performed on the third deformed data to obtain a first data slice. Thereafter, the first data slice is transposed inside the artificial intelligence chip according to the transposition operation method described above to obtain a second data slice, and the slicing operation is completed inside the artificial intelligence chip.

[0077] According to some embodiments, obtaining the first two-dimensional data for parallel computing of the artificial intelligence chip includes: obtaining multi-dimensional data to be processed; and forming the first two-dimensional data through a deformation operation on the multi-dimensional data to be processed.

[0078] According to some embodiments, the method further includes: converting the third deformed data into second multi-dimensional data corresponding to the multi-dimensional data to be processed through a deformation operation.

[0079] According to some embodiments, when the received data is multi-dimensional data to be processed, it is necessary to convert the multi-dimensional data to be processed into two-dimensional data, and then perform a data deformation operation according to the method for optimizing data access in parallel computing of an artificial intelligence chip as described above to obtain the third deformed data, and then convert the third deformed data into second multi-dimensional data corresponding to the multi-dimensional data to be processed through a deformation operation.

[0080] According to some embodiments, the deformation operation is performed in the artificial intelligence chip by a deformation operator. Specifically, when the deformation operation is actually tested on the chip, it is implemented by a Reshape operator. In simulation experiments, the deformation operation can be implemented by Pytorch, etc. For example, the reshape function in the Python tripartite library accepts source data and target size as input and outputs data of a specified shape, for example: y = torch.reshape(x, [8, 8]), where x is 64-size data and y is output data of [8, 8] size.

[0081] Figure 2 A schematic diagram of a method for optimizing data access in parallel computing of an artificial intelligence chip according to another example embodiment is shown.

[0082] See also Figure 2 , the figure shows a schematic diagram of a method for optimizing data access in parallel computing of artificial intelligence chips, where Reshape represents a data deformation operation, which does not actually change the position of the data, but only changes the view of the data. For example, if the data of size

[64] is transformed into [8, 8], each row of data (8 consecutive data) or each column of data (8 non-consecutive data) can be uniformly modified. Transpose represents a transposition operation, which is to exchange the dimensions of the data, and the position of the data will also change. The method is used for transposing the data to be processed inside the chip, including:

[0083] In 201 , the first dimension and the second dimension of the first two-dimensional data and the required parallelism of the calculation are obtained, and a first deformation parameter is obtained according to the relationship between the first dimension and the second dimension of the first two-dimensional data and the required parallelism of the calculation.

[0084] See also Figure 2In the example, assume that the size of the data to be processed is [M, N], and the parallelism requirement is p, but the data size does not meet the parallelism requirement (that is, the last dimension N of the input data is not an integer multiple of p). For example: the input data size is [512, 16], and it needs to be transposed to [16, 512], but the chip's computing parallelism is 64. Since 16 is not an integer multiple of 64, the transposition operation cannot be completed inside the chip.

[0085] According to some embodiments, obtaining a first deformation parameter according to a relationship between a first dimension and a second dimension of the first two-dimensional data and a required parallelism of the calculation includes: assuming that the size of the first two-dimensional data to be processed is [M, N], and the required parallelism of the calculation is p,

[0086] (1)

[0087] (2)

[0088] and are all positive integers, where k is the first deformation parameter.

[0089] At 203, according to the first deformation parameter, the first two-dimensional data is deformed into first deformed data with a number of columns meeting a parallelism requirement.

[0090] According to some embodiments, transforming the first two-dimensional data into first deformed data having a number of columns meeting a parallelism requirement according to the first deformation parameter includes:

[0091] The first two-dimensional data is transformed into first deformed data whose second dimension meets the parallelism requirement through a function, wherein the number of rows of the first deformed data is the first dimension of the first two-dimensional data divided by the first deformation parameter, and the second dimension of the first deformed data is the second dimension of the first two-dimensional data multiplied by the first deformation parameter.

[0092] According to some embodiments, according to the first deformation parameter, the first two-dimensional data is transformed into the first deformed data whose number of columns meets the parallelism requirement. First, the input data of size [M, N] is first transformed into the size [M / k, k * N]. For example: the input data of size [512, 16] can be first transformed into the size [512 / 4, 4 * 16], that is, the size [128, 64]. Specifically, assuming that x is an input data of size [512, 16], it can be implemented in the simulation experiment through the reshape function, for example: y = torch.reshape(x, [512 / 4, 4 * 16]). Or when running on an artificial intelligence chip, it is implemented through the Reshape operator, which takes x and the target size [512 / 4, 4 *16] as input and outputs data of size [512 / 4, 4 * 16].

[0093] At 205, the first transformed data is transposed to obtain first transposed data, and the transformed input data of size [M / k, k* N] is transposed to size [k * N, M / k]. For example, the transformed data of size [128, 64] is transposed to data of size [64, 128]. It can be seen from formula (1) that k * N is an integer multiple of p (that is, 64 is an integer multiple of 64), and the parallelism requirement is met at this time, so this transposition operation can be performed inside the chip.

[0094] At 207 , the second dimension of the first transposed data is kept unchanged, and the first transposed data is deformed to obtain second deformed data.

[0095] According to some embodiments, keeping the second dimension of the first transposed data unchanged and deforming the first transposed data to obtain the second deformed data includes: keeping the second dimension of the first transposed data unchanged and, according to the first deformation parameter, deforming the first transposed data into a second deformed data of a three-dimensional data group. The first dimension of the three-dimensional data group is the first deformation parameter, the second dimension of the three-dimensional data group is the second dimension of the first deformed data divided by the first deformation parameter, and the third dimension of the three-dimensional data group is the second dimension of the first deformed data. Figure 2 In the example, the data of size [k * N, M / k] after transposition is transformed into the size [k, N, M / k]. For example, the data of size [64, 128] obtained after transposition is transformed into the size [4, 16, 128].

[0096] At 209, the third dimension of the three-dimensional data group of the second deformed data remains unchanged, and the three-dimensional data group is transposed to obtain second transposed data. Figure 2 In the example, the transformed data of size [k, N, M / k] is transposed to size [N, k, M / k]. For example, the transformed data of size [4, 16, 128] is transposed to size [16, 4, 128]. From formula (1) and formula (2), we can deduce:

[0097] (3)

[0098] That is, M / k is also an integer multiple of p (that is, 128 is also an integer multiple of 64). At this time, the second transposed data meets the parallelism requirement, so this transposition operation can also be performed inside the chip.

[0099] At 211 , the first dimension of the second transposed data is kept unchanged, and the second transposed data is transformed into a two-dimensional data group to obtain third transformed data.

[0100] According to some embodiments, the first dimension of the two-dimensional data set is the first dimension of the second transposed data, and the second dimension of the two-dimensional data set is the product of the second dimension and the third dimension of the first transposed data. Figure 2 In the example, the data of size [N, k, M / k] after transposition is transformed into data of size [N, M] to obtain the third transformed data. For example: the data of size [16, 4, 128] after transposition is transformed into data of size [16, 512], which is the final result.

[0101] According to some embodiments, the method described in the scheme of the present invention can not only realize accelerated access of transposition operation inside the artificial intelligence chip through the above process, but also realize accelerated access of data slicing operation (Slice) through the above process, except that the above transposition process only needs to be performed once before and after slicing.

[0102] According to some embodiments, the method is used for slicing operations on data to be processed, including: applying the method described above to perform a transposition operation on the first two-dimensional data to obtain the transposed data to be processed; performing a data slicing operation on the transposed data to be processed to obtain a first data slice; applying the method described above to perform a transposition operation on the first data slice to obtain a second data slice. For example: to slice data of size [M, N] into data of size [M, N - S], it is only necessary to first transpose the data of size [M, N] into size [N, M] through the above steps, and then slice the data of size [N, M] into size [N - S, M], and finally, transpose the data of size [N - S, M] into [M, N - S] through the above steps, so that the slicing operation can be completed inside the chip.

[0103] According to some embodiments, the method described in the scheme of the present invention can also be used for transposition and slicing operations of multidimensional data to be processed, including: transforming the multidimensional data to be processed into a two-dimensional data group; applying the method as described above to perform transposition and / or slicing operations on the two-dimensional data group to obtain a transposed and / or sliced ​​two-dimensional data group; restoring the two-dimensional data group to the multidimensional data to be processed. That is, the multidimensional data can be transformed into two-dimensional data (without changing the data position), and then the method described in the scheme of the present invention is used to access the data, and finally the data is transformed back to the multidimensional data. For example: multidimensional data x of size [512, 2, 16] needs to be transposed to size [16, 512, 2], but the parallelism is 64 at this time, that is, it does not meet the parallelism calculation requirements. At this time, x can be first transformed into two-dimensional data of [512*2, 16], and then transposed to [16, 512*2] according to the method of the present invention, and finally transformed to [16, 512, 2].

[0104] In order to verify the effectiveness of the patented method, the Caisa4 chip independently designed by Shenzhen Kunyun Information Technology Co., Ltd. was used as an example to test several common deep learning models before and after data access optimization. The Caisa 4 chip has a vector engine and a matrix engine specifically used to accelerate the calculation of AI operators. The matrix engine mainly calculates uint8 type data with a parallelism of 64, while the vector engine mainly calculates fp32 type data with a parallelism of 32. When the parallelism of the data to be processed does not meet the parallelism requirements, the data needs to be transferred to an external storage device for access, during which the matrix engine or vector engine will be idle and waiting.

[0105] In order to verify the optimization effect of this technology, the ViT-base, Swin-base, DETR, and Mask-RCNN models were used as examples for testing. The input was a single 224*224*3 image, and the single-threaded test was performed. The specific test results are shown in Table 1.

[0106] Table 1 Method optimization effect test table (unit: milliseconds)

[0107]

[0108] Among them, "time consumption after optimization" refers to the access optimization of all transposition operations and slicing operations in the original model that do not meet the parallelism requirements according to the method of the present invention, and "number of transpositions" and "number of slices" refer to the number of transposition operations and slicing operations in the original model that do not meet the chip parallelism requirements.

[0109] It can be seen from the above test results that although the number of access operations that do not meet the parallel computing requirements in different models is different, after optimization by the method of the present invention, the inference time is significantly reduced, and the more access operations that do not meet the parallel computing requirements in the model, the longer the original model takes, and the more obvious the effect of access optimization. This is because compared with the actual data access time, the data transmission time between the AI ​​chip and the external device is longer, which has a greater impact on the computing efficiency of the chip.

[0110] Therefore, the method described in the solution of the present invention effectively avoids the time-consuming transmission of data between the chip and external devices by making these access operations meet the parallel computing requirements of the chip, thereby greatly increasing the utilization rate of the chip.

[0111] According to some embodiments, the method described in the scheme of the present invention can be used for the design of an artificial intelligence chip, and a data preprocessing module is set in the artificial intelligence chip. When the data preprocessing module is executed, the method described in any of the above items is implemented, and access optimization operations are performed in advance on the data inside the artificial intelligence chip that does not meet the parallel computing requirements so that it meets the parallel computing requirements, thereby ensuring that the entire reasoning calculation process is always performed inside the chip, greatly reducing the reasoning time of the AI ​​chip, and increasing the utilization and computing efficiency of the AI ​​chip.

[0112] According to some embodiments, the method described in the scheme of the present invention can also be used for the development and design of computer program products. When the computer program is executed by a processor, the method described in any of the above items is implemented to perform access optimization operations in advance on data that does not meet the parallel computing requirements, thereby improving the overall computing efficiency and increasing the computer running speed.

[0113] According to some embodiments, the method described in the scheme of the present invention optimizes the access to the data to be processed in advance by performing a series of deformation and transposition operations on the data that does not meet the parallel computing requirements, so that it meets the parallel computing requirements and improves the parallel computing speed.

[0114] According to some embodiments, the deformation and transposition operations used in the method described in the solution of the present invention can enable the entire reasoning calculation process to be always performed inside the chip, greatly reducing the reasoning time of the AI ​​chip and increasing the utilization and computing efficiency of the AI ​​chip.

[0115] According to some embodiments, the method described in the solution of the present invention can realize accelerated calculation of access operations (such as data transposition, data slicing, etc.) for data of any dimensionality. The method is simple and convenient and has a wide range of applications.

[0116] Figure 3 A block diagram of a computing device according to an example embodiment of the invention is shown.

[0117] like Figure 3 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.

[0118] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions. According to an example embodiment, the processor 12 is an AI processor chip.

[0119] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 may read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.

[0120] Computing device 30 may also communicate with one or more networks via network interface 16 .

[0121] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.

[0122] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.

[0123] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the program is executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0124] An embodiment of the present invention also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0125] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0126] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0127] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0128] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0132] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0133] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.

Claims

1. A method for optimizing data access in parallel computing of artificial intelligence chips, characterized in that: include: Determine the computational parallelism of AI chips; Acquire first two-dimensional data for parallel computing of the artificial intelligence chip, wherein the first two-dimensional data satisfies the requirement that a second dimension multiplied by a first integer is the computing parallelism, and a first dimension divided by the first integer is a second integer multiple of the computing parallelism, and the first integer is greater than or equal to 1; Performing a preprocessing operation on the first two-dimensional data within the artificial intelligence chip, wherein the preprocessing operation includes a combination of a deformation operation and a transposition operation, transposing the first two-dimensional data into data that meets the parallelism requirement, The preprocessing operation is performed on the first two-dimensional data within the artificial intelligence chip, including: Performing a first deformation operation on the first two-dimensional data to obtain first deformed data; performing a transposition operation on the first deformed data to obtain first transposed data; Performing a second deformation operation on the first transposed data to obtain second deformed data; performing a transposition operation on the second deformed data to obtain second transposed data; A third deformation operation is performed on the second transposed data to obtain third deformed data, wherein the third deformed data is two-dimensional data, a first dimension of the third deformed data is equal to a second dimension of the first two-dimensional data, and a second dimension of the third deformed data is equal to a first dimension of the first two-dimensional data.

2. The method according to claim 1, characterized in that: Performing a first deformation operation on the first two-dimensional data to obtain first deformed data includes: The first dimension of the first two-dimensional data is divided by the first integer as the first dimension of the first deformed data, and the second dimension of the first two-dimensional data is multiplied by the first integer as the second dimension of the first deformed data. The first deformed data meets the computational parallelism requirement, so that a transposition operation is performed on the first deformed data inside the artificial intelligence chip.

3. The method according to claim 1, characterized in that: Performing a second transformation operation on the first transposed data to obtain second transformed data, including: transforming the second dimension data of the first transposed data into second two-dimensional data, the second dimension of the second two-dimensional data is the same as the second dimension of the first two-dimensional data, and the second transformed data meets the computational parallelism requirement, and performing the transposition operation on the second transformed data inside the artificial intelligence chip; The second deformed data is transposed, including: transposing the second two-dimensional data to form transposed second two-dimensional data.

4. The method according to claim 1, characterized in that Also includes: Performing a data slicing operation on the third deformed data to obtain a first data slice; The first data slice is transposed to obtain a second data slice.

5. The method according to claim 1, characterized in that Acquiring first two-dimensional data for parallel computing of the artificial intelligence chip, including: Obtain multi-dimensional data to be processed; The multi-dimensional data to be processed is transformed into the first two-dimensional data through a deformation operation.

6. The method according to claim 5, characterized in that Also includes: The third deformed data is transformed into second multi-dimensional data corresponding to the multi-dimensional data to be processed through a deformation operation.

7. An artificial intelligence chip, characterized in that: include: A data preprocessing module, which, when executed, implements the method according to any one of claims 1 to 6.

8. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images