Artificial intelligence chip, method, device and medium for flexibly accessing data

Abstract

An artificial intelligence processor chip, a method for flexibly accessing data in the artificial intelligence processor chip, an electronic device, and a non-transitory storage medium are provided. The chip includes a memory configured to store read-in tensor data from outside the processor chip, the read-in tensor data including a plurality of elements, a tensor operation of an operator included for artificial intelligence computation; a storage control unit configured to control reading of the elements from the memory to be sent to a computation unit according to the tensor operation of the operator, the storage control unit including an address calculation module having an interface to receive a parameter configured by software, the address calculation module calculating an address in the memory in a one-layer read cycle or a multi-layer read cycle nesting according to the configured parameter received through the interface to read the elements from the calculated address to be sent to the computation unit; and the computation unit configured to perform the tensor operation of the operator with the received elements.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and more particularly, to an artificial intelligence processor chip for flexibly accessing data, a method for flexibly accessing data in an artificial intelligence processor chip, an electronic device, and a non-transitory storage medium. BACKGROUND

[0002] In order to achieve better performance, fully exert computing power, and improve media access control address (MAC) utilization, the design of a data path is also very critical. In a neural network computing process, the provision and storage of input and output data of a computing unit are various, which also determines different chip storage and computing architectures. For example, a graphic processing unit (GPU) is a parallel computing processor, adopts a multi-level cache system, and the storage hierarchy is composed of an L1 cache, a shared memory, a register group, an L2 cache, and an external storage DRAM. These storage hierarchies are mainly divided in order to reduce the delay of data transfer and improve the bandwidth. The L1 cache is generally divided into an L1D cache and an L1I cache for storing data and instructions, respectively. A corresponding L1 cache is usually set for each processor core, and the size is different, being 16k-64k. The L2 cache is usually used as a private cache and does not distinguish between instructions and data. A corresponding L2 cache is usually set for each processor core, and the size is different, being 256k-1M. For example, the L1 cache has the fastest speed but a small space, the L2 cache has a slower speed but a larger space, and the external DRAM has the largest space but the slowest speed. Therefore, storing frequently accessed data from the DRAM into the L1 cache can reduce the delay of transferring data from the external DRAM to the memory each time and improve the efficiency of data processing. However, due to the structure of the processor, there is a certain redundancy in the data path in order to ensure versatility and flexibility. For example, each computing process needs to start from taking data from a register to finally storing data into a register, which has high power consumption.

[0003] Some AI chips can achieve high efficiency through customized paths, but the corresponding cost is the loss of flexibility. Once the network structure is modified, there is a risk of being unable to use. In addition, some AI chips solve the bandwidth and delay problem by increasing a large on-chip buffer, but the access number mode of a static random access memory (SRAM) is initiated by hardware, that is, the computing and storage are coupled by hardware. This will cause the problem of inflexible strategy, and thus the efficiency is reduced in some scenarios, and software cannot intervene. SUMMARY

[0004] According to an aspect of the present application, there is provided an artificial intelligence processor chip for flexibly accessing data, comprising: a memory configured to store read-in tensor data from outside the processor chip, the read-in tensor data comprising a plurality of elements, for tensor operation of an operator included in an artificial intelligence computation; a storage control unit configured to control reading of elements from the memory to send to a computation unit according to the tensor operation of the operator, the storage control unit comprising an address calculation module having an interface to receive parameters configured by software, the address calculation module calculating addresses in the memory in a one-layer read cycle or a multi-layer read cycle nesting according to the configured parameters received through the interface, to read elements from the calculated addresses to send to the computation unit; a computation unit configured to perform the tensor operation of the operator with the received elements.

[0005] In another aspect, there is provided a method for flexibly accessing data in an artificial intelligence processor chip, comprising: storing, by a memory in the artificial intelligence processor chip, read-in tensor data from outside the processor chip, the read-in tensor data comprising a plurality of elements, for tensor operation of an operator included in an artificial intelligence computation; controlling reading of elements from the memory to send to a computation unit according to the tensor operation of the operator, comprising: calculating addresses in the memory in a one-layer read cycle or a multi-layer read cycle nesting according to received parameters configured by software, to read elements from the calculated addresses to send to the computation unit in the artificial intelligence processor chip; performing, by the computation unit, the tensor operation of the operator with the received elements.

[0006] In another aspect, there is provided an electronic device, comprising: a memory for storing instructions; a processor for reading the instructions in the memory and performing the method as in various embodiments of the present application.

[0007] In another aspect, a non-transitory storage medium having stored thereon instructions,

[0008] Wherein the instructions, when read by a processor, cause the processor to perform the method as in various embodiments of the present application.

[0009] In this way, the addresses in the memory can be flexibly calculated by software-configured parameters to flexibly read the elements in the memory, without being limited to the order or address ordering of the elements stored in the memory. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

[0011] Figure 1 An example diagram of a computation graph in a neural network applied to image data processing and recognition is shown.

[0012] Figure 2 A schematic diagram of an artificial intelligence processor chip for flexible access to data according to embodiments of the present application is shown.

[0013] Figure 3 A schematic diagram of an artificial intelligence processor chip for flexible access to data according to embodiments of the present application is shown.

[0014] Figure 4 An example of 2-layer cyclic nested reading of an input tensor according to embodiments of the present application is shown.

[0015] Figure 5 A schematic diagram of calculating an address according to parameters configured by software according to embodiments of the present application is shown.

[0016] Figure 6 An example of 3-layer cyclic nested reading of an input tensor according to embodiments of the present application is shown.

[0017] Figure 7 A schematic diagram of the internal structure of an SRAM according to embodiments of the present application is shown.

[0018] Figure 8 A flowchart of a method for flexible access to data in an artificial intelligence processor chip according to embodiments of the present application is shown.

[0019] Figure 9 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present application is shown.

[0020] Figure 10 A schematic diagram of a non-transitory computer-readable storage medium according to embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0021] Reference will now be made in detail to the specific embodiments of the application, examples of which are illustrated in the accompanying drawings. While the application will be described in conjunction with the specific embodiments, it will be understood that the application is not intended to be limited to the described embodiments. On the contrary, the application is intended to cover alternatives, modifications, and equivalents, which can be included within the spirit and scope of the application as defined by the appended claims. It should be noted that the steps of the methods described herein can all be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0022] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in a proper manner according to relevant laws and regulations.

[0023] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can voluntarily choose whether to provide personal information to the software or hardware such as an electronic device, an application program, a server or a storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0024] As an optional but non-limiting implementation manner, in response to receiving an active request of a user, the manner of sending a prompt information to the user may, for example, be a pop-up window manner, in which the prompt information can be presented in a text manner. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0025] It can be understood that the above notification and obtaining of user authorization process is only illustrative and does not limit the implementation manners of the present disclosure, and other manners meeting relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0026] It can be understood that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the technical solutions should comply with the requirements of relevant laws and regulations and relevant provisions.

[0027] The recognition process of the above application scenarios can receive various application data as input tensors by a neural network, and be realized through the calculation of the neural network. The current neural network and machine learning system use tensors as the basic data structure. The core of the concept of tensor is that it is a data container, and the data it contains is almost always numerical data, so it is a container of numbers. The specific numerical values in the tensor can be application data, such as including image data, natural language data, etc.

[0028] For example, a scalar is a 0-dimensional tensor, such as 2, 3, 5. In specific applications, such as image data, 2 might represent the grayscale value of a pixel in the image data, 3 might represent the grayscale value of a pixel in the image data, and 5 might represent the grayscale value of a pixel in the image data, and so on. On the other hand, a vector is a 1-dimensional tensor, such as [0, 3, 20], and a matrix is ​​a 2-dimensional tensor, such as... Or [[2,3],[1,5]]. For example, there can also be three-dimensional tensors (e.g., a:(shape:(3,2,1)), [[[1],[2]],[[3],[4]],[[5],[6]]]), four-dimensional tensors, and so on. These tensors can be used to represent data in specific application scenarios, such as image data, natural language data, and so on. The neural network functions for these application data can include image recognition (e.g., input image data and identify what animal is in the image), natural language recognition (e.g., input user language and identify the user's intention, such as whether the user is saying to open the music player), and so on.

[0029] The identification process for the aforementioned application scenarios can be achieved by a neural network receiving various application data as input tensors and performing calculations. As explained above, the calculations of a neural network can consist of a series of tensor operations, which can be complex geometric transformations of the input data of multi-dimensional tensors. These tensor operations can be called operators, and the calculations of the neural network can be converted into a computation graph. This computation graph contains multiple operators, which can be connected by lines to represent the dependencies between the calculations of each operator.

[0030] Artificial intelligence (AI) chips are chips specifically designed for performing neural network operations, primarily to accelerate the execution of neural networks. Neural networks can be expressed using pure mathematical formulas. Based on these mathematical formulas, the neural network can be represented using a computational graph model. The computational graph is the visual representation of these mathematical formulas. The computational graph model can break down a complex operation into multiple sub-operations, each called an operator (Op).

[0031] Neural network computations generate a large amount of intermediate data. If all of this data is stored in Dynamic Random Access Memory (DRAM), overall performance will suffer due to excessive latency and insufficient bandwidth. Adding an L2 cache can alleviate this problem. Its advantages include being invisible to programming and thus not affecting programming, while also reducing latency. However, issues with L2 cache access addresses and timing can lead to a higher cache miss rate. Furthermore, it's difficult to hide data access times when locality is poor.

[0032] The data needed and generated when using on-chip SRAM of an artificial intelligence chip to store neural network computation can be actively controlled by software to flow in a direction, and the time of data transfer between SRAM and DRAM can be hidden by pre-configuration. Since the data access mode of neural network computation is relatively flexible, if the access flexibility of SRAM is not enough, then some operators will use multiple computing processes to complete.

[0033] If the data computation and data transfer are completely coupled and these operations are initiated by hardware, then the computing mode of the operator is fixed and there is no room for software adjustment.

[0034] Therefore, there is still a need for a more flexible way to access the on-chip SRAM of an artificial intelligence chip.

[0035] Figure 1 An example diagram of a computation graph in a neural network applied to image data processing and recognition is shown.

[0036] For example, a tensor carrying image data (such as chrominance values of pixels) is input into an example computation graph as shown in FIG. 1. Figure 1 The computation graph only shows some operators to facilitate the reader to read. The operation process of the computation graph is to first calculate the tensor through the Transpose operator, then one branch calculates through the Reshape operator, and the other branch calculates through the Fullyconnected operator.

[0037] Among them, it is assumed that the tensor is first input into the Transpose operator, which is a tensor operation that does not change the values in the input tensor. The role of the Transpose operator is to change the dimension (axis) arrangement order of the array. For example, if the order of the two dimensions is exchanged, it is matrix transposition. The Transpose operator can be applied to more dimensional cases. The input parameter of the Transpose operator is the dimension arrangement order of the output array, and the sequence number is counted from 0. It is assumed that the input tensor of the Transpose operator is, for example, a two-dimensional matrix [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]] or represented as For example, it is represented that the image data is a two-dimensional matrix of 4*3. And transpose([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]) represents transposing this two-dimensional matrix, that is, becoming [[1, 4, 7, 10], [2, 5, 8, 11], [3, 6, 9, 12]], or represented as a 3*4 matrix.

[0038] It can be seen that the Transpose operator changes the dimension arrangement order, i.e. changes the shape of the tensor, but does not change the numerical values in the tensor, for example, still 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12. Of course, the Transpose operator can also not change the shape of the tensor, for example, the 3*3 matrix after transposition is still a 3*3 matrix, and the numerical values in the tensor are not changed, but the arrangement order of the numerical values in the transposed matrix is different.

[0039] Then, the tensor after the Transpose operator operation described above is divided into two branches. One branch is calculated by the Reshape operator, and the other branch is calculated by the Fully connected operator.

[0040] The specific operation of the Reshape operator is to change the shape attribute of the tensor, which can arrange the m*n matrix a into an i*j matrix b. For example, the Reshape operator (Reshape(A, 2, 6), where A is the input tensor) changes the shape of the above tensor from 3*4 to 2*6. Therefore, the output tensor obtained after the Reshape operator is, for example,

[0041] It can be seen that the Reshape operator also changes the shape of the tensor, but does not change the numerical values in the tensor, for example, still 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12.

[0042] The Fully connected operator (also called Full Connection operator) can be regarded as a special convolution layer, or as the product of tensors, which is an operation of inputting the entire tensor as a feature map for feature extraction. That is, linear transformation from one feature space to another feature space, and the output tensor is the weighted sum of the input tensor. For example, the Fully connected operator is to multiply the input tensor (the output tensor of the Transpose operator) matrix x, for example That is, the first row of the transposed matrix is read by the weight matrix x, and then the second row is read by the weight matrix x, and then the third row is read by the weight matrix x, respectively. Specifically, 1 times 40 plus 4 times 50 plus 7 times 60 plus 10 times 70 is the first value of the result tensor of the Fully connected operator, 2 times 40 plus 5 times 50 plus 8 times 60 plus 11 times 70 is the second value of the result tensor of the Fully connected operator, and 3 times 40 plus 6 times 50 plus 9 times 60 plus 12 times 70 is the third value of the result tensor of the Fully connected operator.

[0043] In the prior art, if the calculation process from the Transpose operator to the Fully connected operator in the calculation graph as shown in Figure 1 is to be performed by an artificial intelligence chip, the input tensor of the Transpose operator needs to be first read into the memory of the storage unit in the chip, and the input tensor is assumed to be The storage into the memory of the storage unit in the chip is generally continuous, that is, stored as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, and then the calculation unit in the chip runs the operation of the Transpose operator (for example, 4*3 is converted to 3*4), so that it is converted to Then, the chip stores the result tensor of the conversion into the memory of the storage unit in the chip as intermediate data. At this time, the storage into the memory of the storage unit in the chip is generally continuous, that is, stored as 1, 4, 7, 10, 2, 5, 8, 11, 3, 6, 9, 12. At this point, the operation of the Transpose operator is completed.

[0044] Then, the operation of the Fully connected operator is performed. The input tensor (the output tensor of the Transpose operator) is multiplied by the weight matrix x of 4*1 size, for example First, the chip reads in the weight matrix x, for example The storage into the storage unit in the chip is generally continuous, that is, 40, 50, 60, and 70 are stored. Then, the computing unit reads one value in the output tensor of the Transpose operator from the storage unit in the storage in the order of storage, respectively, and reads the corresponding value in the weight matrix x in the order of storage, respectively, and performs multiplication and addition. Specifically, 1 times 40 plus 4 times 50 plus 7 times 60 plus 10 times 70 is the first value of the result tensor of the Fully connected operator, 2 times 40 plus 5 times 50 plus 8 times 60 plus 11 times 70 is the second value of the result tensor of the Fully connected operator, and 3 times 40 plus 6 times 50 plus 9 times 60 plus 12 times 70 is the third value of the result tensor of the Fully connected operator. Then, the computing unit of the chip stores the calculated result into the storage unit in the chip.

[0045] That is, for the calculation of the Transpose operator and the subsequent calculation of the Fully connected operator, the process of reading, calculating, storing, re-reading, re-calculating, and re-storing needs to be cooperated by each hardware unit of the chip, but the calculation efficiency of the whole process is very low, and the flexibility is also low.

[0046] The present disclosure proposes a way of an artificial intelligence processor chip for flexibly accessing data, which can replace the operation of certain operators by the flexible reading operation of accessing data of the chip by using the software configuration and related parameters of the artificial intelligence processor chip, and improve the calculation efficiency.

[0047] Figure 2 A schematic diagram of an artificial intelligence processor chip for flexibly accessing data according to an embodiment of the present application is shown.

[0048] As shown in Figure 2 An artificial intelligence processor chip 200 for flexibly accessing data includes a memory 201 configured to store read-in tensor data from outside the processor chip 200, the read-in tensor data including a plurality of elements, and a tensor operation of an operator included in artificial intelligence calculation; a storage control unit 202 configured to control reading of the elements from the memory to send to a computing unit 203 according to the tensor operation of the operator, the storage control unit 202 including an address calculation module 2021 having an interface to receive parameters configured by software, the address calculation module 2021 calculating an address in the memory in a one-layer read cycle or a multi-layer read cycle nesting according to the configured parameters to read the elements from the calculated address to send to the computing unit 203; and the computing unit 203 configured to perform the tensor operation of the operator with the received elements.

[0049] According to this embodiment, an address calculation module 2021 is provided in the storage control unit 202. This address calculation module 2021 has an interface for receiving parameters configured via software. The address calculation module 2021 can calculate the address in the memory within a single read loop or multiple nested read loops based on the configured parameters, and then read elements from the calculated addresses to send them to the calculation unit 203. This allows for flexible calculation of addresses in the memory for reading based on software-configurable parameters. In other words, the calculated address can differ from the stored order, and the new read order can be configured by the user via software parameters, rather than requiring elements to be read in the stored order as in existing technologies.

[0050] In this way, the addresses in the memory can be flexibly calculated through software-configured parameters to flexibly read elements from the memory, without being limited by the order in which these elements are stored in the memory or the order of their addresses.

[0051] Combination Figure 1 For example, although the Transpose operator changes the order of dimensions, it does not change the elements in the tensor; for instance, they remain 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12. In other words, if the addresses in memory can be calculated in one or more nested read loops according to configured parameters, and elements are read from the calculated addresses to be sent to the computation unit 203, then the user can software-configure a new read order using parameters, so that the elements 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 are read from the stored tensor as a transposed tensor by the Transpose operator.

[0052] Specifically, the input tensor assumption is... The storage into the memory of the storage unit within the chip is generally continuous, i.e. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 are stored (for example, the storage addresses are (hexadecimal) 00000001, 00000002, 00000003, 00000004, 00000005, 00000006, 00000007, 00000008, 00000009, 00000010, 00000011, 00000012). Then the user can use parameters to software-configure a new reading order, so that the address calculation module 2021 can calculate the addresses in the memory according to the configured parameters, for example, the order of the addresses in the memory calculated according to the configured parameters is respectively: 00000001, 00000004, 00000007, 00000010, 00000002, 00000005, 00000008, 00000011, 00000003, 00000006, 00000009, 00000012, i.e. the order of reading the addresses can directly replace the transposition operation of the Transpose operator.

[0053] In this way, the addresses in the memory can be flexibly calculated by the software-configured parameters for reading, that is, the flexibly calculated addresses can be different from the stored order, and the user can software-configure a new reading order by parameters, instead of having to read the elements in the stored order as in the prior art.

[0054] In one embodiment, the tensor operation of the operator is replaced by the way of reading the elements from the addresses in the memory according to the tensor operation of the Transpose operator indicated by the software-configured parameters. Figure 1 For example, the tensor operation of the Transpose operator is replaced by the way of reading the elements from the addresses in the memory according to the tensor operation of the Transpose operator indicated by the software-configured parameters.

[0055] In one embodiment, the software-configured parameters include: a value indicating the interval between the address of the first element read in the first step of each layer reading cycle and the initial address of the input tensor in the memory, a value indicating the number of steps of reading in one reading cycle, and a value indicating the step size between each step within one layer reading cycle. Note that here the step size is similar to the stride in the concept of neural networks.

[0056] In this embodiment, if there is only one reading cycle, the software-configured parameters only need to indicate the starting address, the total number of elements to be read, and the interval between the elements each time. That is, only 3 parameters are needed.

[0057] For example, assume that an operation of a Gather operator is to be performed. The operation of the Gather operator is to pick some values from an input tensor as an output tensor. As can be seen, the operation of the Gather operator is a tensor operation that does not change the values in the input tensor. At this time, the input tensor is [1, 2, 3, 4, 5, 6, 7, 8], and assume that the operation of the Gather operator is to pick [1, 2, 3, 4, 5, 6, 7, 8] as [1, 3, 5, 7].

[0058] In the prior art, to complete the operation of the Gather operator, the chip first needs to continuously store the input tensor [1, 2, 3, 4, 5, 6, 7, 8] in the memory as storage addresses (hexadecimal): for example, 00000001, 00000002, 00000003, 00000004, 00000005, 00000006, 00000007, 00000008, and then the computing unit of the chip performs the operation of the Gather operator to obtain [1, 3, 5, 7], and then stores [1, 3, 5, 7] as addresses (hexadecimal): for example, 00000009, 000000010, 00000011, 00000012. Then the chip continues to perform further operator operations on the result [1, 3, 5, 7] by the computing unit.

[0059] However, through the present embodiment, the address calculation module can directly calculate the addresses of 1, 3, 5, 7 from the input tensor [1, 2, 3, 4, 5, 6, 7, 8] through the software-configured parameters, and read out [1, 3, 5, 7] from these addresses to perform further operator operations on the result [1, 3, 5, 7] by the computing unit. Specifically, the software-configured parameters can include: a value 4 indicating that several steps of reading are performed in one reading cycle (indicating that a total of 4 readings are performed), and a value 2 indicating a step size between steps within one reading cycle (2 addresses are added after each reading to perform the next reading).

[0060] Therefore, according to these parameters, the address calculation module can calculate that the address sequence to be read is 00000001, 00000003, 00000005, 00000007 (starting from the 00000001 address, 2 addresses are added after each reading to perform the next reading, and a total of 4 readings are performed). The computing unit reads the addresses stored at 00000001, 00000003, 00000005, 00000007, i.e., 1, 3, 5, 7, correspondingly according to the calculated address sequence.

[0061] Thus, the calculation unit reads the addresses stored at 00000001, 00000003, 00000005, 00000007, i.e., 1, 3, 5, 7, according to the calculated address sequence by the software-configured parameters, which directly replaces the operation of the Gather operator and saves the time and hardware cost of calculating the operation of the Gather operator, the time and hardware cost of storing the result tensor of the operation of the Gather operator, and the time and hardware cost of reading out each element from the addresses of the result tensor of the operation of the Gather operator.

[0062] For more complex and flexible reading forms, there can be multi-layer read loop nesting. In one embodiment, the software-configured parameters can include: a value representing the number of steps of reading in each layer of read loop, a value representing the step length between each step in each layer of read loop, wherein each layer of read loop is nested in a manner from outer layer to inner layer. In one embodiment, the software-configured parameters can also include: a value representing the interval of several addresses between the address of the first element of the first step of reading in each layer of read loop and the initial address of the input tensor in the memory. In this way, the reading mode can be more flexible.

[0063] In this embodiment, there is multi-layer read loop nesting. The meaning of loop nesting is that the outer loop is executed once, and after the inner loop is executed, the outer loop is entered for the second time, and the inner loop is executed again. The above-mentioned outer loop and inner loop are an example of 2-layer loop nesting. For example, when reading a two-dimensional matrix, the outer loop can be used to control which column is read, and the inner loop can be used to control which row in the column is read. In C language, multi-layer for loop nesting statements can be used to execute multi-layer read loop nesting.

[0064] For example, in combination with the example of Figure 1 , an example of 2-layer loop nesting is taken. The input tensor is assumed to be The memory stored into the storage unit in the chip is generally stored continuously, i.e., 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 are stored (for example, the storage addresses are (hexadecimal) 00000001, 00000002, 00000003, 00000004, 00000005, 00000006, 00000007, 00000008, 00000009, 00000010, 00000011, 00000012).

[0065] To replace the operation of the Transpose operator, i.e., to read out 1, 4, 7, 10, 2, 5, 8, 11, 3, 6, 9, 12 in turn, 2 layers of read loops can be set. It is assumed that the first layer of read loops is the inner loop and the second layer of read loops is the outer loop. That is, in the present disclosure, for multi-layer loop nesting, the larger the number of layers is, the more outer loop it is, and the smaller the number of layers is, the more inner loop it is.

[0066] Then the parameters configured by software can include: a value representing the number of steps of reading in each layer of read loops (the number of steps of the second layer (outer) read loop is 3, i.e., traversing 3 columns of the tensor, and the number of steps of the first layer (inner) read loop is 4, i.e., traversing all rows in a column), a value representing the step length between steps in each layer of read loops (the step length between steps of the second layer of read loops is 1, i.e., the first step starts reading from 00000001, the second step starts reading from 00000001 plus 1 address equal to 00000002; the step length between steps of the second layer of read loops is 3, i.e., the first step starts reading from 00000001, the second step starts reading from 00000001 plus 3 addresses equal to 00000004), wherein each layer of read loops is nested from the outer layer to the inner layer.

[0067] It is assumed that the pseudo code of 2-layer read loop nesting is, for example:

[0068] For loop_1 from 1 to loop_1_cnt_max

[0069] For loop_0 from 1 to loop_0_cnt_max

[0070] Wherein loop_1 represents the second layer of read loops, and loop_0 represents the first layer of read loops. And according to the above parameter setting, loop_1_cnt_max is 3, and loop_0_cnt_max is 4.

[0071] In this way, next, the address calculation module reads from the storage addresses 00000001, 00000002, 00000003, 00000004, 00000005, 00000006, 00000007, 00000008, 00000009, 00000010, 00000011, 00000012 storing 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 according to these software-configured parameters.

[0072] Specifically, according to the parameters of the software configuration, in the second layer read cycle, assuming that the compiler calculates the initial address of reading, in this example, for example, the initial address 00000001 at which the input tensor is stored in the memory, in the second layer read cycle, reading is started from 00000001. In the first step of the second layer read cycle, all steps of the first layer read cycle are executed. That is, in the 4 steps of the first layer read cycle, 4 times of reading are performed with a step length of 4 addresses, that is, in the first step of the second layer read cycle, the address calculation module calculates that the reading addresses are 00000001, 00000004, 00000007, and 00000010, so in the first step of the second layer read cycle, the elements to be read are 1, 4, 7, and 10 stored at the addresses 00000001, 00000004, 00000007, and 00000010 respectively.

[0073] The step length between the initial address of the second step of the second layer read cycle and the initial address of the first step is 1, that is, this time reading is started from the address 00000001 plus 1, that is, the address 00000002. In the second step of the second layer read cycle, all steps of the first layer read cycle are executed. That is, in the 4 steps of the first layer read cycle, 4 times of reading are performed with a step length of 4 addresses, that is, in the second step of the second layer read cycle, the address calculation module calculates that the reading addresses are 00000002, 00000005, 00000008, and 00000011, so in the second step of the second layer read cycle, the elements to be read are 2, 5, 8, and 11 stored at the addresses 00000002, 00000005, 00000008, and 00000011 respectively.

[0074] The step length between the initial address of the third step of the second layer read cycle and the initial address of the second step is 1, that is, this time reading is started from the address 00000002 plus 1, that is, the address 00000003. In the third step of the second layer read cycle, all steps of the first layer read cycle are executed. That is, in the 4 steps of the first layer read cycle, 4 times of reading are performed with a step length of 4 addresses, that is, in the third step of the second layer read cycle, the address calculation module calculates that the reading addresses are 00000003, 00000006, 00000009, and 00000012, so in the third step of the second layer read cycle, the elements to be read are 3, 6, 9, and 12 stored at the addresses 00000003, 00000006, 00000009, and 00000012 respectively.

[0075] Thus, according to the parameters configured by the software, the order of the addresses calculated by the address calculation module is 00000001, 00000004, 00000007, 00000010, 00000002, 00000005, 00000008, 00000011, 00000003, 00000006, 00000009, 00000012, and thus the elements read from these addresses in turn are 1, 4, 7, 10, 2, 5, 8, 11, 3, 6, 9, 12.

[0076] According to the parameters configured by the software, the number of steps of the second layer read cycle is 3, and thus the address calculation module can stop after performing address calculation for three times of the second layer read cycle and the first layer read cycle within each second layer read cycle.

[0077] In this embodiment, the case of only one cycle can also be configured by parameters configured by the software, for example, the number of steps of the second layer read cycle is set to 1, that is, the entire steps of the first layer read cycle are executed only once.

[0078] In addition, the values of the above-mentioned specific parameters are only examples, that is, the number several directly represents the meaning of several, but this is not a limitation, other numbers or other contents other than numbers can be used to represent the meaning, or 0 can be used to represent, for example, the step length or the number of steps is 1 (because the chip usually starts counting from 0, etc.), or A can be used to represent, for example, the step length or the number of steps is 1, as long as the chip can deduce that the value represents the corresponding meaning.

[0079] Thus, the parameters configured by the software and the address calculation module cooperating to calculate the address can directly replace various tensor operations, and the address calculation process with more than one cycle nesting can make the read addresses calculated by the address calculation module more flexible and not limited to the order of the addresses of the values of the input tensor stored therein. And can directly replace the operation of the operator, and save the time and hardware cost of calculating the operation of the operator, the time and hardware cost of storing the result tensor of the operation of the operator, and the time and hardware cost of reading out each element from the address of the result tensor of the operation of the operator, thereby reducing the calculation delay, reducing the hardware calculation cost, and improving the running efficiency of the artificial intelligence chip.

[0080] In general, the parameters configured by the software and the address calculation module cooperating to calculate the address can not only make the calculation more flexible arrangement of address sequence, but also can replace various tensor operations. In one embodiment, the replaced tensor operation can be a tensor operation that does not change the values in the input tensor, wherein the address calculation module calculates the address in the memory to be read at each read according to the configured parameters received through the interface in one layer read cycle or multiple layer read cycle nesting, so as to read the replaced tensor operation.

[0081] In one embodiment, the tensor operation can include operations of transpose operator, reshape operator, broadcast operator, gather operator, reverse operator, concat operator, cast operator, etc. Of course, there are many other types of operator operations that can also be flexibly implemented by the address calculation module and the software configured parameters of the embodiments of the present application.

[0082] Of course, using software configured parameters and using the address calculation module to cooperate with the calculation of the address can not only be used to replace specific tensor operations, but also can calculate the address to be read in various flexible ways, thereby realizing the flexible reading function beyond the fixed limitation of the sequential reading of the hardware itself.

[0083] The following will be described in combination with a specific chip hardware example and a parameter example to describe the specific application in the actual scene.

[0084] Figure 3 A disassembled schematic diagram of an artificial intelligence processor chip for flexibly accessing data according to an embodiment of the present application is shown.

[0085] Figure 3 Units inside a process engine (PE) 300 of an artificial intelligence processor chip and used parameters are shown.

[0086] The process engine 300 can include a configuration unit 301, a calculation unit 302, and a storage unit 303. The storage control unit 304 is used to configure the calculation unit 302 and the storage unit 303; the calculation unit 302 is mainly used for convolution / matrix calculation / vector calculation, etc.; the storage unit 303 includes an on-chip SRAM memory 3031 (the size is, for example, 8MB, but this is not a limitation) and a memory access control module 3032, which is used to interact and process the internal data of the process engine 300 and the data access of the calculation unit 302 inside the process engine 300.

[0087] The process engine 300 further includes a storage control unit 304, which realizes the following specific functions:

[0088] The sram_read function is used to read data from the SRAM 3031 and send it to the calculation unit 302 for calculation on the read data, such as tensor operations of convolution / matrix calculation / vector calculation, etc.;

[0089] The sram_write function is used to obtain the calculation result data from the calculation unit 302 and write and store it into the SRAM 3031;

[0090] The sram_upload function is used to transfer data stored in the SRAM 3031 to outside of the processing engine 300 (e.g., to other processing engines or DRAM);

[0091] The sram_download function is used to transfer data outside of the processing engine 300 (e.g., from other processing engines or DRAM) to the SRAM 3031.

[0092] That is, the sram_upload function and the sram_download function are used for data interaction with devices outside of the processing engine 300. The sram_read function and the sram_write function are used for data interaction between the computing unit 302 and the storage unit 303 inside the processing engine 300.

[0093] The SRAM 3031 is a shared memory inside the processing engine 300, and the size is not limited to 8 MB, and is mainly used to store intermediate data (including data to be calculated and results of calculation) inside the processing engine 300. The SRAM 3031 can be divided into multiple banks to improve the overall data bandwidth.

[0094] The crossbar 3041 is a full interconnection structure of the memory control access interface and the SRAM multiple banks inside the processing engine 300. The crossbar 3041 controls, together with the memory access control module 3032 of the storage unit 303, reading an element from the SRAM memory 3031 in the storage unit 303 at an address calculated by the address calculation module 3042.

[0095] As can be seen, the computing path (computing unit 302) and the data path (storage unit 303) of the entire processing engine 300 are configured separately. If the calculation of an operator is to be completed, multiple modules need to be coordinated to complete the calculation. For example, for the calculation of a convolution, the sram_read is configured to input feature data to the computing unit 302, the sram_read is configured to input weights to the computing unit 302, the computing unit 302 performs matrix convolution calculation, and the sram_write is configured to output the calculation result to the SRAM memory 3031 in the storage unit 303. In this way, the selection of the calculation mode can be made more flexible.

[0096] Specifically, the storage control unit 304 is configured to control reading data from the memory 3031 to send to the computing unit according to tensor operation of the operator, the storage control unit 304 includes an address calculation module 3042, the address calculation module 3042 has an interface receiving a parameter data_noc configured by software, and the address calculation module 3042 calculates an address in the memory 3031 in a one-layer read cycle or a multi-layer read cycle nesting according to the configured parameter, to read an element from the calculated address to send to the computing unit 302.

[0097] Since most of the calculations of artificial intelligence AI are regular addressing, such as matrix multiplication, full connection, convolution, etc. The calculation is regularly read from the address of the storage tensor. Therefore, it can be considered to realize the calculation of various complex addresses by means of software configuration parameters.

[0098] In one embodiment, if only one read cycle is set, the parameters configured by software include: a value representing the number of steps of reading in one read cycle, and a value representing the step length between steps in one read cycle.

[0099] In one embodiment, if a plurality of read cycle nestings are set, the parameters configured by software include: a value representing the number of steps of reading in each layer of read cycle, and a value representing the step length between steps in each layer of read cycle, wherein each layer of read cycle is nested from the outer layer to the inner layer.

[0100] The above setting of multiple read cycle nestings can realize multiple readings of the same tensor storage address or for the same address, thereby realizing the calculation of various complex addresses to obtain the flexibility of more flexible reading of addresses.

[0101] For example, in the form of 8-layer cycle nesting (from the outer loop to the inner loop, loop_7 to Loop_0 in turn), the pseudo code is as follows:

[0102]

[0103] The following parameters related to addressing are configured in the register: a value loop_xx_cnt_max representing the number of steps of reading in each layer of read cycle (where xx represents the first read cycle), and a value jump_xx_addr representing the step length between steps in each layer of read cycle (where xx represents the first read cycle).

[0104] The 8-layer read loop is run as follows: first, the total loop_7_cnt_max steps of the Loop7 layer loop are run, in each step of the Loop7 layer loop, the total loop_6_cnt_max steps of the Loop6 layer loop are run, in each step of the Loop6 layer loop, the total loop_5_cnt_max steps of the Loop5 layer loop are run, in each step of the Loop5 layer loop, the total loop_4_cnt_max steps of the Loop4 layer loop are run, in each step of the Loop4 layer loop, the total loop_3_cnt_max steps of the Loop3 layer loop are run, in each step of the Loop3 layer loop, the total loop_2_cnt_max steps of the Loop2 layer loop are run, in each step of the Loop2 layer loop, the total loop_1_cnt_max steps of the Loop1 layer loop are run, in each step of the Loop1 layer loop, the total loop_0_cnt_max steps of the Loop0 layer loop are run. It can be seen that the total steps of the Loop0 layer loop at the bottom layer are loop_0_cnt_max*loop_1_cnt_max*loop_2_cnt_max*loop_3_cnt_max*loop_4_cnt_max*loop_5_cnt_max*loop_6_cnt_max*loop_7_cnt_max, and the total steps of the Loop1 layer loop at the next layer are loop_1_cnt_max*loop_2_cnt_max*loop_3_cnt_max*loop_4_cnt_max*loop_5_cnt_max*loop_6_cnt_max*loop_7_cnt_max. In this way, the total steps of the Loop7 layer loop at the top layer are loop_7_cnt_max.

[0105] It can be seen that, as the eight-layer nested loop set as described above, from the bottom layer to the upper layer, it is a multiplication relationship, that is, the number of times of the bottom layer to be looped is equal to the product of the number of times of all upper layers.

[0106] Taking an example of 2-layer loop nested reading of a 2-dimensional space tensor, as shown in Figure 4 Figure 4 An example of 2-layer loop nested reading of an input tensor according to an embodiment of the present application is shown.

[0107] Suppose the input tensor is Figure 4 ​The numbers in the boxes represent the addresses of the SRAMs storing the elements of the input tensor (for the sake of convenience, the addresses of the stored elements are written directly as numbers corresponding to the elements themselves). Assume that the read (access) addresses to be implemented by the software-configured parameters are Figure 4 The gray blocks in the figure are the read sequence 0-2-4-6-9-11-13-15.

[0108] Then the base_address is the starting address, which can be pre-calculated by the compiler and is usually the initial address (i.e., the position of the first element, which is address 0 in this example) of a segment of addresses in which the input tensor is stored in the SRAM. By software-configuring a parameter loop_0_cnt_max=4, it is indicated that the step number of the first layer (inner layer) read loop is 4 or the size of the loop is 4, by software-configuring another parameter jump0_addr=2, it is indicated that the step length between steps in the first layer read loop is 2, by software-configuring another parameter loop_1_cnt_max=2, it is indicated that the step number of the second layer (outer layer) read loop is 2 or the size of the loop is 2, and by software-configuring another parameter jump1_addr=9, it is indicated that the step length between steps in the second layer read loop is 9.

[0109] The pseudo code is as follows:

[0110] For loop_1 from 1 to 2

[0111] For loop_0 from 1 to 4

[0112] Each read loop sets a corresponding counter loop_xx_cnt (xx represents the number of the read loop), for example, loop_0_cnt is incremented from 1 to 4, and loop_1_cnt is incremented from 1 to 2.

[0113] According to the above-mentioned configured parameters, combined with Figure 5 ( Figure 5 The schematic diagram for calculating the address according to the software-configured parameters is shown in FIG. 1, wherein sram_addr represents the addresses 0-15 stored in the SRAM, and the address calculation module calculates the address as follows:

[0114] First, the first step (total of 2 steps) of the second layer (outer layer) read loop loop_1 is run, starting from the base_address initial address 0, and the first step of the first layer (inner layer) read loop loop_0 is run (as Figure 5In the first level (memory) read loop 0 (4 steps in total), element 0 is read from address 0. Since the parameter `jump0_addr = 2`, indicating a step size of 2 between each step within the first level read loop, the second step of the first level (memory) read loop `loop_0` is executed (e.g., ...). Figure 5 (0_1 in total 4 steps), read element 2 from address 2 (address 0+2), and run the third step of the first level (memory) read loop loop_0 (such as... Figure 5 In the 0_2 (a total of 4 steps), element 4 is read from address 4 (address 2+2), and the 4th step of the first level (memory) read loop loop_0 is executed (e.g. Figure 5 In the first layer (memory), loop 0 (4 steps in total) reads element 6 from address 6 (address 4+2). Then the first layer (memory) reads loop 0, which completes its 4 steps.

[0115] Next, the second step of the second (outer) read loop `loop_1` is executed (there are two steps in total). Since the parameter `jump1_addr = 9`, it indicates that the step size between each step in the second read loop is 9. Figure 5 The arrow in the image indicates that the first step of the first level (memory) read loop, loop_0, begins at address 9 (the address obtained by adding 9 to the initial address of base_address). Figure 5 In step 1_0 (a total of 4 steps), element 9 is read from address 9. Since the parameter jump0_addr = 2, indicating a step size of 2 between each step within the first-level read loop, the second step of the first-level (memory) read loop_0 is executed (e.g., step 2). Figure 5 (1_1, a total of 4 steps), read element 11 from address 11 (address 9+2), and run the 3rd step of the first level (memory) read loop loop_0 (as shown in step 1_1). Figure 5 (Steps 1-2 in total, 4 steps), read element 13 from address 13 (address 11+2), and run the 4th step of the first level (memory) read loop loop_0 (as shown in step 1_2). Figure 5 (Steps 1-3 in total, 4 steps), read element 15 from address 15 (address 13+2). Then the first level (memory) read loop loop_0 completes its 4 steps.

[0116] At this point, both steps of the second (outer) read loop_1 have been completed, and the address calculation and reading by the address calculation module have terminated. Thus, the order of reading elements is 0-2-4-6-9-11-13-15.

[0117] Of course, in one embodiment, the parameters configured by software may also include a value representing a gap of several addresses between the address of the first element read by a read loop and the initial address of the input tensor in memory. This also allows for flexible configuration of the initial address of each read loop.

[0118] It can be seen that by software configuring corresponding parameters of the above 2-layer read loop nesting, the element can be flexibly read from the address of the SRAM.

[0119] Similarly, a mechanism of more than 2-layer read loop nesting can be used, which is not limited here.

[0120] Suppose that eight layers of nested loops are set as described above, from an abstract point of view, from the bottom layer to the upper layer is a multiplication relationship, that is, the number of times of the bottom layer to be looped is equal to the product of the number of times of all upper layers. This is a completely aligned way, that is, each nested loop layer is regular, and the read address also presents a certain regularity.

[0121] However, for some special scenarios, there may be a situation that is not completely aligned, for example, the order of the read address has a first rule on a certain address, but on the other end of the address, it may have a second rule different from the first rule. Then there can be 2 different ways of loop nesting. In this case, the parameters configured by software can include conditions for the parameters, and the parameters take different values when the conditions are met and not met.

[0122] In one embodiment, the parameter is a value representing the number of steps of reading in a certain layer of read loop, and the condition is that it is in another layer of read loop outside the certain layer. For example, a configuration can be added in the specified read loop, which is bound to another read loop to solve the misalignment. For example, loop_1_cnt_max0 and loop_1_cnt_max1 are selected for Loop1 read loop, which are different step numbers of configuration for Loop5 read loop.

[0123]

[0124] That is, the parameters configured by software can include conditions for the parameters (the value of the number of steps of reading in the Loop1 layer read loop, that is, loop_1_cnt_max) (loop_5_cnt==loop_5_cnt_max), and the parameters loop_1_cnt_max take different values loop_1_cnt_max_1, loop_1_cnt_max_0 when the conditions are met (in the Loop5 layer read loop outside the Loop1 layer read loop, the last step is performed, that is. loop_5_cnt==loop_5_cnt_max) and not met (loop_5_cnt<>loop_5_cnt_max).

[0125] That is, when running to Loop5, Loop1 is run at least once in each step of Loop5, and if Loop5 does not run to the last step, i.e. loop_5_cnt< loop_5_cnt_max, the number of steps of Loop1 run in the step of Loop5 is loop_1_cnt_max_0. If Loop5 runs to the last step, i.e. loop_5_cnt== loop_5_cnt_max, the number of steps of Loop1 run in the step of Loop5 is loop_1_cnt_max_1.

[0126] In this way, the way of calculating the address for reading can be more flexible.

[0127] Taking an example of 3-layer loop nested reading of a tensor in 2-dimensional space in the case of incomplete alignment as above, as shown in Figure 6 , it can be seen that the reading order is 0-8-1-9-2-10-3-11-4-12-16-17-18-19-20. Figure 6 An example of 3-layer loop nested reading of an input tensor in the case of incomplete alignment according to an embodiment of the present application is shown.

[0128] Suppose the input tensor is The address stored in the SRAM is as shown in Figure 6 . Figure 6 The numbers in the block of Figure 6 represent the address of the SRAM storing each element of the input tensor (for the sake of convenience, the address of the stored element is directly written as a number corresponding to the element itself). Suppose that the reading (access) address to be achieved by the software-configured parameters is the gray block in , and the reading order is 0-8-1-9-2-10-3-11-4-12-16-17-18-19-20.

[0129] It can be seen that the reading 0-8-1-9-2-10-3-11-4-12 is a completely aligned reading mode with a rule, while the reading 16-17-18-19-20 is not completely aligned with the reading mode of the reading 0-8-1-9-2-10-3-11-4-12 and has another rule. At this time, consider using software-configured parameters to achieve incomplete alignment of loop nested reading.

[0130]

[0131] The software configured parameters can be: the step number loop_2_cnt_max of the highest layer Loop2 is 2, the step length jump2_addr is 16, the step number loop_1_cnt_max of the next layer Loop1 is 5, the step length jump1_addr is 1, the step number loop_0_cnt_max_0 of the bottom layer Loop0 is 2, the step length jump0_addr is 8, then the configuration specifies that Loop2 is bound with Loop0, and the condition is that when Loop2 goes to the last step (loop_2_cnt==loop_2_cnt_max), the step number of Loop0 changes from loop_0_cnt_max_0, i.e., 2 to loop_0_cnt_max_1, which is 1.

[0132] The step number loop_2_cnt_max of the highest layer Loop2 is set to 2, which is considering that the order of reading 0-8-1-9-2-10-3-11-4-12 is executed in the first step, and the order of reading 16-17-18-19-20 is executed in the second step. Since the orders and rules of the two steps are different, how the lower layer reading loops Loop1 and Loop0 cooperate to realize different reading orders in the second step needs to be considered.

[0133] Specifically, according to the software configured parameters described above, the three-layer loop nesting is run, and the calculated addresses are as follows:

[0134] The first step (total 2 steps) of the highest layer Loop2 is executed first, and in the first step, 5 steps of Loop1 are run.

[0135] In the first step of Loop1, all steps of Loop0 are run, i.e., 2 steps starting from 0 are run, and the step length is 8, so 0 is read first, then 8 addresses are added to read 8, and so on for 2 steps, 0-8 is read.

[0136] In the second step of Loop1, the step length is 1, i.e., all steps of Loop0 starting from 1 are run, i.e., 2 steps starting from 1 are run, and the step length is 8, so 1 is read first, then 8 addresses are added to read 9, and so on for 2 steps, 1-9 is read.

[0137] In the third step of Loop1, the step length is 1, i.e., all steps of Loop0 starting from 2 are run, i.e., 2 steps starting from 2 are run, and the step length is 8, so 2 is read first, then 8 addresses are added to read 10, and so on for 2 steps, 2-10 is read.

[0138] In step 4 of Loopl, the step size is 1, i.e. all steps of LoopO are run starting from 3, i.e. 2 steps starting from 3 with a step size of 8, thus, first 3 is read, then 8 addresses are added to read 11, and so 2 steps, 3-11 are read.

[0139] In step 5 of Loopl, the step size is 1, i.e. all steps of LoopO are run starting from 4, i.e. 2 steps starting from 4 with a step size of 8, thus, first 4 is read, then 8 addresses are added to read 12, and so 2 steps, 4-12 are read.

[0140] Then, step 2 of the highest Loop2 is executed (total 2 steps), starting from the initial address 9 plus the step size 16, i.e. starting from 16 is read.

[0141] At this point, the condition loop_2_cnt == loop_2_cnt_max is fulfilled. Thus, the number of steps of LoopO is loop_0_cnt_max_1, i.e. not 2 steps, but 1 step. In this 2nd step, 5 steps of Loopl are run, in each step of Loopl, the LoopO read loop with 1 step is run.

[0142] In particular, in step 1 of Loopl, all steps of LoopO are run, i.e. 1 step starting from 16 is run, which means that only once is read, then the step size 8 is not used, thus, only 16 is read.

[0143] In step 2 of Loopl, the step size is 1, i.e. 1 step of LoopO is run starting from 16+1 = 17, thus, 17 is read.

[0144] In step 3 of Loopl, the step size is 1, i.e. 1 step of LoopO is run starting from 17+1 = 18, thus, 18 is read.

[0145] In step 4 of Loopl, the step size is 1, i.e. 1 step of LoopO is run starting from 18+1 = 19, thus, 19 is read.

[0146] In step 5 of Loopl, the step size is 1, i.e. 1 step of LoopO is run starting from 19+1 = 20, thus, 20 is read.

[0147] Thus, by the parameters configured by the software, a complex address read sequence is achieved by the 3-level read loop nesting, 0-8-1-9-2-10-3-11-4-12-16-17-18-19-20.

[0148] Of course, the above only illustrates that the condition is set to be in the next step in the read cycle of another layer which is outside of a specific layer, and then in the case of whether the condition is met, the number of steps in the read cycle of the specific layer is set respectively, but the present application is not limited to this, other conditions and changes of other parameters that meet the conditions can be considered to more flexibly realize more complex address reading sequences.

[0149] Therefore, according to the parameters of the software configuration of the present application, the address in the memory can be flexibly calculated to read the element from the calculated address to send to the computing unit, to realize a flexible address reading mode, increase the computing efficiency and cost of the artificial intelligence processor chip, and in some cases, the specific tensor operation in artificial intelligence calculation can also be replaced, thereby simplifying the operation of the operator.

[0150] For specific address calculation of read cycle nesting, in one embodiment, according to the current step of one or each layer read cycle and the step length of each layer read cycle, the address to be read currently is calculated.

[0151] Specifically, for the address calculation of which address in the SRAM to be read finally, according to the above-mentioned parameters of the software configuration, the address calculation unit can calculate the address Address of each read by a method similar to determining the position of a point in a multi-dimensional (read cycle) space coordinate system:

[0152] Address = base_address + offset_address_dim;

[0153] The base_address can be pre-calculated by the compiler, which is usually the initial address (i.e., the position of the first element) in a segment of addresses in which the input tensor is stored in the SRAM, and the offset_address_dim is the sum of the address offsets of each dimension (read cycle):

[0154] offset_address_dim = offset_addr_0 + offset_addr_1 + offset_addr_2 + offset_addr_3 + offset_addr_4 + offset_addr_5 + offset_addr_6 + offset_addr_7

[0155] Wherein, offset_addr_xx represents the offset of the initial address of the xx-th layer read loop relative to the initial address of the last layer read loop (or relative to base_address in the case of the highest layer read loop). That is, offset_address_xx of each dimension = (loop_xx_counter-1)*jump_xx_addr, wherein loop_xx_counter represents the step of the current xx-th layer read loop.

[0156] That is, when the actual address calculation module calculates the address, it only needs to know the step of the one or each layer read loop and the step of the one or each layer read loop, and the address to be read at present can be obtained.

[0157] Figure 7 A schematic diagram of the internal structure of the SRAM according to an embodiment of the present application is shown.

[0158] In addition, the SRAM can also be divided into multiple banks. In order to make the speed of data writing and reading faster, the data written from the outside can be directly placed into different banks, and the read data, intermediate calculation data and result data can also be directly placed into different banks. The address addressing mode in the SRAM is also configurable. By default, the highest bit of the address can be used to distinguish different banks, or other granularity interleaving can be performed through a configurable address hash mode. From the hardware design, a multi-bit bank selection signal bank_sel will be finally generated, which is used for data selection of different SRAM banks (sram_bank0-sram_bank3, etc.) of each port port0, port1, port2, port3, etc.

[0159] The multi-port access can adopt a handshake signal, and the data path supports back pressure (when the inlet flow is greater than the outlet flow, back pressure is needed at this time, or when the next stage is not ready, if the current stage performs data transmission, it needs to back pressure the previous stage, so the previous stage needs to keep the data unchanged until the handshake is successful to update the data). The storage control unit 304 includes a full cross interconnection path structure with separated read and write functions of multiple banks of SRAM that can be accessed in parallel, which is equivalent to a two-stage full cross interconnection path cascade mode to alleviate the wiring problem of hardware implementation. The bottom layer memory uses a single-port SRAM to save power consumption and area. The full cross interconnection path structure can access multiple banks storing read data, intermediate result data and final result data in parallel or simultaneously, thereby further accelerating the reading and writing speed and improving the running efficiency of the artificial intelligence chip.

[0160] Thus, the parameters configured by the software and the address calculation module cooperating to calculate the address can directly replace various tensor operations, and the address calculation process of more than one loop nesting can make the read address calculated by the address calculation module more flexible and not limited to the order of the addresses of the values of the input tensor stored therein. Moreover, the operator operation can be directly replaced, and the time and hardware cost of calculating the operator operation, the time and hardware cost of storing the result tensor of the operator operation, and the time and hardware cost of reading out each element from the address of the result tensor of the operator operation are saved, thereby reducing the calculation delay, reducing the hardware calculation cost, and improving the operation efficiency of the artificial intelligence chip.

[0161] In summary, by the separate configuration of the data path and the calculation path, complete controllability of the software is achieved, and flexibility is maximized. By the multi-layer read cycle addressing mode, various complex address access patterns can be achieved. By the multi-layer asymmetric loop mode, non-aligned configuration can be achieved to achieve more complex address access patterns. By splitting the on-chip shared memory into multiple channels and maintaining the addressing mode between the channels by software, the flexibility of data access is ensured. At the same time, the data transfer and the calculation path can be hidden from each other to achieve concurrent effect of different modules. By reasonable intermediate data division of the compiler, different channels of the SRAM store different types of data, for example, different types of data that can be accessed simultaneously, so that when these data need to be accessed simultaneously, they can be read from different channels of the SRAM in parallel to speed up the efficiency. After starting the calculation, the utilization rate of the convolution calculation medium access control address (MAC) can reach almost 100%.

[0162] Figure 8 A flowchart of a method for flexibly accessing data in an artificial intelligence processor chip according to an embodiment of the present application is shown.

[0163] As Figure 8 shown, the method 800 for flexibly accessing data in an artificial intelligence processor chip includes: step 801, storing read-in tensor data from outside the processor chip in a memory in the artificial intelligence processor chip, the read-in tensor data including a plurality of elements, for performing tensor operations of an operator included in an artificial intelligence calculation; step 802, controlling reading of the elements from the memory to send to a calculation unit according to the tensor operations of the operator, including: calculating an address in the memory in a one-layer read cycle or a multi-layer read cycle nesting according to the received parameters configured by the software, to read the elements from the calculated address to send to the calculation unit in the artificial intelligence processor chip; and step 803, performing the tensor operations of the operator by the calculation unit using the received elements.

[0164] In this way, the address in the memory can be flexibly calculated by the software configured parameters to flexibly read the elements in the memory without being limited to the order or address ordering of the elements stored in the memory.

[0165] In one embodiment, the software configured parameters include a value representing how many elements are read in a read cycle of a layer, and a value representing a step size between steps in the read cycle of the layer.

[0166] In one embodiment, the software configured parameters include a value representing how many steps are read in the read cycles of layers, and a value representing a step size between steps in the read cycles of the layers, wherein the read cycles of the layers are nested from an outer layer to an inner layer.

[0167] In one embodiment, the software configured parameters include a value representing how many addresses are spaced between the address of the first element read in a read cycle of a layer and the initial address of the input tensor in the memory.

[0168] In one embodiment, the software configured parameters include a condition for the parameter, and wherein the parameter takes different values when the condition is met and when the condition is not met.

[0169] In one embodiment, the parameter is a value representing how many steps are read in a particular read cycle of a layer, and the condition is that the step is in another read cycle of a layer that is further out than the particular layer.

[0170] In one embodiment, the method 800 further includes calculating the address to be read according to the current step of the read cycle of the layer or the layers and the step size of the read cycle of the layer or the layers.

[0171] In this way, the way of calculating the address to read can be made more flexible.

[0172] In one embodiment, the software configured parameters indicate a way of reading elements from the address in the memory according to the tensor operation of the operator to replace the tensor operation of the operator.

[0173] In one embodiment, the tensor operation is a tensor operation that does not change the values in the input tensor, wherein the address in the memory to be read at each read is calculated according to the configured parameters received through the interface in a read cycle of a layer or a multi-layer read cycle nesting, so that the read replaces the tensor operation.

[0174] In one embodiment, the tensor operation includes at least one of an operation of a transpose operator, an operation of a reshape operator, an operation of a broadcast operator, an operation of a gather operator, an operation of a reverse operator, an operation of a concat operator, an operation of a cast operator.

[0175] In this way, by using the parameters configured by the software to make the computing unit read the address stored at the calculated address in correspondence with the calculated address sequence, some tensor operator operations can be directly replaced, and the time and hardware cost for calculating these tensor operator operations, the time and hardware cost for storing the result tensors of these tensor operator operations, and the time and hardware cost for reading out each element from the address of the result tensor of these tensor operator operations can be saved.

[0176] In one embodiment, the memory is divided into multiple channels for storing data that can be accessed in parallel, and the method further includes a full cross interconnection path separated by the reading function and the writing function to access the data stored in the multiple channels of the memory in parallel.

[0177] In this way, using the parameters configured by the software and calculating the address can directly replace various tensor operations, and setting more than one address calculation process of nested loops can make the calculated read address more flexible and not limited to the order of the address of the input tensor itself. The operator operation can be directly replaced, and the time and hardware cost for calculating the operator operation, the time and hardware cost for storing the result tensor of the operator operation, and the time and hardware cost for reading out each element from the address of the result tensor of the operator operation can be saved, thereby reducing the calculation delay, reducing the hardware calculation cost, and improving the running efficiency of the artificial intelligence chip.

[0178] Figure 9 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present application is shown.

[0179] The electronic device can include a processor (H1); a storage medium (H2) coupled to the processor (H1) and storing computer executable instructions therein for performing the steps of the various methods of the embodiments of the present application when executed by the processor.

[0180] The processor (H1) can include, but is not limited to, for example, one or more processors or microprocessors, etc.

[0181] The storage medium (H2) can include, but is not limited to, for example, a random access memory (RAM), a read only memory (ROM), a flash memory, an EPROM memory, an EEPROM memory, a register, a computer storage medium (for example, a hard disk, a floppy disk, a solid state disk, a removable disk, a CD-ROM, a DVD-ROM, a Blu-ray disk, etc.).

[0182] In addition, the electronic device can further include a data bus (H3), an input / output (I / O) bus (H4), a display (H5), and an input / output device (H6) (for example, a keyboard, a mouse, a speaker, etc.), etc.

[0183] The processor (H1) can communicate with external devices (H5, H6, etc.) via a wired or wireless network (not shown) through the I / O bus (H4).

[0184] The storage medium (H2) can further store at least one computer-executable instruction for performing steps of various functions and / or methods in the embodiments described in the present technology when executed by the processor (H1).

[0185] In one embodiment, the at least one computer-executable instruction can also be compiled or composed as a software product in which one or more computer-executable instructions are executed by the processor to perform steps of various functions and / or methods in the embodiments described in the present technology.

[0186] Figure 10 A schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the disclosure is shown.

[0187] As Figure 10 shown, the computer-readable storage medium 1020 stores instructions, for example, computer-readable instructions 1010. When the computer-readable instructions 1010 are executed by a processor, various methods described above can be performed. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory can include, for example, a random access memory (RAM), a cache, etc. The non-volatile memory can include, for example, a read only memory (ROM), a hard disk, a flash memory, etc. For example, the computer-readable storage medium 1020 can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions 1010 stored on the computer-readable storage medium 1020, various methods described above can be performed.

[0188] The present application provides the following items:

[0189] Item 1. An artificial intelligence processor chip for flexibly accessing data, comprising:

[0190] a memory configured to store read-in tensor data from outside the processor chip, the read-in tensor data including a plurality of elements, for tensor operation of an operator included in an artificial intelligence computation;

[0191] a memory control unit configured to control reading of elements from the memory to send to a computation unit according to the tensor operation of the operator, the memory control unit including an address calculation module having an interface to receive a parameter configured by software, the address calculation module calculating an address in the memory in a one-level read loop or a multi-level read loop nesting according to the configured parameter received through the interface, to read elements from the calculated address to send to the computation unit;

[0192] a computation unit configured to perform the tensor operation of the operator with the received elements.

[0193] Item 2. The processor chip according to item 1, wherein,

[0194] the parameter configured by software includes a value indicating how many elements are read in a one-level read loop, a value indicating a step size to be taken between steps within the one-level read loop,

[0195] or the parameter configured by software includes a value indicating how many steps are taken in each level read loop, a value indicating a step size to be taken between steps within each level read loop, wherein each level read loop is taken in a nested manner from an outer level to an inner level.

[0196] Item 3. The processor chip according to item 1, wherein the parameter configured by software includes a value indicating how many addresses are spaced between an initial address of the input tensor in the memory and an address of a first element read in a one-level read loop.

[0197] Item 4. The processor chip according to item 1, wherein the parameter configured by software includes a condition for the parameter, and wherein the parameter takes different values in the case of meeting the condition and not meeting the condition.

[0198] Item 5. The processor chip according to item 4, wherein the parameter is a value indicating how many steps are taken in a particular one-level read loop, and the condition is how many steps are taken in another one-level read loop that is more outer than the particular one-level.

[0199] Item 6. The processor chip according to any one of items 2-5, wherein the address calculation module calculates the address to be read currently according to how many steps a one-level or each level read loop is currently taken and a step size of the one-level or each level read loop respectively.

[0200] Item 7. The processor chip according to item 1, wherein the parameter configured by software indicates a way of reading elements from addresses in the memory according to a tensor operation of an operator to replace the tensor operation of the operator.

[0201] Item 8. The processor chip according to item 7, wherein the tensor operation is a tensor operation that does not change values in an input tensor, wherein the address calculation module calculates addresses in the memory to read at each reading according to a configured parameter received through the interface in a one-level read loop or a multi-level read loop nesting, so that the reading replaces the tensor operation.

[0202] Item 9. The processor chip according to item 8, wherein the tensor operation includes at least one of an operation of a transpose operator, an operation of a reshape operator, an operation of a broadcast operator, an operation of a gather operator, an operation of a reverse operator, an operation of a concat operator, an operation of a cast operator, wherein the memory is divided into multiple channels for respectively storing data that can be accessed in parallel, and the memory control unit includes a full crossbar interconnect path with separate read and write functions to access data stored in multiple channels of the memory in parallel.

[0203] Item 10. A method for flexibly accessing data in an artificial intelligence processor chip, comprising:

[0204] storing, by a memory in the artificial intelligence processor chip, read-in tensor data from outside the processor chip, the read-in tensor data including multiple elements for a tensor operation of an operator included in an artificial intelligence computation;

[0205] controlling reading of elements from the memory according to the tensor operation of the operator to send to a computation unit, including calculating addresses in the memory according to a received parameter configured by software in a one-level read loop or a multi-level read loop nesting to read elements from the calculated addresses to send to the computation unit in the artificial intelligence processor chip;

[0206] performing, by the computation unit, the tensor operation of the operator with the received elements.

[0207] Item 11. The method according to item 10, wherein,

[0208] the parameter configured by software includes a value representing how many elements of the read-in tensor data are read in a one-level read loop, a value representing a step size between steps within the one-level read loop,

[0209] Alternatively, the parameters configured by software include a value representing the number of steps of reading in each tiered read cycle, and a value representing the step size between steps in each tiered read cycle, wherein the tiered read cycles are performed in a nested manner from outer tier to inner tier.

[0210] Item 12. The method of item 10, wherein the parameters configured by software include a value representing the number of addresses spaced between the address of the first element read in a tiered read cycle and the initial address of the input tensor in memory.

[0211] Item 13. The method of item 10, wherein the parameters configured by software include a condition for the parameters, and wherein the parameters take different values in the case of meeting the condition and not meeting the condition.

[0212] Item 14. The method of item 13, wherein the parameters are a value representing the number of steps of reading in a particular tiered read cycle, and the condition is the number of steps of reading in another tiered read cycle which is further out than the particular tier.

[0213] Item 15. The method of any of items 11-14, further comprising calculating the address to be read at present according to the number of steps of reading in a tiered read cycle or each tiered read cycle at present and the step size of each tiered read cycle.

[0214] Item 16. The method of item 10, wherein the parameters configured by software indicate a way of reading elements from addresses in memory according to a tensor operation of an operator to replace the tensor operation of the operator.

[0215] Item 17. The method of item 16, wherein the tensor operation is a tensor operation which does not change the numerical values in an input tensor, wherein the address in memory to be read at each reading is calculated according to the configured parameters received through the interface in a tiered read cycle or a nested multi-tiered read cycle, so that the reading replaces the tensor operation.

[0216] Item 18. The method of item 17, wherein the tensor operation includes at least one of the following: an operation of a transpose operator, an operation of a reshape operator, an operation of a broadcast operator, an operation of a gather operator, an operation of a reverse operator, an operation of a concat operator, an operation of a cast operator, wherein the memory is divided into multiple channels for storing data which can be accessed in parallel respectively, and the method further comprises a full crossbar interconnect path separated by a reading function and a writing function, and accessing data stored in multiple channels of the memory in parallel.

[0217] Item 19. An electronic device, comprising:

[0218] a memory for storing instructions;

[0219] a processor for reading instructions in the memory and performing the method of any one of items 10-18.

[0220] Item 20. A non-transitory storage medium having stored thereon instructions,

[0221] wherein the instructions, when read by a processor, cause the processor to perform the method of any one of items 10-18.

[0222] Of course, the above-mentioned specific embodiments are only examples and are not limiting, and those skilled in the art can combine and combine some steps and devices from the above separately described various embodiments according to the concept of the present application to achieve the effect of the present application, and such combined and combined embodiments are also included in the present application, and such combined and combined embodiments are not described here.

[0223] Note that the advantages, advantages, effects, etc. mentioned in the present disclosure are only examples and are not limiting, and these advantages, advantages, effects, etc. cannot be considered as the various embodiments of the present application must have. In addition, the above-mentioned specific details are only for the purpose of example and for the purpose of understanding, and are not limiting, and the above-mentioned details do not limit the present application to be necessarily implemented with the above-mentioned specific details.

[0224] The block diagrams of the devices, apparatuses, equipment, systems involved in the present disclosure are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0225] The flowchart illustrations and above method descriptions in the present disclosure are only examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. As will be appreciated by one of ordinary skill in the art, the order of steps in the foregoing embodiments can be performed in any order. Words such as "thereafter," "then," "next," etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Further, any reference to claim elements in the singular, for example, using the articles "one," "a" or "an," is not used to limit the element to the singular.

[0226] In addition, the steps and apparatuses in various embodiments herein are not limited to be performed only in certain embodiments, in fact, new embodiments can be conceived by combining relevant partial steps and partial apparatuses in various embodiments herein according to the concepts of the present application, and these new embodiments are also included in the scope of the present application.

[0227] The various operations of methods described above can be performed by any suitable means caused by any suitable apparatuses or modules. The means can include any suitable hardware and / or software coupled to and / or configured to work in conjunction with the apparatuses or modules. The apparatuses or modules can include, but are not limited to, circuitry, dedicated hardware, an application specific integrated circuit (ASIC) or processor.

[0228] The various illustrative logical blocks, modules, and circuits described in connection with the present disclosure can be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an ASIC, a Field Programmable Gate Array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the alternative, the processor can be any commercially available processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0229] The steps of a method or algorithm described in connection with the present disclosure can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in any form of storage medium that is readable by a machine. Some examples of storage media that can be used include random access memory (RAM), read only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM and so forth. The storage media can be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The software module can comprise single instruction or many instructions, and can be distributed over several different code segments, among different programs, and across multiple storage media.

[0230] The methods disclosed herein include actions for implementing the described methods. The methods and / or actions can be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions can be modified without departing from the scope of the claims.

[0231] The functions described above can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the functions can be stored as instructions on a tangible computer-readable medium, when executed, to cause a processor to carry out actions. Storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0232] Computer program products can therefore be provided to perform the operations presented herein. For example, such computer program products can be tangibly embodied in a computer-readable tangible medium having instructions that can be executed by a processor to perform the operations described herein. The computer program products can also be provided as a package having the package and materials.

[0233] Software or instructions can also be transmitted over a transmission medium. For example, software can be transmitted from a website, server, or other remote source using a transmission medium such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. Thus, a medium that is tangible, such as a physical storage medium, can also be referred to as a computer-readable medium.

[0234] Also, the modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by a user terminal and / or base station as applicable. For example, such a device can be coupled to a server to facilitate the transfer of means for performing the methods described herein. Alternatively, various methods described herein can be provided via a storage means (e.g., RAM, ROM, a physical storage medium such as a CD or floppy disk, etc.), such that a user terminal and / or base station can obtain the various methods upon coupling or providing the storage means to the device.

[0235] Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or a combination of any of these. Features implementing functions can also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations. Also, as used herein, including in the claims, "or" as used in a list of items prefaced by "at least one of indicates a disjunctive list such that, for example, a list of "at least one of A, B, or C" means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Furthermore, as used herein, the phrase "exemplary" does not mean that example embodiments described are preferred or better than other embodiments.

[0236] Various changes, modifications, and alterations in the techniques described herein can be made without departing from the teachings of the attached claims. Moreover, the scope of the claims of the present disclosure is not limited to the specific aspects described above. Rather, the scope of the claims of the present disclosure includes all alternatives, modifications, and alterations that can be made to the above-described aspects that would be apparent to one of ordinary skill in the art having the benefit of this disclosure. Accordingly, the appended claims include within their scope all such alternatives, modifications, and alterations.

[0237] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0238] The above description has been presented for the purpose of illustration and description. Furthermore, the description is not intended to limit the embodiments of the application to the forms disclosed herein. Although several example aspects and embodiments have been discussed, those skilled in the art will recognize that certain modifications, substitutions, changes, additions and sub-combinations are possible.

Claims

1. An artificial intelligence processor chip for flexible data access, comprising: The memory is configured to store tensor data read from outside the processor chip, the read tensor data including multiple elements for tensor operations of operators included in artificial intelligence computing; A storage control unit is configured to control the reading of elements from the memory to be sent to a computing unit based on tensor operations of the operator. The storage control unit includes an address calculation module having an interface for receiving parameters configured by software. The address calculation module calculates an address in the memory in one or more nested read loops based on the configured parameters received through the interface, so as to read an element from the calculated address to be sent to the computing unit. The computation unit is configured to perform tensor operations on the operator using the received elements. The parameter instruction configured by the software indicates that the tensor operation of the operator is replaced by reading elements from the address in the memory according to the type of tensor operation of the operator.

2. The processor chip according to claim 1, wherein, The parameters configured via software include: values ​​indicating how many elements to read from the input tensor data in a single read loop, and values ​​indicating the step size between each step within the single read loop. Alternatively, the parameters configured by the software include: a value indicating how many steps to read in each layer of the read loop, and a value indicating the step size between each step in each layer of the read loop, wherein each layer of the read loop is performed in a nested manner from the outer layer to the inner layer.

3. The processor chip according to claim 1, wherein, The parameters configured via software include: a value representing the number of addresses between the address of the first element read in a single read loop and the initial address of the read tensor data in memory.

4. The processor chip according to claim 1, wherein, The parameters configured via software include conditions for the parameters, wherein the parameters take different values ​​when the conditions are met and when the conditions are not met.

5. The processor chip according to claim 4, wherein, The parameter represents the number of read steps performed in a specific layer of read loop, and the condition is the number of steps performed in another read loop at an outer layer than the specific layer.

6. The processor chip according to any one of claims 2-5, wherein, The address calculation module calculates the address to be read based on the current step of the read loop of one or more layers and the step size of each read loop.

7. The processor chip according to claim 1, wherein, The tensor operation is a tensor operation that does not change the values ​​in the input tensor, wherein the address calculation module calculates the address in the memory to be read each time in a single read loop or multiple nested read loops based on the configured parameters received through the interface, so that the read replaces the tensor operation.

8. The processor chip according to claim 7, wherein, The tensor operations include at least one of the following: transpose operator operations, reshape operator operations, broadcast operator operations, gather operator operations, reverse operator operations, concat operator operations, and cast operator operations. The memory is divided into multiple channels for storing data that can be accessed in parallel. The memory control unit includes fully cross-connected paths with separate read and write functions to access the data stored in the multiple channels of the memory in parallel.

9. A method for flexibly accessing data in an artificial intelligence processor chip, comprising: The memory in the artificial intelligence processor chip stores tensor data read from outside the processor chip. The read tensor data includes multiple elements and is used for tensor operations of operators included in artificial intelligence calculations. Controlling the reading of elements from the memory to be sent to the computing unit based on the tensor operation of the operator includes: calculating the address in the memory in one layer of read loop or multiple nested read loops according to the received parameters configured by software, so as to read the element from the calculated address and send it to the computing unit in the artificial intelligence processor chip. The computing unit uses the received elements to perform tensor operations on the operator. The parameter instruction configured by the software indicates that the tensor operation of the operator is replaced by reading elements from the address in the memory according to the type of tensor operation of the operator.

10. The method according to claim 9, wherein, The parameters configured via software include: values ​​indicating how many elements to read from the input tensor data in a single read loop, and values ​​indicating the step size between each step within the single read loop. Alternatively, the parameters configured by the software include: a value indicating how many steps to read in each layer of the read loop, and a value indicating the step size between each step in each layer of the read loop, wherein each layer of the read loop is performed in a nested manner from the outer layer to the inner layer.

11. The method according to claim 9, wherein, The parameters configured via software include: a value representing the number of addresses between the address of the first element read in a single read loop and the initial address of the read tensor data in memory.

12. The method according to claim 9, wherein, The parameters configured via software include conditions for the parameters, wherein the parameters take different values ​​when the conditions are met and when the conditions are not met.

13. The method according to claim 12, wherein, The parameter represents the number of read steps performed in a specific layer of read loop, and the condition is the number of steps performed in another read loop at an outer layer than the specific layer.

14. The method according to any one of claims 10-13, further comprising: The address to be read is calculated based on the current step of the read loop in one or more layers and the step size of each read loop in one or more layers.

15. The method according to claim 9, wherein, The tensor operation is a tensor operation that does not change the values ​​in the input tensor, wherein the address in the memory to be read at each read is calculated in one or more nested read loops based on the configured parameters received through the interface, such that the read replaces the tensor operation.

16. The method according to claim 15, wherein, The tensor operations include at least one of the following: transpose operator operations, reshape operator operations, broadcast operator operations, gather operator operations, reverse operator operations, concat operator operations, and cast operator operations. The memory is divided into multiple channels for storing data that can be accessed in parallel. The method also includes accessing the data stored in the multiple channels of the memory in parallel through a fully cross-connected path with separate read and write functions.

17. An electronic device comprising: Memory, used to store instructions; A processor for reading instructions from the memory and executing the method as described in any one of claims 9-16.

18. A non-temporary storage medium having instructions stored thereon, in, When the instruction is read by the processor, it causes the processor to execute the method as described in any one of claims 9-16.

Images