Method for preloading descriptors, artificial intelligence chip, computing device, medium and program product

Abstract

The application relates to a method for preloading descriptors, an artificial intelligence chip, a computing device, a medium and a program product. The method comprises the following steps: in response to confirming a kernel function task start, a computing task construction unit sends a descriptor preloading request, so that a bus interface acquires descriptor data from a global memory based on the descriptor preloading request; the bus interface broadcasts the acquired descriptor data to all computing units for executing the kernel function task, so that the computing units store the descriptor data into local static random access memories; and the kernel function task is executed based on the descriptor data stored in the static random access memories. The application can significantly shorten the path for loading descriptor data in the kernel function task execution process, shorten the loading time of the descriptor data, and improve the computing efficiency.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application generally relate to the field of artificial intelligence, and more particularly, to a method for preloading descriptors, an artificial intelligence chip, a computing device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] In a conventional method for loading descriptors, particularly in a descriptor loading method during kernel function task execution, descriptors are usually stored in a global memory for loading. However, the problem of descriptor loading from the global memory is that, in the address calculation process, descriptor data needs to be loaded from the global memory via a cluster bus interface (CBI) multiple times, the path of each loading is long, a long time is needed for loading, and the loading time is not fixed, thereby affecting the execution efficiency of the kernel function task.

[0003] In summary, the conventional method for preloading descriptors has the problem of a long path for loading descriptor data during kernel function task execution, a long time needed for descriptor loading, and thus affecting the computing efficiency. SUMMARY

[0004] The present application provides a method for preloading descriptors, an artificial intelligence chip, a computing device, a medium, and a program product, which can significantly shorten the path for loading descriptor data during kernel function task execution, shorten the loading time of descriptor data, and improve the computing efficiency.

[0005] According to a first aspect of the present application, a method for preloading descriptors is provided, which comprises: in response to confirming the start of a kernel function task, a task construction unit sending a descriptor preloading request, so that a cluster bus interface acquires descriptor data from a global memory based on the descriptor preloading request; the cluster bus interface broadcasting the acquired descriptor data to all computing units for executing the kernel function task, so that the computing units store the descriptor data in local static random access memories; and executing the kernel function task based on the descriptor data stored in the static random access memories.

[0006] In some embodiments, the method comprises: in response to a tensor access instruction issued by an instruction sender, executing the kernel function task based on the descriptor data in the static random access memories; and in response to a descriptor read-write instruction issued by the instruction sender, performing a read-write operation on the descriptor data in the static random access memories.

[0007] In some embodiments, the executing the kernel function task based on the descriptor data in the static random access memory in response to the tensor access instruction issued by the instruction sender comprises: the instruction sender issuing the position index and the coordinate information of the descriptor to the address calculation unit along with the tensor access instruction; the address calculation unit reading the descriptor data corresponding to the position index of the descriptor from the static random access memory based on the position index of the descriptor, and performing address calculation operation according to the descriptor data to generate a target address; and the address calculation unit accessing the tensor data stored in the global memory according to the target address via the bus interface.

[0008] In some embodiments, the accessing the tensor data stored in the global memory according to the target address via the bus interface by the address calculation unit comprises: in response to determining that the tensor access instruction indicates reading the tensor data, the address calculation unit reads the tensor data stored in the global memory at the target address via the bus interface, and stores the read tensor data into the thread register.

[0009] In some embodiments, the accessing the tensor data stored in the global memory according to the target address via the bus interface by the address calculation unit comprises: in response to determining that the tensor access instruction indicates writing the tensor data, the address calculation unit stores the to-be-written tensor data into the global memory at the target address via the bus interface, the to-be-written tensor data being issued to the address calculation unit along with the tensor access instruction.

[0010] In some embodiments, the performing the read-write operation on the descriptor data in the static random access memory in response to the descriptor read-write instruction issued by the instruction sender comprises: the instruction sender issuing the position index of the descriptor to the address calculation unit along with the descriptor read-write instruction.

[0011] In some embodiments, the method comprises: the address calculation unit reading the descriptor data indicated by the position index of the descriptor from the static random access memory in response to the descriptor read-write instruction indicating reading the descriptor data, and storing the read descriptor data into the thread register.

[0012] In some embodiments, the method comprises: the address calculation unit overwriting the descriptor data corresponding to the position index of the descriptor in the static random access memory as updated descriptor data in response to the descriptor read-write instruction indicating overwriting the descriptor data, the updated descriptor data being issued to the address calculation unit along with the descriptor read-write instruction.

[0013] In some embodiments, the sending, by the bus interface, the descriptor preloading request to cause the bus interface to fetch the descriptor data from the global memory based on the descriptor preloading request comprises that the descriptor preloading request comprises an address and a size of the descriptor to be fetched, and the bus interface fetches the descriptor data matching the address and the size of the descriptor from the global memory.

[0014] In some embodiments, the broadcasting, by the bus interface, the fetched descriptor data to all the compute units for executing the kernel function task so that the compute units store the descriptor data into local static random access memories further comprises that the compute unit sends an acknowledgement to the compute task building unit in response to having stored the corresponding descriptor data into the static random access memory inside the compute unit.

[0015] According to a second aspect of the present application, there is also provided an artificial intelligence chip, comprising: a global memory configured to store descriptor data and tensor data; a compute task building unit configured to send, in response to an acknowledgement of a kernel function task start, a descriptor preloading request to a bus interface to cause the bus interface to fetch the descriptor data from the global memory based on the descriptor preloading request; the bus interface configured to broadcast the fetched descriptor data to all the compute units for executing the kernel function task; and a compute kernel comprising: a plurality of compute units, at least part of the compute units in the plurality of compute units being configured to receive the descriptor data broadcasted by the bus interface, store the descriptor data into local static random access memories, and execute the kernel function task based on the descriptor data stored in the static random access memories.

[0016] In some embodiments, the compute unit further comprises: an address computation unit configured to execute the kernel function task based on the descriptor data in the static random access memory in response to a tensor access instruction issued by the instruction transmitter, and perform a read-write operation on the descriptor data in the static random access memory in response to a descriptor read-write instruction issued by the instruction transmitter.

[0017] According to a third aspect of the present application, there is also provided a computing device. The computing device comprises: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the computing device to perform the method of the first aspect of the present application.

[0018] According to a fourth aspect of the present application, there is also provided a computer readable storage medium. The computer readable storage medium has stored thereon a computer program, and the computer program is executed by a machine to perform the method of the first aspect of the present application.

[0019] According to a fifth aspect of the present application, there is also provided a computer program product comprising a computer program which, when executed by a machine, performs the method of the first aspect of the present application.

[0020] The present application, after the kernel function task is started, initiates a descriptor preloading request, stores the descriptor data stored in the global memory into all the computing units used for executing the kernel function task through the bus interface, so that the computing units can directly obtain the corresponding descriptor data by accessing the local static random access memory when executing the kernel function task, without accessing the global memory through the bus interface. Thus, the present application preloads the descriptor data into the static random access memory of the computing units when executing the kernel function task, thereby improving the loading speed of the descriptor data. Therefore, the present application can significantly shorten the path for loading the descriptor data during the execution of the kernel function task, shorten the loading time of the descriptor data, and improve the computing efficiency.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other features, aspects, and advantages of embodiments of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. In the drawings similar or corresponding elements are denoted by like or corresponding reference numbers.

[0023] Figure 1 A schematic diagram of a computing device for implementing a method for preloading descriptors according to embodiments of the present application is schematically shown.

[0024] Figure 2 A flowchart of a method for preloading descriptors according to some embodiments of the present application is shown.

[0025] Figure 3 A structural schematic diagram of an artificial intelligence chip according to some embodiments of the present application is shown.

[0026] Figure 4 A flowchart of a method for executing a kernel function task based on descriptor data according to some embodiments of the present application is shown.

[0027] Figure 5 A flowchart of a method for performing read-write operations on descriptor data according to some embodiments of the present application is shown.

[0028] In the various drawings, the same or corresponding reference numbers denote the same or corresponding parts. DETAILED DESCRIPTION

[0029] Preferred embodiments of the present application will be described herein below with reference to the accompanying drawings. While the preferred embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0030] The term "comprising" and variations thereof as used herein are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0031] To at least partially solve one or more of the above problems and other potential problems, example embodiments of the present application propose a method for preloading descriptors. In the method, a descriptor preloading request is sent by a computing task construction unit to cause a bus interface to fetch descriptor data from a global memory based on the descriptor preloading request; the bus interface broadcasts the fetched descriptor data to all computing units for executing a kernel function task, so that the computing units store the descriptor data into local static random access memories; and the kernel function task is executed based on the descriptor data stored in the static random access memories. Since the computing units have preloaded the descriptor information related to the kernel function task into the local static random access memories before executing the kernel function task, the kernel function task does not need to fetch from the global memory during the execution of the kernel function task, but directly fetches from the local static random access memories, thereby the present application can significantly shorten the path for loading the descriptor data during the execution of the kernel function task, shorten the loading time of the descriptor data, and improve the computing efficiency.

[0032] Figure 1 A schematic diagram of a computing device 100 for implementing the method for preloading descriptors according to an embodiment of the present application is schematically shown. As shown in FIG. 1, the computing device 100 includes a bus interface 110, a global memory 120, a plurality of computing units 130, and a plurality of static random access memories 140. Figure 1As shown, the computing device 100 can have one or more processing units and memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor. The processing units include special purpose processing units such as graphics processing units (GPUs), field programmable gate arrays (FPGAs), and application specific integrated circuits (ASICs), general-purpose computing on graphics processing units (GPGPU), and general purpose processing units such as CPUs. The computing device 100 further includes at least a descriptor preloading request sending unit 102, a descriptor data broadcasting unit 104, and a kernel function task executing unit 106.

[0033] With respect to the descriptor preloading request sending unit 102, it is configured to, in response to confirming a kernel function task is initiated, send, by the computing task building unit, a descriptor preloading request to cause the bus interface to fetch descriptor data from the global memory based on the descriptor preloading request.

[0034] With respect to the descriptor data broadcasting unit 104, it is configured to cause the bus interface to broadcast the fetched descriptor data to all computing units used to execute the kernel function task, so that the computing units store the descriptor data to local static random access memories.

[0035] With respect to the kernel function task executing unit 106, it is configured to execute the kernel function task based on the descriptor data stored in the static random access memories.

[0036] Embodiments of the present application will be described below with reference to the accompanying drawings. Figure 2 and Figure 3 A method 200 for preloading descriptors of an embodiment of the present application will be described. It should be understood that the method 200 can be performed, for example, at the computing device 100 described above. The method 200 can also include additional actions not shown and / or can omit actions shown, without limitation in this regard. Figure 1 The method 200 can also include additional actions not shown and / or can omit actions shown, without limitation in this regard.

[0037] At step 202, the computing device 100 sends, by the computing task building unit, a descriptor preloading request to cause the bus interface to fetch descriptor data from the global memory based on the descriptor preloading request, in response to confirming a kernel function task is initiated.

[0038] As to the computing device 100, which at least comprises the artificial intelligence chip 300, it should be understood that the computing device can further comprise other components. As shown in Figure 3 the artificial intelligence chip 300 at least comprises a computing core 302, which further comprises one or more computing units 309, a compute task construction unit 304, and a global memory 308. The computing unit 309 further comprises an instruction transmitter 310, an address computing unit 312, a thread register 314, and a static random access memory 316.

[0039] As to the descriptor preloading request, it is, for example, a request sent by the compute task construction unit (CTC) (as shown in 304) of the computing core 302 in response to the kernel function task initiation, about preloading descriptor data. Figure 3

[0040] As to the descriptor data, it is, for example, descriptor data stored in the global memory (as shown in 308) of the computing device 100. In some embodiments, the descriptor data, for example, indicates data for describing the configuration information (such as height, width, data type, etc.) of the data type, and the descriptor data and its corresponding tensor data for executing the kernel function task are stored in the global memory 308 in a segmented manner (for example, in a segmented one-dimensional data manner). In some embodiments, the computing core 302 obtains the descriptor data located in the global memory 308 based on the descriptor index, and calculates the address of the tensor information of the kernel function task to be executed in the global memory 308 according to the obtained descriptor data and the preset coordinate information issued by the instruction transmitter 310, which is, for example, pre-configured by the user. Figure 3

[0041] As to the global memory, it is, for example, the global memory 308 in the artificial intelligence chip 300. The tensor data (such as tensor 0, tensor 1, and tensor 2) and the descriptor data for describing the configuration information of the data type of the tensor data are stored in the global memory 308 in a segmented manner. In some embodiments, the descriptor data and the tensor data are copied to the global memory of the device end by the host end. As to the storage location of the descriptor data in the global memory, it is, for example, determined by the driver in a pre-filled configuration template manner.

[0042] ​​At step 204, the bus interface of the computing device 100 broadcasts the fetched descriptor data to all the computing units for performing the kernel function task, so that the computing units store the descriptor data into local static random access memories.

[0043] As to the method of broadcasting the fetched descriptor data to all the computing units for performing the kernel function task by the bus interface, it for example comprises that the bus interface (as shown in label 306) broadcasts the fetched descriptor data to all the computing units 309 (as shown in label 308) for performing the task of the kernel function, so that the computing units 309 receiving the descriptor data store the descriptor data into local static random access memories (desc SRAM) 316 according to the descriptor addresses, as shown in label 324. Figure 3 Figure 3 Only one of all the computing units is shown by way of example, and it should be understood that the computing units for performing the task of the kernel function can be multiple computing units. Figure 3

[0044] As to the local static random access memory, it for example is a static random access memory 316 arranged inside the computing unit 309. In some embodiments, the static random access memory 316 is configured to store preloaded descriptor information related to performing the kernel function task. In some embodiments, the static random access memory 316 provides the descriptor information into the thread registers (as shown in label 314) of the address computing unit 309 in response to the descriptor-related instructions of the instruction transmitter (as shown in label 310). Figure 3 Figure 3

[0045] In some embodiments, the method of broadcasting the fetched descriptor data to all the computing units for performing the kernel function task by the bus interface of the computing device 100 for example comprises that the computing unit 309 sends an acknowledgement (as shown in label 326) to the computing task building unit 304 in response to having stored the corresponding descriptor data into the static random access memory inside the computing unit 309. In some embodiments, the acknowledgement (e.g., ack) indicates the prompt information sent by each computing unit 309 to the computing task building unit 304 for confirming that the descriptor data has been stored into the static random access memory inside the computing unit 309. Figure 3

[0046] At step 204, the computing device 100 performs the kernel function task based on the descriptor data stored in the static random access memory.

[0047] ​​​​​In some embodiments, the method of performing a kernel function task based on the descriptor data stored in the static random access memory 316, for example, comprises: the computing core 302 calculates the target address of the corresponding tensor data based on the descriptor data stored in the static random access memory 316, and acquires the corresponding tensor data from the global memory 308 according to the address of the tensor data, so that the computing unit 309 performs the kernel function task according to the acquired tensor data.

[0048] The method of acquiring the corresponding tensor data from the global memory 308 according to the target address of the tensor data, for example, comprises: the computing device 100 accesses the bus interface 306 according to the address of the tensor data (as indicated by mark 332), the bus interface 306 accesses the tensor data indicated by the target address in the global memory 308 (as indicated by mark 334), and reads the tensor data into the bus interface 306 (as indicated by mark 336), and then returns to the thread register 314 of the computing unit 309 (as indicated by mark 338).

[0049] In the above scheme, the computing task construction unit sends a descriptor preloading request, the bus interface acquires descriptor data from the global memory based on the descriptor preloading request, and the acquired descriptor data is broadcast to all computing units for performing kernel function tasks, so that the computing units store the descriptor data in the local static random access memory and perform the kernel function task. Therefore, the present application can pre-load the descriptor data for performing the kernel function task into the local static random access memory, so that the kernel function does not need to be acquired from the global global memory during execution, but is directly acquired from the local static random access memory. The present application can greatly shorten the descriptor acquisition link during kernel function task execution, accelerate the loading speed of descriptor information, and thus improve the execution efficiency of the kernel function task. Therefore, the present application can significantly improve the computing efficiency of the hardware computing unit of the artificial intelligence chip.

[0050] The following will be described in detail Figure 3 The structure of the artificial intelligence chip 300 of the embodiment of the present application will be described. As shown in Figure 3 The artificial intelligence chip 300, for example, at least comprises a global memory 308, a computing task construction unit 304, a bus interface 306 and a computing core 302.

[0051] Regarding global memory 308, it is configured, for example, to store descriptor data and tensor data. Global memory 308 is, for example, high bandwidth memory (HBM). In some embodiments, the descriptor data and tensor data stored in global memory 308 are copied together from the host side to the global memory on the device side used to execute the kernel function task. The descriptor data and tensor data are, for example, pre-configured by the user. In some embodiments, tensor data and descriptor data are stored in global memory 308 in segments; for example, tensor data is stored in global memory 308 as segmented one-dimensional data.

[0052] Regarding the computation task construction unit 304, it is configured, for example, to send a descriptor preload request to the bus interface 306 in response to acknowledging the start of the kernel function task. Figure 3 As indicated by label 318, this enables bus interface 306 to retrieve descriptor data from global memory 308 based on a descriptor preload request, as shown below. Figure 3 As shown in the markings 320 and 322.

[0053] Regarding bus interface 306, it is configured, for example, to broadcast the acquired descriptor data to all computational units used to execute kernel function tasks. Specifically, as... Figure 3 As shown, the bus interface 306 broadcasts the acquired descriptor data to all computing units used to execute kernel function tasks. Figure 3 (Only one computing unit among all computing units is illustrated schematically). In some embodiments, in response to having stored the corresponding descriptor data in its internal static random access memory 316, all computing units performing the kernel function task send an acknowledgment (e.g., an ack response) to the computing task construction unit, such as... Figure 3 As shown in the middle label 326.

[0054] Regarding the computing core 302, it includes, for example, a plurality of computing units 309 (CUs), at least some of which are configured to receive descriptor data broadcast by the bus interface 306 and store the descriptor data in local static random access memory 316, such as Figure 4 As shown in label 324, the kernel function task is executed based on the descriptor data stored in static random access memory 316.

[0055] In some embodiments, the computing unit 309 further comprises, for example, an address computation unit 312 configured to, in response to a tensor access instruction issued by the instruction transmitter 310, perform a kernel function task based on descriptor data in the static random access memory 316, and in response to a descriptor read-write instruction issued by the instruction transmitter 310, perform a read-write operation on the descriptor data in the static random access memory 316.

[0056] In the above scheme, the computing task construction unit of the artificial intelligence chip sends a descriptor preloading request, so that the bus interface acquires descriptor data from the global memory based on the descriptor preloading request, and the descriptor data is broadcast to all computing units for performing a kernel function task, and the received descriptor data is stored in the local static random access memory through the broadcast, and the kernel function task is directly performed based on the descriptor data stored in the static random access memory. Thus, the application can pre-load the descriptor data into the local static random access memory before performing the kernel function task, and then directly acquire the descriptor data from the local memory during the execution of the kernel function task, instead of acquiring the descriptor data from the global memory through the bus interface, which greatly shortens the acquisition time of the descriptor data. The application can significantly shorten the path of loading the descriptor data during the execution of the kernel function task, shorten the loading time of the descriptor data, and improve the computing efficiency.

[0057] As described above, the computing device 100 performs a kernel function task based on descriptor data in the static random access memory in response to a tensor access instruction issued by the instruction transmitter. Therefore, the method 200 may, for example, further comprise a method 400 of performing a kernel function task based on descriptor data. The method 400 of performing a kernel function task based on descriptor data according to an embodiment of the application will be described below in combination with Figure 1 and Figure 3 The method 400 of performing a kernel function task based on descriptor data according to an embodiment of the application will be described below in combination with Figure 3 The method 400 may, for example, be performed at the computing device 100 described above. The method 400 may, for example, further comprise additional actions not shown and / or may omit the actions shown, and the scope of the application is not limited in this respect.

[0058] At step 402, the instruction transmitter of the computing device 100 issues the position index and coordinate information of the descriptor together with a tensor access instruction to the address computation unit.

[0059] As to the tensor access instructions, they include, for example, a tensor move instruction, a tensor read instruction, and a tensor write instruction, to perform a corresponding read operation or write operation on the tensor data (e.g., tensor 0, tensor 1, tensor 2) stored in the global memory 308 based on the preloaded descriptor information. The tensor move instruction (e.g., tma instruction) indicates the address calculation unit 312 to calculate the address of the target tensor in the global memory 308 based on the position index (e.g., desc_idx) and the coordinate information (e.g., desc_pos) of the descriptor; the tensor read instruction (e.g., load instruction) indicates to read the corresponding tensor data from the global memory 308; and the tensor write instruction (e.g., store instruction) indicates to write the tensor data to be written to the preset position in the global memory 308 via the bus interface 306.

[0060] For example, as shown in Figure 3 the instruction transmitter 310 sends the position index and the coordinate information of the descriptor to the address calculation unit 312 together with the tensor access instruction, so as to enable the address calculation unit 312 to perform a corresponding operation, as indicated by the mark 328.

[0061] At step 404, the address calculation unit of the computing device 100 reads the descriptor data corresponding to the position index of the descriptor from the static random access memory based on the position index of the descriptor, and performs an address calculation operation according to the descriptor data to generate a target address.

[0062] In some embodiments, the method of performing an address calculation operation according to the descriptor data to generate a target address includes, for example, the address calculation unit 312 accessing the tensor data stored in the global memory 308 according to the target address via the bus interface 306, which includes, for example, reading the tensor data or writing the tensor data.

[0063] For example, as shown in Figure 3 the address calculation unit 312 reads the descriptor data corresponding to the position index of the descriptor from the static random access memory 316 based on the position index of the descriptor (as indicated by the mark 340), and provides the read descriptor data to the address calculation unit 312 (as indicated by the mark 330), so that the address calculation unit 312 performs an address calculation operation according to the descriptor data and generates a target address, which indicates the address of the corresponding tensor in the global memory 308 in the tensor access instruction.

[0064] In some embodiments, the static random access memory 316 is independent of the shared memory of the computing unit 309. It should be understood that the reason for using a separately configured static random access memory 316 for storing descriptor data in the computing unit 309 is that if descriptor data were stored in the shared memory (not shown) of the computing unit 309, high-frequency read and write operations on the shared memory would consume a large amount of memory bandwidth, thus affecting the normal read and write functions of the computing unit.

[0065] At step 406, in response to the determination of the tensor access instruction, the computing device 100 reads the tensor data. The address calculation unit reads the tensor data stored in the target address of the global memory via the bus interface and stores the read tensor data in the thread register.

[0066] In some embodiments, such as Figure 3 As shown, in response to a tensor access instruction, the address calculation unit 312 of the computation core 302 reads tensor data. The address calculation unit 312 provides the target address of the tensor data to the bus interface 306 (as shown by label 332). The bus interface 306 reads the tensor data stored in the target address of the global memory 308 based on the target address of the tensor data. The bus interface 306 reads the tensor data and then stores the read tensor data in the thread register 314 (as shown by label 338).

[0067] At step 408, in response to the tensor access instruction, the computing device 100 indicates that tensor data should be written. The address calculation unit stores the tensor data to be written in the target address of the global memory via the bus interface. The tensor data to be written is sent to the address calculation unit along with the tensor access instruction.

[0068] For example, such as Figure 5 As shown, in response to the address calculation unit 312 determining that a tensor access instruction indicates that tensor data is to be written, the tensor data to be written and the target address are provided to the bus interface 306, and then the tensor data to be written is stored in the target address of the global memory 308. In some embodiments, the tensor data to be written is sent from the instruction sender 310 to the address calculation unit 312 along with the tensor access instruction, as shown by reference numeral 328.

[0069] In the above solution, the position index and coordinate information of the descriptor are sent by the instruction sender to the address calculation unit together with the tensor access instruction, so that the address calculation unit performs a read or write operation on the tensor data stored in the global memory via the bus interface. Since the calculation unit has preloaded the descriptor information related to the kernel function task into the local static random access memory before performing the kernel function task, the acquisition link of the descriptor data is shortened, thereby accelerating the loading speed of the descriptor information. In addition, since the descriptor information is stored in the independently arranged static random access memory instead of the shared memory of the calculation unit, the read and write operations of the descriptor data do not occupy the bandwidth of the calculation unit, thereby avoiding the influence of the read and write functions of the calculation unit caused by the local storage of the descriptor information. Therefore, the computing efficiency of the hardware calculation unit of the artificial intelligence chip is significantly improved.

[0070] As described above, the computing device 100 performs a read or write operation on the descriptor data in the static random access memory in response to the descriptor read or write instruction sent by the instruction sender. Therefore, the method 200 may, for example, further include a method 500 of performing a read or write operation on the descriptor data. The method 500 of performing a read or write operation on the descriptor data will be described below in combination with Figure 1 and Figure 3 The method 500 of performing a read or write operation on the descriptor data of the embodiments of the present application will be described. It should be understood that the method 500 may, for example, be performed at the computing device 100 described above. The method 500 may further include additional actions not shown and / or the actions shown may be omitted, and the scope of the present application is not limited in this respect. Figure 3 The method 500 may, for example, be performed at the computing device 100 described above. The method 500 may further include additional actions not shown and / or the actions shown may be omitted, and the scope of the present application is not limited in this respect.

[0071] At step 502, the instruction sender of the computing device 100 sends the position index of the descriptor to the address calculation unit together with the descriptor read or write instruction.

[0072] Regarding the method of sending the position index of the descriptor by the instruction sender 310, the instruction sender 310 may, for example, send the position index of the descriptor (e.g., desc idx) to the address calculation unit 312 together with the descriptor read or write instruction as shown in Figure 3 The descriptor read or write instruction may, for example, include a read descriptor instruction and a rewrite descriptor instruction to perform a read operation and a write operation on the descriptor data stored in the static random access memory 316.

[0073] At step 504, the address calculation unit of the computing device 100 reads the descriptor data indicated by the position index of the descriptor from the static random access memory in response to the descriptor read or write instruction indicating a read operation on the descriptor data, and stores the read descriptor data into a thread register.

[0074] As described above, the computing device 100 performs a read or write operation on the descriptor data in the static random access memory in response to the descriptor read or write instruction sent by the instruction sender. Therefore, the method 200 may, for example, further include a method 500 of performing a read or write operation on the descriptor data. The method 500 of performing a read or write operation on the descriptor data will be described below in combination with ​As shown, the address calculation unit 312 responds to a descriptor read instruction (e.g., a load instruction) issued by the instruction sender 310, instructing the read descriptor data to read the descriptor data indicated by the location index of the descriptor from the static random access memory, and stores the read descriptor data in the thread register 314, as shown in label 342.

[0075] At step 506, the address calculation unit of the computing device 100 responds to the descriptor read / write instruction by rewriting the descriptor data, rewriting the descriptor data corresponding to the location index of the descriptor in the static random access memory to update the descriptor data, which is sent to the address calculation unit along with the descriptor read / write instruction.

[0076] like ​ As shown, the address calculation unit 312 responds to a descriptor read instruction (e.g., a store instruction) issued by the instruction sender 310, instructing it to rewrite the descriptor data, and rewrites the descriptor data corresponding to the location index of the descriptor in the static random access memory 316 to updated descriptor data. In some embodiments, the updated descriptor data is issued to the address calculation unit along with the descriptor read / write instruction.

[0077] In the above scheme, the descriptor's position index is sent to the address calculation unit along with the descriptor read / write instructions via the instruction sender. This allows the address calculation unit to perform read or write operations on the descriptor data in the static random access memory (SRAM) locally based on the descriptor read / write instructions. It should be understood that during the execution of kernel function tasks such as tensor quantization, transpose, and concatenation, each thread in the thread block needs to frequently read the same descriptor data, increasing the access frequency of global memory and thus increasing the computation time of the AI ​​chip. However, the descriptor preloading method of this invention allows control of the tensor computation process without accessing global memory by reading or modifying the descriptor data in the local SRAM, making it particularly suitable for computation scenarios involving tensor quantization, transpose, and concatenation. Therefore, this invention further improves the flexibility of kernel function tasks, shortens the descriptor preloading time, and thus optimizes the execution efficiency of kernel function tasks.

[0078] The various processes and processes described above, such as the methods 200, 400, 500, can be performed at a computing device. The computing device includes, for example, at least one processor (at least one graphics processor and at least one central processor) and a memory communicatively connected with the at least one processor, wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor. In some embodiments, the methods 200, 400, 500 can be implemented as a computer software program or program product tangibly embodied in a machine-readable medium. In some embodiments, part or all of the computer program can be loaded and / or installed onto the computing device via a read-only memory (ROM) and / or a communication unit. When the computer program is loaded onto a random-access memory (RAM) and executed by the GPU and CPU, one or more actions of the methods 200, 400, 500 described above can be performed.

[0079] The present application can be a method, an apparatus, a system, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for performing various aspects of the present application. The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.

[0080] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The various aspects of the application can be described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0081] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium can also have other meanings inhered thereby, which will be apparent to those reading the present disclosure. The instructions stored in the computer readable storage medium, when executed by a computer or other programmable data processing apparatus, produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0082] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to the present application. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions ("instructions"). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0083] It should be understood that various forms of flow can be used over the course of the above described steps. For example, the steps loaded in the present application can be executed in parallel, in sequence, or in a different order, as long as the desired results of the technical solutions disclosed in the present application can be achieved, which are not limited herein.

[0084] The specific embodiments described above have been disclosed by way of example and not limitation, and modifications, combinations, sub-combinations, and alternatives can occur to others skilled in the art upon reading the description. It is intended that the application be construed as including all such modifications, combinations, sub-combinations, and alternatives.

Claims

1. A method for preloading descriptors, characterized in that, The method comprises: In response to confirming the kernel function task is started, the computing task construction unit sends a descriptor preloading request, so that the bus interface acquires descriptor data from the global memory based on the descriptor preloading request; The bus interface broadcasts the acquired descriptor data to all computing units for executing the kernel function task, so that the computing units store the descriptor data into local static random access memories; and The kernel function task is executed based on the descriptor data stored in the static random access memories.

2. The method of claim 1, wherein, The method comprises: In response to a tensor access instruction issued by the instruction sender, the kernel function task is executed based on the descriptor data in the static random access memories; and In response to a descriptor read-write instruction issued by the instruction sender, a read-write operation is performed on the descriptor data in the static random access memories.

3. The method of claim 2, wherein, In response to a tensor access instruction issued by the instruction sender, the kernel function task is executed based on the descriptor data in the static random access memories, which comprises: The instruction sender issues the position index and coordinate information of the descriptor together with the tensor access instruction to the address calculation unit; The address calculation unit reads the descriptor data corresponding to the position index of the descriptor from the static random access memories based on the position index of the descriptor, and performs address calculation operation according to the descriptor data to generate a target address; And The address calculation unit accesses the tensor data stored in the global memory according to the target address via the bus interface.

4. The method of claim 3, wherein, The address calculation unit accesses the tensor data stored in the global memory according to the target address via the bus interface, which comprises: In response to determining that the tensor access instruction indicates reading tensor data, the address calculation unit reads the tensor data stored in the global memory at the target address via the bus interface, and stores the read tensor data into a thread register.

5. The method of claim 3, wherein, The address calculation unit accesses the tensor data stored in the global memory according to the target address via the bus interface, which comprises: In response to determining that the tensor access instruction indicates writing tensor data, the address calculation unit stores the to-be-written tensor data into the global memory at the target address via the bus interface, the to-be-written tensor data being issued to the address calculation unit together with the tensor access instruction.

6. The method of claim 2, wherein, In response to a descriptor read-write instruction issued by the instruction sender, a read-write operation is performed on the descriptor data in the static random access memories, which comprises: The instruction sender issues the position index of the descriptor together with the descriptor read-write instruction to the address calculation unit.

7. The method of claim 6, wherein, The method comprises: In response to the descriptor read-write instruction indicating reading descriptor data, the address calculation unit reads the descriptor data indicated by the position index of the descriptor from the static random access memories, and stores the read descriptor data into a thread register.

8. The method of claim 6, wherein, The method comprises: In response to the descriptor read-write instruction indicating rewriting descriptor data, the address calculation unit rewrites the descriptor data corresponding to the position index of the descriptor in the static random access memories as updated descriptor data, the updated descriptor data being issued to the address calculation unit together with the descriptor read-write instruction.

9. The method of claim 1, wherein, The sending of the descriptor preloading request to cause the bus interface to fetch the descriptor data from the global memory based on the descriptor preloading request comprises: The descriptor preloading request comprises an address and a size of the descriptor to be fetched, and the bus interface fetches the descriptor data matching the address and the size of the descriptor from the global memory.

10. The method of claim 1, wherein, The broadcasting of the fetched descriptor data to all the compute units for performing the kernel function task by the bus interface so that the compute units store the descriptor data into local static random access memories further comprises: The compute unit sends an acknowledgement to the compute task building unit in response to having stored the corresponding descriptor data into the static random access memory inside the compute unit.

11. An artificial intelligence chip, characterized by, The artificial intelligence chip comprises: A global memory configured to store the descriptor data and the tensor data; A compute task building unit configured to send a descriptor preloading request to the bus interface to cause the bus interface to fetch the descriptor data from the global memory based on the descriptor preloading request in response to the acknowledgement of the kernel function task start; The bus interface is configured to broadcast the fetched descriptor data to all the compute units for performing the kernel function task; The compute kernel comprises: A plurality of compute units, at least part of the compute units in the plurality of compute units are configured to receive the descriptor data broadcasted by the bus interface, store the descriptor data into local static random access memories, and perform the kernel function task based on the descriptor data stored in the static random access memories.

12. The artificial intelligence chip according to claim 11, characterized in that, The compute unit further comprises: An address computing unit configured to perform the kernel function task based on the descriptor data in the static random access memory in response to a tensor access instruction issued by the instruction sender, and perform a read-write operation on the descriptor data in the static random access memory in response to a descriptor read-write instruction issued by the instruction sender.

13. A computing device, comprising: comprise: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method in any one of claims 1-10. The computer program stored on the computer readable storage medium is executed by a machine to perform the method in any one of claims 1-10.

14. A computer-readable storage medium, characterized in that, The computer program is executed by a machine to perform the method in any one of claims 1-10.

15. A computer program product, characterised in that, ​

Images