Methods for performing atomic addition operations, artificial intelligence chips, computing devices, media, and program products.

Abstract

This invention relates to a method for performing atomic addition operations, an artificial intelligence chip, a computing device, a medium, and a program product. The method includes: in response to receiving an atomic addition operation request, determining multiple thread requests to add a predetermined value to the same address; merging the multiple thread requests into a single request; performing the addition operation in a single cycle based on the merged request to generate a calculation result of the addition operation; and determining the respective offset values ​​of the multiple threads pointing to the same address, for generating return values ​​for the multiple thread requests based on initial values ​​in static random access memory and the respective offset values ​​of the multiple threads. This invention can complete the addition calculation of multiple thread requests within one cycle, effectively reducing the computation time of multiple threads at the same address. Furthermore, the return values ​​can be used for further deep optimization of the current operator.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of artificial intelligence technology, and more specifically to a method for performing atomic addition operations, an artificial intelligence chip, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In traditional methods for performing atomic addition operations, especially those involving thread data, load store caches typically employ a multi-cycle approach. For example, if multiple thread (e.g., lane) addresses (e.g., addresses of 32 threads) are passed in during an atomic addition operation, and multiple threads point to the same address, the load store cache (LSC) splits the atomic addition requests from these threads into multiple requests and executes each request sequentially over multiple cycles. Thus, although multiple threads pointing to the same address are merged into a single memory bank (e.g., a bank) for atomic addition, only one atomic addition operation can be performed on a single memory bank per clock cycle, meaning the atomic addition operation for multiple threads pointing to the same address still requires multiple computation cycles.

[0003] In summary, the traditional methods for performing atomic addition operations have the following drawbacks: the atomic addition calculation process for multiple threads at the same address requires multiple cycles, which increases the time consumption of the atomic addition calculation of the threads and thus reduces the computational efficiency of the threads. Summary of the Invention

[0004] This invention provides a method, an artificial intelligence chip, a computing device, a computer-readable storage medium, and a computer program product for performing atomic addition operations, which can significantly reduce the computation time of atomic addition operations on multiple threads at the same address and improve computational efficiency.

[0005] According to a first aspect of the present invention, a method for performing an atomic addition operation is provided, the method comprising: in response to receiving an atomic addition operation request, determining multiple thread requests to add a predetermined value to the same address; merging the multiple thread requests into a single request; performing the addition operation in a single-cycle manner based on the merged request to generate a calculation result of the addition operation; and determining respective offset values ​​for the multiple threads pointing to the same address, for generating return values ​​for the multiple thread requests based on an initial value in a static random access memory and the respective offset values ​​for the multiple threads.

[0006] In some embodiments, in response to receiving an atomic addition operation request, determining multiple thread requests to add a predetermined value to the same address includes: in response to the load storage cache confirming that the received atomic addition operation request is based on a first type of atomic addition instruction or a second type of atomic addition instruction, the address calculation module included in the front end of the load storage cache determines multiple thread requests to add a predetermined value to the same address, wherein the atomic addition operation request based on the first type of atomic addition instruction needs to return a value, and the atomic addition operation request based on the second type of atomic addition instruction does not need to return a value.

[0007] In some embodiments, merging the multiple thread requests into one request includes: merging the multiple thread requests into one request via an address merging module at the front end of the load storage cache; and providing the merged request to the static random access memory controller of the load storage cache controller.

[0008] In some embodiments, performing a summation operation in a single-cycle manner based on the merged request to generate the result of the summation operation includes: reading an initial value associated with the same address from static random access memory to perform the summation operation using an atomic arithmetic logic unit; and writing the result of the summation operation into static random access memory.

[0009] In some embodiments, determining the respective offset values ​​of multiple threads pointing to the same address for generating return values ​​of multiple thread requests based on initial values ​​in static random access memory and the respective offset values ​​of the multiple threads includes: in response to determining that the received atomic addition operation request is based on a first type of atomic addition instruction or that the atomic addition operation request requires a return value, determining the respective offset values ​​of multiple threads pointing to the same address; using an atomic arithmetic logic unit, adding the respective offset values ​​of the multiple threads to the initial values ​​related to the same address in the read static random access memory to generate return values ​​of multiple thread requests; and providing the generated return values ​​of multiple thread requests to a thread register.

[0010] In some embodiments, providing the return values ​​of the generated multiple thread requests to the thread register includes: the return values ​​of the multiple thread requests generated by the atomic arithmetic logic unit are provided to the thread register via the backend of the load storage cache.

[0011] In some embodiments, the method further includes: the execution unit confirming whether the atomic addition operation included in the current operator needs a return value; in response to confirming that the atomic addition operation included in the current operator needs a return value, providing an atomic addition operation request based on a first type of atomic addition instruction to the front end of the loading memory cache; and in response to confirming that the atomic addition operation included in the current operator does not need a return value, providing an atomic addition operation request based on a second type of atomic addition instruction to the front end of the loading memory cache.

[0012] According to a second aspect of the present invention, an artificial intelligence chip is also provided, the artificial intelligence chip comprising: an execution unit configured to issue an atomic addition operation request to a load memory cache; the load memory cache comprising: a front end of the load memory cache configured to, in response to receiving the atomic addition operation request, determine multiple thread requests to add a predetermined value to the same address; and merge the multiple thread requests into a single request; and a load memory cache controller configured to, based on the merged request, perform the addition operation in a single-cycle manner to generate a calculation result of the addition operation; and determine the respective offset values ​​of the multiple threads pointing to the same address for generating return values ​​of the multiple thread requests based on initial values ​​in static random access memory and the respective offset values ​​of the multiple threads.

[0013] In some embodiments, the front end of the load storage cache includes: an address calculation module configured to determine, in response to receiving an atomic addition operation request, multiple thread requests to add a predetermined value to the same address; and an address merging module configured to merge the multiple thread requests into a single request; and to provide the merged request to the static random access memory controller of the load storage cache controller.

[0014] In some embodiments, the load storage cache controller includes: an atomic arithmetic logic unit configured at least to perform an addition operation in a single-cycle manner to generate a result of the addition operation; and to add corresponding offset values ​​of multiple threads to an initial value associated with the same address in the read static random access memory to generate return values ​​of multiple thread requests; and to provide the generated return values ​​of multiple thread requests to a thread register.

[0015] In some embodiments, the atomic arithmetic logic unit is further configured to: in response to determining that the received atomic addition operation request is based on a second type of atomic addition instruction or that the atomic addition operation request does not require a return value, perform the addition operation in a single-cycle manner based on an initial value to generate the calculation result of the addition operation; and write the calculation result of the addition operation into a static random access memory.

[0016] According to a third aspect of the present invention, a computing device is also provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the computing device to perform the method of the first aspect of the present invention.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0018] According to a fifth aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0019] This invention merges multiple thread requests that add a predetermined value to the same address into a single request and performs the addition operation in a single cycle. This avoids the problem of multiple threads at the same address needing to occupy multiple cycles to perform addition calculations separately. Therefore, this invention can significantly reduce the computation time of atomic addition operations for multiple threads at the same address, thus improving computational efficiency. Furthermore, this invention also determines the respective offset values ​​of the multiple threads pointing to the same address, which are used to generate the return values ​​of the multiple thread requests based on the initial value in the static random access memory and the respective offset values ​​of the multiple threads. Thus, this invention not only significantly improves the execution efficiency of artificial intelligence chips when processing atomic addition operations, but also further optimizes the performance of atomic addition operations.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0022] Figure 1 A schematic diagram of a computing apparatus for implementing a method of performing atomic addition operations according to an embodiment of the present invention is shown.

[0023] Figure 2 A flowchart of a method for performing atomic addition operations according to some embodiments of the present invention is shown.

[0024] Figure 3A schematic diagram of the structure of an artificial intelligence chip according to some embodiments of the present invention is shown.

[0025] Figure 4 A flowchart of a method for generating return values ​​for multiple thread requests is shown according to some embodiments of the present invention.

[0026] Figure 5 A flowchart of a method for determining whether a return value is required, according to some embodiments of the present invention, is shown.

[0027] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0028] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0029] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0030] To at least partially address one or more of the aforementioned problems and other potential issues, an exemplary embodiment of the present invention proposes a method for performing atomic addition operations. In this method, upon receiving an atomic addition operation request, multiple thread requests to add a predetermined value to the same address are identified; these multiple thread requests are merged into a single request; and based on the merged request, the addition operation is performed in a single-cycle manner to generate the calculation result of the addition operation. Because multiple thread requests to add a predetermined value to the same address are merged into a single request, the multiple threads adding the predetermined value can complete the addition calculation of multiple thread requests within one cycle, effectively reducing the computation time of multiple threads at the same address and improving computational efficiency. Furthermore, the present invention also determines the respective offset values ​​of the multiple threads pointing to the same address, which are used to generate the return values ​​of the multiple thread requests based on the initial value in the static random access memory and the respective offset values ​​of the multiple threads. Therefore, the present invention significantly improves the execution efficiency of artificial intelligence chips when processing atomic addition operations and further optimizes the performance of atomic addition operations.

[0031] Figure 1 A schematic diagram of a computing device 100 for implementing a method of performing atomic addition operations according to an embodiment of the present invention is shown. Figure 1 As shown, the computing device 100 may have one or more processing units and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. The processing unit includes dedicated processing units such as graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), general-purpose computing on graphics processing units (GPGPUs), and general-purpose processing units such as CPUs. The computing device 100 also includes at least: a thread request determination unit 102, a thread request merging unit 104, an addition operation unit 106, and a return value generation unit for multiple thread requests 108.

[0032] Regarding the thread request determination unit 102, it is used to determine multiple thread requests to add a predetermined value to the same address in response to receiving an atomic addition operation request.

[0033] Regarding the thread request merging unit 104, it is used to merge the multiple thread requests into one request.

[0034] Regarding the summation operation unit 106, it is used to perform a summation operation in a single cycle based on the merged request in order to generate the calculation result of the summation operation.

[0035] The return value generation unit 108 for multiple thread requests is used to determine the respective offset values ​​of multiple threads pointing to the same address, so as to generate the return value of multiple thread requests based on the initial value in the static random access memory and the respective offset values ​​of the multiple threads.

[0036] The following will combine Figure 2 and Figure 3 The present invention describes a method 200 for performing atomic addition operations according to embodiments of the invention. It should be understood that method 200 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 200 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0037] At step 202, in response to receiving an atomic addition operation request, the computing device 100 determines multiple thread requests to add a predetermined value to the same address.

[0038] Regarding requests for atomic addition operations, such as those made by execution units (EUs) within the computational core of computing device 100 (e.g.) Figure 3 The request received (as indicated by label 304) concerns the atomic addition operation performed on the relevant thread addresses (this request is configured, for example, by software code and then sent to execution unit 304 after hardware decoding). In some embodiments, multiple (e.g., 32) thread addresses are sent to execution unit 304 along with the atomic addition operation request, so that the storage cache 302 to be loaded performs an atomic addition operation on the relevant thread addresses with a predetermined value (predetermined to be, for example, "1").

[0039] Regarding thread requests, these are, for example, execution units in computing device 100 (such as...). Figure 3 (As shown by reference numeral 304) Based on the atomic addition operation request, several thread requests are determined to add a predetermined value to different addresses. It should be understood that multiple thread requests may exist, for example, thread requests to add a predetermined value to the same address. In some embodiments, execution unit 304 determines multiple thread requests to add a predetermined value to the same address based on the address pointed to by each thread in the atomic addition operation request.

[0040] Regarding the computing device 100, which includes at least an artificial intelligence chip 300, it should be understood that the computing device 100 may also include other components. For example... Figure 3 As shown, the artificial intelligence chip 300 includes at least a computing core, which further includes: a load memory cache 302, an execution unit 304, and a thread register 306. The load memory cache 302 further includes: a load memory cache front-end 308, a load memory cache controller 310, and a load memory cache back-end 312.

[0041] Regarding the method for responding to receiving an atomic addition operation request and determining multiple thread requests to add a predetermined value to the same address, for example, the execution unit 304 determines, based on the atomic addition request operation, whether multiple thread requests to add a predetermined value to the same address need to return a value, and sends the corresponding multiple thread requests to the address calculation module 314 included in the front end 308 of the loading storage cache according to the determination result.

[0042] For example, in some embodiments, the computing device 100 responds to the load memory cache 302 to confirm that the received atomic addition operation request is based on a first type of atomic addition instruction or a second type of atomic addition instruction (such as...). Figure 3As shown in label 324, the address calculation module 314 included in the front-end 308 of the loading storage cache determines multiple thread requests to add a predetermined value (the predetermined value is, for example, "1") to the same address. Atomic addition requests based on a first type of atomic addition instruction require a return value, while atomic addition requests based on a second type of atomic addition instruction do not. Further, the address calculation module 314 sends the thread requests based on either the first or second type of atomic addition instruction to the address merging module 316, as shown in label 326.

[0043] Regarding the return value, it indicates, for example, the original data before each thread performs the atomic addition operation when multiple threads perform an addition operation on the same address. For instance, threads 0, 4, 5, 10, and 12 point to the same address, where the original data at the target address is 2. Therefore, thread 0's return value is 2. After thread 0 performs the atomic addition operation, the target address performs an atomic addition operation of 2+1 on the original data 2, resulting in thread 0's atomic addition result 3, which is then written to static random access memory (SRAM). Similarly, thread 4's return value is 3. After thread 4 performs the atomic addition operation, the atomic addition operation 3+1 yields thread 4's atomic addition result 4, which is also written to SRAM. And so on, with threads 0, 4, 5, 10, and 12 returning 2, 3, 4, 5, and 6 respectively.

[0044] At step 204, the computing device 100 merges the multiple thread requests into a single request.

[0045] Regarding load caches, such as load caches in the compute core (e.g.) Figure 3 (As shown by reference numeral 302). The load cache includes at least: a load cache front-end 308, a load cache controller 310, and a load cache back-end 312. The load cache front-end 308 may further include, for example, an address calculation module 314 and an address merging module 316.

[0046] In some embodiments, after receiving multiple thread requests from the execution unit 304, the address calculation module 314 in the front-end of the load memory cache performs address calculation based on the multiple thread requests and sends the address calculation result (i.e., determining multiple thread requests with a predetermined value added to the same address) to the address merging module 316 (as indicated by label 326), so that the address merging module 316 can merge requests with the same address and addresses that do not have bank conflicts into one request, so as to send the merged request to the static random access memory controller 318 of the load memory cache controller 310.

[0047] For example, threads 0, 4, 5, 10, and 12 are determined by the address calculation module 314 to be multiple threads pointing to the same address, and the thread requests of threads 0, 4, 5, 10, and 12 are sent to the address merging module 316 to merge multiple thread requests into one thread request based on the merged address.

[0048] At step 206, the computing device 100 performs a summation operation in a single-cycle manner based on the merged request in order to generate the calculation result of the summation operation.

[0049] For example, in some embodiments, a method for performing a summation operation in a single-cycle manner based on a merged request to generate a result of the summation operation includes, for example, reading an initial value associated with the same address from a static random access memory to perform the summation operation using an atomic arithmetic logic unit; and writing the result of the summation operation into the static random access memory.

[0050] Regarding the single-cycle summation operation, for example, it involves directly accumulating the merged threads with the same address using an atomic addition calculation within one operation cycle to obtain the cumulative value of the atomic addition of multiple thread addresses. In some embodiments, multiple threads with the same address are, for example, threads 0, 4, 5, 10, and 12. After the threads request merging, a +5 operation is performed on the target address at once. For example, if the original data of the target address is 2, then based on the merged request, the calculation result of the single-cycle summation operation is 7.

[0051] In some embodiments, the merged request enters the static random-access memory controller 318, and the old value (or "original value") of the target address stored in the static random-access memory (SRAM) 320 is read by the static random-access memory controller 318. Based on the old value of the target address read, the atomic arithmetic and logic unit (Atomic ALU) is used to perform a summation calculation, and the calculation result is written into the static random-access memory 320.

[0052] Regarding atomic arithmetic logic units, such as the atomic arithmetic logic units included in the load-memory cache controller 310 in computing device 100 (e.g. Figure 3 (As shown by label 322), the merged request is provided to the static random access memory controller 318 via the address merging module 316 (e.g., ...). Figure 3 (As shown by label 328), under the control of static random access memory controller 318 (e.g.) Figure 3(As shown in label 330), the atomic arithmetic logic unit 322 performs an atomic summation operation based on the old value of the target address stored in the static random access memory 320.

[0053] At step 208, the computing device 100 determines the respective offset values ​​of multiple threads pointing to the same address, in order to generate return values ​​for multiple thread requests based on the initial values ​​in the static random access memory and the respective offset values ​​of the multiple threads.

[0054] In some embodiments, regarding the address calculation module (such as...) Figure 3 (As shown in label 314) In a method for determining multiple threads having the same address, it further includes, for example, recording the offset value for each thread having the same address.

[0055] For example, for threads 0, 4, 5, 10, and 12, the address calculation module 314 additionally records the offset value of each thread, such as offset0, offset4, offset5, offset10, and offset12, respectively. The offset value of each thread is transmitted to the static random access memory controller 318 via the address merging module 316, along with the merged address. Under the control of the static random access memory controller 318, the atomic arithmetic logic unit 322 performs addition calculations based on the old value (or "original value") of the target address read from the static random access memory 320 and the offset value of each thread to generate the return value requested by multiple threads.

[0056] In some embodiments, a method for providing the return values ​​of the generated multiple thread requests to a thread register includes, for example, providing the return values ​​of the multiple thread requests generated by the atomic arithmetic logic unit 322 to the thread register 306 via the back end 312 of the load storage cache.

[0057] In the above scheme, multiple thread requests for adding a predetermined value to the same address are identified based on the received atomic addition operation requests. These multiple thread requests are merged into one request, and the addition operation is performed in a single cycle based on the merged request to generate the calculation result of the addition operation. Therefore, this invention avoids the problem of multiple threads at the same address needing to occupy multiple cycles to perform addition calculations separately by merging multiple thread requests for adding a predetermined value to the same address into one request and performing the addition operation in a single cycle. Thus, this invention can significantly reduce the calculation time of atomic addition operations for multiple threads at the same address, improving computational efficiency. Furthermore, this invention also determines the corresponding offset values ​​of the multiple threads pointing to the same address, which are used to generate the return values ​​of the multiple thread requests based on the initial value in the static random access memory and the corresponding offset values ​​of the multiple threads. Therefore, this invention not only significantly improves the execution efficiency of artificial intelligence chips in processing atomic addition operations but also further optimizes the performance of atomic addition operations.

[0058] The following combination Figure 3 Provide a detailed description of the structure of the AI ​​chip 300. For example... Figure 3 As shown, the artificial intelligence chip 300 includes, for example, a computing core. The computing core includes, for example, an execution unit 304, a load-memory cache 302, and a thread register 306. The load-memory cache 302 includes, for example, a load-memory cache front-end 308, a load-memory cache controller 310, and a load-memory cache back-end 312. The load-memory cache controller 310 includes, for example, a static random access memory controller 318, a static random access memory 320, and an atomic arithmetic logic unit 322.

[0059] Regarding execution unit 304, it is configured to issue an atomic addition operation request to load memory cache 302. For example, execution unit 304 issues an atomic addition operation request to address calculation module 314 of front-end 308 of load memory cache. In some embodiments, the atomic addition operation request is based on a first type of atomic addition instruction or a second type of atomic addition instruction, wherein the atomic addition operation request based on the first type of atomic addition instruction requires a return value, and the atomic addition operation request based on the second type of atomic addition instruction does not require a return value.

[0060] Regarding the front end 308 of the loading storage cache, it is configured to, in response to receiving an atomic addition operation request, identify multiple thread requests to add a predetermined value to the same address; and merge the multiple thread requests into a single request.

[0061] In some embodiments, the front end 308 of the load cache includes, for example, an address calculation module 314 and an address merging module 316. The address calculation module 314 is configured to, in response to receiving an atomic addition operation request, determine multiple thread requests to add a predetermined value to the same address, and pass the determined thread requests for the same address to the address merging module 316. The address merging module 316 is configured, for example, to merge the multiple thread requests into a single request; and to provide the merged request to the static random access memory controller 318 of the load cache controller 310.

[0062] Regarding the load storage cache controller 310, it is configured to: perform a summation operation in a single cycle through the atomic arithmetic logic unit 322 based on the received merged request, so as to generate the calculation result of the summation operation; and determine the respective offset values ​​of multiple threads pointing to the same address, so as to generate the return value of the multiple thread requests based on the initial value in the static random access memory 320 and the respective offset values ​​of the multiple threads.

[0063] In some embodiments, the atomic arithmetic logic unit 322 is configured to perform an addition operation in a single-cycle manner to generate a result of the addition operation; and to add the respective offset values ​​of multiple threads (as indicated by label 332) to the initial values ​​associated with the same address read from the static random access memory 320 to generate return values ​​requested by multiple threads; and to provide the generated return values ​​of multiple thread requests to the back end 312 of the load memory cache (as indicated by label 334); and then to the thread register 306 (as indicated by label 336) via the back end 312 of the load memory cache.

[0064] In the above scheme, by enabling the address calculation module of the artificial intelligence chip to respond to the received atomic addition operation request, multiple thread requests for adding a predetermined value to the same address are identified. The address merging module merges the multiple thread requests into one request. The merged request is then provided to the static random access memory controller of the loading storage cache controller. The atomic arithmetic logic unit performs the addition operation in a single cycle to generate the calculation result of the addition operation. The initial value related to the same address in the read static random access memory is then added to the corresponding offset values ​​of the multiple threads to generate the return values ​​of the multiple thread requests. Finally, the generated return values ​​of the multiple thread requests are provided to the thread register. This invention enables the atomic arithmetic logic unit to simultaneously perform a single-cycle addition operation and return value calculation on the merged thread requests for the same address. This allows multiple threads adding a predetermined value to complete the addition calculation of multiple thread requests within one cycle, effectively reducing the computation time of multiple threads at the same address. Furthermore, the return value can be used for further deep optimization of the current operator. Therefore, the execution efficiency of the atomic addition operation of the operator is significantly improved.

[0065] As described above, the computing device 100 can also determine the respective offset values ​​of multiple threads pointing to the same address, for use in generating return values ​​requested by multiple threads based on the initial values ​​in the static random access memory and the respective offset values ​​of the multiple threads. Therefore, the method 200 may, for example, also include method 400 for generating return values ​​requested by multiple threads. The following will be combined with... Figure 3 and Figure 4 This describes a method 400 for generating return values ​​for multiple thread requests, as described in an embodiment of the present invention. It should be understood that method 400 can, for example, be used in... Figure 1 The described computing device 100 performs the operation. Method 400 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0066] At step 402, the load memory cache of the computing device 100 determines the respective offset values ​​of multiple threads pointing to the same address in response to determining that the received atomic addition operation request is based on a first type atomic addition instruction or that the atomic addition operation request requires a return value.

[0067] Regarding the first type of atomic add instruction, it indicates, for example, that the load store buffer requires a return value for the received atomic add operation request and provides the calculated return value to the thread register.

[0068] In some embodiments, the address calculation module 314 of the front-end 308 of the load memory cache judges the received atomic addition operation request. If the received atomic addition operation request indicates that the atomic addition operation request needs to return a value, the address calculation module 314 determines the corresponding offset values ​​of multiple threads pointing to the same address. In some embodiments, the address calculation module 314 passes the generated offset value to the address merging module 316, and then passes it to the static random access memory controller 318 of the load memory cache controller 310 via a flag 328.

[0069] At step 404, the load memory cache of the computing device 100 uses atomic arithmetic logic units to add the respective offset values ​​of multiple threads to the initial values ​​associated with the same address in the read static random access memory, so as to generate the return values ​​requested by multiple threads.

[0070] In some embodiments, the static random access memory controller 318 controls the reading of the initial value of the target address stored in the static random access memory 320, and performs the following calculation through the atomic arithmetic logic unit 322: the atomic arithmetic logic unit 322 adds the respective offset values ​​of multiple threads to the initial value of the same address in the static random access memory 320 that has been read, so as to generate the return value requested by multiple threads.

[0071] At step 406, the load memory cache of the computing device 100 provides the return values ​​of the generated multiple thread requests to the thread register.

[0072] In some embodiments, the return values ​​of multiple thread requests generated by the atomic arithmetic logic unit are provided to the thread register via the backend of the load storage cache.

[0073] For example, such as Figure 3 As shown, the atomic arithmetic logic unit 322 of the load storage cache 302 passes the calculated return values ​​of multiple thread requests to the backend 312 of the load storage cache, which provides the values ​​to the thread register 306 via label 336.

[0074] In the above scheme, the front end 308 of the load cache determines that the received atomic addition operation request is based on a first type of atomic addition instruction or that the atomic addition operation request requires a return value. It then determines the corresponding offset values ​​for multiple threads pointing to the same address and adds the corresponding offset values ​​for each thread to the initial values ​​related to the same address read from the static random access memory by the load cache controller 310, thereby generating return values ​​for multiple thread requests. Finally, the back end 312 of the load cache provides the generated return values ​​to the thread register 306. This invention, while performing the addition operation on the merge request in a single cycle, can also generate return values ​​for multiple thread requests using offset values ​​for further in-depth optimization of the operator addition operation.

[0075] As mentioned above, the computing device 100 can also determine whether the atomic addition operations included in the current operator require a return value. Therefore, the method 200 may, for example, also include a method 500 for determining whether a return value is required. The following will be combined with... Figure 3 and Figure 5 This describes a method 500 for confirming whether a return value is needed, as described in an embodiment of the present invention. It should be understood that method 500 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 500 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0076] At step 502, the execution unit of the computing device 100 confirms whether the atomic addition operation included in the current operator requires a return value.

[0077] In some embodiments, the execution unit 304 of the computing device 100 determines whether the atomic addition operation included in the current operator needs a return value by hardware decoding the incoming code to be executed. Whether a return value is needed can be determined by configuring the corresponding parameters in the code to be executed.

[0078] At step 504, in response to confirming that the atomic addition operation included in the current operator requires a return value, the execution unit of the computing device 100 provides an atomic addition operation request based on the first type of atomic addition instruction to the front end of the loading memory cache.

[0079] For example, if the code to be executed passed to execution unit 304 is "pos = T. atpopc(s_num_input[0], 1,return_prev=True)", then after hardware decoding, execution unit 304 obtains that an atomic addition operation is performed on the variable s_num_input[0], its value is incremented by 1, and the old value before the atomic addition operation is performed is returned and assigned to the variable pos. It should be understood that when the variable return_prev is configured to True, execution unit 304 confirms that the atomic addition operation included in the current operator needs to return a value, and provides the atomic addition operation request based on the first type of atomic addition instruction to the front end 308 of the loading memory cache.

[0080] At step 506, in response to confirming that the atomic addition operation included in the current operator does not require a return value, the execution unit of the computing device 100 provides an atomic addition operation request based on the second type of atomic addition instruction to the front end of the loading memory cache.

[0081] For example, if the code to be executed passed to execution unit 304 is "pos = T. atpopc(s_num_input[0],1)", then after hardware decoding, execution unit 304 obtains that an atomic addition operation is performed on the variable s_num_input[0], incrementing its value by 1, and assigning the calculation result of the atomic addition operation to the variable pos. It should be understood that execution unit 304 confirms that the current operator has not configured a return value variable. Therefore, execution unit 304 confirms that the atomic addition operation included in the current operator does not need a return value, and provides the atomic addition operation request based on the second type of atomic addition instruction to the front end 308 of the loading memory cache.

[0082] In the above scheme, the execution unit in the computing device 100 provides an atomic addition operation request based on a first type of atomic addition instruction to the front end of the loading memory cache by confirming whether the atomic addition operation included in the current operator needs a return value, or provides an atomic addition operation request based on a second type of atomic addition instruction to the front end of the loading memory cache. Therefore, the present invention can realize flexible configuration of whether to calculate a return value.

[0083] The various processes and procedures described above, such as methods 200, 400, and 500, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, methods 200, 400, and 500 may be implemented as a computer software program or program product tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200, 400, and 500 described above can be performed.

[0084] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0085] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0086] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0088] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps loaded in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0089] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.

Claims

1. A method for performing atomic addition operations, characterized in that, The method includes: In response to receiving an atomic addition operation request, determine multiple thread requests to add a predetermined value to the same address; Merge the multiple thread requests into a single request; Based on the merged request, the summation operation is performed in a single cycle to generate the calculation result of the summation operation; and Determine the respective offset values ​​of multiple threads pointing to the same address, so as to generate the return values ​​of multiple thread requests based on the initial values ​​in the static random access memory and the respective offset values ​​of the multiple threads.

2. The method according to claim 1, characterized in that, In response to receiving an atomic addition operation request, it is determined that multiple thread requests to add a predetermined value to the same address include: In response to the load cache acknowledging that the received atomic addition operation request is based on either the first type of atomic addition instruction or the second type of atomic addition instruction, the address calculation module included in the front end of the load cache determines multiple thread requests to add a predetermined value to the same address. Among them, the atomic addition operation request based on the first type of atomic addition instruction needs to return a value, while the atomic addition operation request based on the second type of atomic addition instruction does not need to return a value.

3. The method according to claim 1, characterized in that, Merging the multiple thread requests into a single request includes: The multiple thread requests are merged into a single request via the address merging module at the front end of the loading storage cache; and The merged request is provided to the static random access memory controller that loads the storage cache controller.

4. The method according to claim 3, characterized in that, Based on the merged request, the summation operation is performed in a single cycle to generate the calculation result of the summation operation, including: Read the initial value associated with the same address from the static random access memory so that an addition operation can be performed using the atomic arithmetic logic unit; and The result of the summation operation is written into static random access memory.

5. The method according to claim 2, characterized in that, Determine the respective offset values ​​for multiple threads pointing to the same address, so as to generate the return values ​​for multiple thread requests based on the initial values ​​in the static random access memory and the respective offset values ​​of the multiple threads, including: In response to determining that the received atomic addition operation request is based on a first type atomic addition instruction or that the atomic addition operation request requires a return value, determine the corresponding offset values ​​for multiple threads pointing to the same address. Using atomic arithmetic logic units, the initial values ​​associated with the same address in the read static random access memory are supplemented with the respective offset values ​​of multiple threads to generate return values ​​requested by multiple threads; and Provide the return values ​​of the generated multiple thread requests to the thread register.

6. The method according to claim 5, characterized in that, The return values ​​of the generated multiple thread requests are provided to the thread register, including: The return values ​​of multiple thread requests generated by the atomic arithmetic logic unit are provided to the thread register via the backend of the load storage cache.

7. The method according to claim 2, characterized in that, The method further includes: The execution unit determines whether the atomic addition operations included in the current operator require a return value; In response to the confirmation that the atomic addition operation included in the current operator requires a return value, an atomic addition operation request based on the first type of atomic addition instruction is provided to the front end of the load memory cache; and In response to the confirmation that the atomic addition operation included in the current operator does not require a return value, an atomic addition operation request based on the second type of atomic addition instruction is provided to the front end of the loading storage cache.

8. An artificial intelligence chip, characterized in that, The artificial intelligence chip includes: The execution unit is configured to issue atomic addition operation requests to the load storage cache; Load the storage cache, including: The front-end for loading the storage cache is configured to, in response to receiving an atomic addition operation request, identify multiple thread requests to add a predetermined value to the same address; and merge the multiple thread requests into a single request; and The loading storage cache controller is configured to perform a summation operation in a single cycle based on the merged request to generate the calculation result of the summation operation; and to determine the respective offset values ​​of multiple threads pointing to the same address, so as to generate the return value of the multiple thread requests based on the initial value in the static random access memory and the respective offset values ​​of the multiple threads.

9. The artificial intelligence chip according to claim 8, characterized in that, The front end for loading the storage cache includes: The address calculation module is configured to, in response to receiving an atomic addition operation request, determine multiple thread requests to add a predetermined value to the same address; and The address merging module is configured to merge the multiple thread requests into a single request; and to provide the merged request to the static random access memory controller that loads the storage cache controller.

10. The artificial intelligence chip according to claim 8, characterized in that, The storage cache controller includes: The atomic arithmetic logic unit is configured at least to perform a summation operation in a single-cycle manner to generate the result of the summation operation; and to add the respective offset values ​​of multiple threads to the initial values ​​associated with the same address in the read static random access memory to generate the return values ​​requested by multiple threads; and to provide the generated return values ​​of multiple thread requests to the thread register.

11. The artificial intelligence chip according to claim 10, characterized in that, The atomic arithmetic logic unit is also configured as follows: In response to determining that the received atomic addition operation request is based on a second type atomic addition instruction or that the atomic addition operation request does not require a return value, the addition operation is performed in a single cycle based on the initial value in order to generate the calculation result of the addition operation; And write the result of the summation operation into static random access memory.

12. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 1-7.

14. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 1-7.

Images