Processing circuit and operating method for processing circuit

Abstract

The present invention provides a processing circuit and an operating method for the processing circuit. The processing circuit includes a shared memory. The shared memory performs a pipeline data access operation over multiple cycles. The shared memory includes a first memory block and a second memory block. The first memory block performs a first data access operation during a single cycle among the multiple cycles. The second memory block performs a second data access operation during the same single cycle. The first data access operation is different from the second data access operation. The processing circuit is capable of optimizing the data access operation.

Description

Technical Field

[0001] The present invention relates to an electronic circuit and an operating method for the electronic circuit, and in particular to a processing circuit capable of optimizing data access operations and an operating method for the processing circuit. Background Art

[0002] Typically, existing processing circuits, such as graphics processing units (GPUs), utilize load / store (LD / ST) units to transfer data. However, the bandwidth of LD / ST units is limited. Using only LD / ST units makes it difficult for processing circuits to fully utilize the bandwidth of network-on-chip (NoC). Therefore, optimizing data access operations within processing circuits has become a key research topic for those skilled in the art. Summary of the Invention

[0003] The present invention provides a processing circuit capable of optimizing data access operations and an operating method for the processing circuit.

[0004] According to an embodiment of the present invention, a processing circuit includes a shared memory. The shared memory performs a plurality of cycles of pipelined data access operations. The shared memory includes a first memory block and a second memory block. The first memory block performs a first data access operation during a single cycle among the plurality of cycles. The second memory block performs a second data access operation during the same single cycle. The first data access operation is different from the second data access operation.

[0005] According to an embodiment of the present invention, an operating method is applicable to a processing circuit. The processing circuit includes a shared memory. The operating method includes: dividing the shared memory into a plurality of memory banks; and performing a plurality of cycles of pipelined data access operations. The plurality of cycles of pipelined data access operations include: during a single cycle among the plurality of cycles, performing a first data access operation by a first memory bank among the plurality of memory banks; and during the same single cycle, performing a second data access operation by a second memory bank among the plurality of memory banks. The first data access operation is different from the second data access operation.

[0006] Based on the above, the processing circuit utilizes shared memory to participate in pipelined data access operations across multiple cycles. Furthermore, the first memory block and the second memory block each perform different data access operations during the same cycle. By leveraging the shared memory, the processing circuit can fully utilize the bandwidth of the network-on-chip (NoC). This optimizes the processing circuit's data access operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of a processing circuit according to an embodiment of the present invention.

[0008] Figure 2 FIG. 1 is a schematic diagram illustrating a pipeline data access operation according to an embodiment of the present invention.

[0009] Figure 3 is a schematic diagram of a processing circuit according to an embodiment of the present invention.

[0010] Figure 4 FIG. 1 is a schematic diagram illustrating data and status flags according to an embodiment of the present invention.

[0011] Figure 5 is a schematic diagram of a processing circuit according to an embodiment of the present invention.

[0012] Figure 6 is a flow chart illustrating an operating method according to an embodiment of the present invention.

[0013] Description of Reference Numerals

[0014] 100, 300, 400, 500: Processing circuit

[0015] 110, 210, 310, 410, 510: shared memory

[0016] 320, 420, 520: global memory

[0017] 330, 430, 530: Execution units

[0018] A1_1~A1_4:Address data

[0019] BF0, BF0A, BF0B, BF1, BF1A, BF1B, BF2, BF2A, BF2B: Buffers

[0020] BK0, BK1, BK2, BK3: storage blocks

[0021] D0_1 to D0_4, D0_1A to D0_4A, D1_1 to D1_4, D1_1B to D1_4B, D2_0A, D2_0B: data

[0022] DB0, DB1, DB2: data blocks

[0023] DE, DE1, DE2: operation data

[0024] FB1, FB2: Flag blocks

[0025] FG1_1~FG1_4: Status flag

[0026] OPAD_0: First data access operation

[0027] OPAD_1: Second data access operation

[0028] OPAD_2: Third data access operation

[0029] OPAD_3: Fourth data access operation

[0030] S100: How to operate

[0031] S110, S120, S121, S122: Steps DETAILED DESCRIPTION

[0032] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0033] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of a processing circuit according to an embodiment of the present invention. In this embodiment, the processing circuit 100 may be, for example, a graphics processing unit (GPU). The processing circuit 100 includes a shared memory 110. The shared memory 110 performs a plurality of loops of pipeline data access operations. The shared memory 110 includes at least memory blocks BK0 and BK1. During a single loop among the plurality of loops, the memory block BK0 performs a first data access operation OPAD_0. Furthermore, the memory block BK1 performs a second data access operation OPAD_1 during the same single loop. The first data access operation OPAD_0 is different from the second data access operation OPAD_1. In other words, the memory blocks BK0 and BK1 perform different data access operations during the same loop.

[0034] It's worth noting that processing circuit 100 utilizes shared memory 110 to participate in pipelined data access operations across multiple cycles. Furthermore, memory blocks BK0 and BK1 each perform different data access operations during the same single cycle. By utilizing the shared memory space and parallelizing data access operations, processing circuit 100 can fully utilize the bandwidth of the network-on-chip (NoC). This optimizes data access operations within processing circuit 100.

[0035] In this embodiment, during the next cycle, the memory blocks BK0 and BK1 also respectively perform different data access operations.

[0036] For ease of description, the shared memory 110 of the present invention is illustrated as two memory blocks BK0 and BK1. However, the present invention is not limited to the number of memory blocks. In some embodiments, the number of memory blocks can be greater than two.

[0037] In this embodiment, the first data access operation OPAD_0 and the second data access operation OPAD_1 are, respectively, different bulk copy operations, data input operations, and data output operations. For example, the first data access operation OPAD_0 involves performing a first bulk copy operation on the memory block BK0 and a memory in the processing circuit 100. The second data access operation OPAD_1 involves performing a second bulk copy operation on the memory block BK1 and another memory in the processing circuit 100.

[0038] Please refer to Figure 1 as well as Figure 2 , Figure 2 Figure 2 is a schematic diagram illustrating a pipeline data access operation according to an embodiment of the present invention. In this embodiment, shared memory 210 participates in multiple cycles of pipeline data access operations. Shared memory 210 includes memory blocks BK0-BK3. During a first cycle, memory blocks BK0-BK3 perform different data access operations. For example, during the first cycle, memory block BK0 performs a first data access operation OPAD_0 with the first memory in processing circuit 100. Memory block BK1 performs a second data access operation OPAD_1 with the second memory in processing circuit 100. Memory block BK2 performs a third data access operation OPAD_2 with the third memory in processing circuit 100. Memory block BK3 performs a fourth data access operation OPAD_3 with the fourth memory in processing circuit 100.

[0039] In this embodiment, during the first loop, the first data access operation OPAD_0 , the second data access operation OPAD_1 , the third data access operation OPAD_2 , and the fourth data access operation OPAD_3 may be performed simultaneously or sequentially.

[0040] In some embodiments, based on the readiness of the memory blocks BK0 - BK3 , at least one of the first data access operation OPAD_0 , the second data access operation OPAD_1 , the third data access operation OPAD_2 , and the fourth data access operation OPAD_3 may be delayed during the first cycle.

[0041] During the first cycle, at least one of the memory blocks BK0-BK3 may not perform a data access operation. For example, during the first cycle, memory block BK0 and the first memory in processing circuit 100 perform a first data access operation OPAD_0. Memory block BK2 and the third memory in processing circuit 100 perform a third data access operation OPAD_2. Memory blocks BK1 and BK3 do not perform data access operations, but the present invention is not limited to this.

[0042] During a second loop following the first loop, the first, second, third, and fourth data access operations OPAD_0, OPAD_1, OPAD_2, and OPAD_3 are performed again.

[0043] In this embodiment, the first data access operation OPAD_0 , the second data access operation OPAD_1 , the third data access operation OPAD_2 , and the fourth data access operation OPAD_3 may be one of a block copy operation, a data input operation, and a data output operation, respectively.

[0044] Based on the above, the shared memory 210 provides a parallel pipeline function.

[0045] Please refer to Figure 3 , Figure 3 FIG3 is a schematic diagram of a processing circuit according to an embodiment of the present invention. In this embodiment, processing circuit 300 includes a shared memory 310, a global memory 320, and an execution unit 330. Shared memory 310 performs pipelined data access operations over multiple cycles. Shared memory 310 includes at least memory blocks BK0-BK2. Global memory 320 is coupled to shared memory 310. During a single cycle, global memory 320 performs a first data access operation in conjunction with memory block BK0 and a second data access operation in conjunction with memory block BK1.

[0046] For example, the global memory 320 includes buffers BF0 and BF1. During a first data access operation, the global memory 320 transfers data stored in the buffer BF0 to the memory block BK0. During a second data access operation, the global memory 320 transfers data stored in the second buffer of the global memory to the memory block BK1.

[0047] In addition, during the same cycle, the global memory 320 and the memory block BK2 jointly perform a third data access operation. For example, the global memory 320 further includes a buffer BF2. The memory block BK3 and the buffer BF2 also perform a third data access operation during the same cycle.

[0048] For example, buffer BF0 of global memory 320 may be an input buffer. Buffer BF1 of global memory 320 may be a first-in, first-out (FIFO) buffer. Buffer BF2 of global memory 320 may be an output buffer. During a first cycle, when memory block BK0 is ready, processing circuit 300 inputs data D0_1 from buffer BF0 of global memory 320 to memory block BK0 of shared memory 310 (i.e., a first data access operation). When memory block BK1 is ready, processing circuit 300 inputs data D1_1 from buffer BF1 of global memory 320 to memory block BK1 of shared memory 310 (i.e., a second data access operation). Furthermore, during the first cycle, buffer BF2 of global memory 320 outputs data D2_0 stored during the previous cycle to buffer BF2 of global memory 320 (i.e., a third data access operation). The above operation is a block copy operation. Therefore, based on the above operations, the processing circuit 300 can utilize the shared memory 310 to implement the asynchronous block copy operation of the tensor memory accelerator (TMA) during the first cycle.

[0049] Continuing with the above example, after the first and second data access operations are completed, data D0_1 and D1_1 from memory blocks BK0 and BK1 of shared memory 310 are provided to execution unit 330. Therefore, execution unit 330 can perform arithmetic operations on data D0_1 and D1_1 to generate computational data DE. When memory block BK2 is ready, computational data DE is output to memory block BK2. For further example, data D0_1 and D1_1 from memory blocks BK0 and BK1 of shared memory 310 are stored in thread local registers (TLRs). Execution unit 330 receives data D0_1 and D1_1 from the TLRs. Execution unit 330 can perform arithmetic operations on data D0_1 and D1_1 to generate computational data DE and store the computational data DE in the TLRs.

[0050] Continuing with the above example, the operations of the thread local register and the execution unit 330 may be performed during the first cycle or between the first cycle and the second cycle.

[0051] Continuing with the above example, in the second cycle period following the first cycle period, the memory block BK2 outputs the operation data DE to the buffer BF2 of the global memory 320 .

[0052] In this embodiment, memory block BK0 includes data block DB0, memory block BK1 includes data block DB1 and flag block FB1, and memory block BK2 includes data block DB2 and flag block FB2.

[0053] Taking the data block DB1 as an example, the data block DB1 is used to store data D1_1. The flag block FB1 is used to store a status flag corresponding to the data D1_1.

[0054] Please refer to Figure 3 as well as Figure 4 , Figure 4 FIG. 1 is a schematic diagram illustrating data and status flags according to an embodiment of the present invention. Figure 4 Data D0_1 to D0_4 and D1_1 to D1_4 are shown. Data D1_1 includes status flag FG1_1. Data D1_2 includes status flag FG1_2. Data D1_3 includes status flag FG1_3. Data D1_4 includes status flag FG1_4. For example, data D0_1 to D0_4 each have 496 bytes, but the present invention is not limited to this.

[0055] Taking data D1_1 as an example, data D1_1 has a first number of bytes. Status flag FG1_1 has a second number of bytes. The first number of bytes is greater than or equal to 32 times the second number of bytes. For example, data D1_1 has 512 bytes (i.e., the first number of bytes), but the present invention is not limited to this. Status flag FG1_1 has 8 bytes (i.e., the second number of bytes), but the present invention is not limited to this.

[0056] For example, data D0_1 and data D1_1 can be used for thread operations during the first cycle, and data D0_2 and data D1_2 can be used for thread operations during the second cycle. Therefore, data D0_1-D0_4 and D1_1-D1_4 can be used for warp operations.

[0057] In this embodiment, when the status flag FG1_1 having the second number of bytes is written into the flag block FB1, it indicates that the transmission of the data D1_1 has been completed. Therefore, the processing circuit 300 determines that the second data access operation has been completed.

[0058] In addition, the data D1_1 also includes address data A1_1. The address data A1_1 has 8 bytes, but the present invention is not limited thereto. The address data is used for address alignment in the second data access operation.

[0059] It should be noted that the first number of bytes can be increased while the second number of bytes remains unchanged. Thus, the granularity of warp operations can also be increased. For example, the first number of bytes can be 1024 bytes or 2048 bytes, but the present invention is not limited thereto.

[0060] Please refer to Figure 5 , Figure 5 is a schematic diagram of a processing circuit according to an embodiment of the present invention. In this embodiment, Figure 5 Processing circuits 400 and 500 are shown. Processing circuit 400 includes shared memory 410, global memory 420, and execution unit 430. Processing circuit 500 includes shared memory 510, global memory 520, and execution unit 530. In this embodiment, during a pipelined data access operation, processing circuit 400 and processing circuit 500 perform data access operations.

[0061] In this embodiment, the shared memory 410 includes memory blocks BK0A-BK2A. The global memory 420 includes buffers BF0A-BF2A. The shared memory 510 includes memory blocks BK0B-BK2B. The global memory 520 includes buffers BF0B-BF2B.

[0062] During the first cycle, when memory block BK0A is ready, processing circuit 400 inputs data D0_1A in buffer BF0A to memory block BK0A (i.e., a first data access operation to shared memory 410). When memory block BK1A is ready, processing circuit 400 inputs data D1_1A in buffer BF1A to memory block BK1A (i.e., a second data access operation to shared memory 410). Furthermore, during the first cycle, buffer BF2A outputs data D2_0A stored during the previous cycle to buffer BF2A (i.e., a third data access operation to shared memory 410). The above operation is a block copy operation. Therefore, based on the above operation, processing circuit 400 can implement an asynchronous block copy operation of the tensor memory accelerator using shared memory 410 during the first cycle.

[0063] During the first cycle, when memory block BK0B is ready, processing circuit 500 inputs data D0_1B in buffer BF0B to memory block BK0B (i.e., a first data access operation to shared memory 510). When memory block BK1B is ready, processing circuit 500 inputs data D1_1B in buffer BF1B to memory block BK1B (i.e., a second data access operation to shared memory 510). Furthermore, during the first cycle, buffer BF2B outputs data D2_0B stored during the previous cycle to buffer BF2B (i.e., a third data access operation to shared memory 510). The above operation is a block copy operation. Therefore, based on the above operation, processing circuit 500 can implement an asynchronous block copy operation of the tensor memory accelerator using shared memory 510 during the first cycle.

[0064] After the first and second data access operations are completed on shared memory 410, data D0_1A and D1_1A from memory blocks BK0A and BK1A are provided to execution unit 330. Therefore, execution unit 430 performs arithmetic operations on data D0_1A and D1_1A to generate computational data DE1. When memory block BK2A is ready, computational data DE1 is output to memory block BK2A.

[0065] Similar to Figure 3 In this embodiment, data D0_1A and D1_1A from memory blocks BK0A and BK1A of shared memory 410 are stored in thread-local registers. The execution unit 430 receives data D0_1A and D1_1A from the thread-local registers. The execution unit 430 performs an operation on the data D0_1A and D1_1A to generate operation data DE1, and stores the operation data DE1 in the thread-local registers.

[0066] After the first and second data access operations are completed on the shared memory 510, the data D0_1B and D1_1B from the memory blocks BK0B and BK1B are provided to the execution unit 330. Therefore, the execution unit 530 can perform a computation on the data D0_1B and D1_1B to generate computation data DE2. When the memory block BK2B is ready, the computation data DE2 is output to the memory block BK2B.

[0067] During a second cycle following the first cycle, storage block BK2A outputs computation data DE1 to buffer BF2A. It should be noted that during the second cycle, storage block BK2A performs data access operations with global memory 520, but the present invention is not limited thereto. For example, buffers BF0A and BF0B can each be an input buffer. Buffers BF1A and BF1B can each be a first-in, first-out buffer. Buffers BF2A and BF2B can each be an output buffer. Storage block BK2A outputs computation data DE1 to buffer BF2A and also outputs computation data DE1′ to buffer BF1B. The format of computation data DE1′ is different from that of computation data DE1. For example, compared to computation data DE1, computation data DE1′ includes at least a status flag.

[0068] Furthermore, during a second cycle period following the first cycle period, the memory block BK2B outputs the operation data DE2 to the buffer BF2B.

[0069] Please refer to Figure 1 as well as Figure 6 , Figure 6 is a flow chart illustrating an operating method according to one embodiment of the present invention. In this embodiment, operating method S100 is applicable to processing circuit 100. Operating method S100 includes steps S110 and S120. In step S110, shared memory 110 of processing circuit 100 is divided into a plurality of memory blocks. In step S120, shared memory 110 utilizes the plurality of memory blocks to perform a plurality of cycles of pipelined data access operations. Step S120 includes at least steps S121 and S122.

[0070] The plurality of memory blocks include at least memory blocks BK0 and BK1. In step S121, memory block BK0 (i.e., the first memory block) performs a first data access operation OPAD_0 during a single cycle. In step S122, memory block BK1 (i.e., the second memory block) performs a second data access operation OPAD_1 during the same cycle. The first data access operation OPAD_0 is different from the second data access operation OPAD_1.

[0071] During the next cycle, the memory blocks BK0 and BK1 also perform different data access operations respectively.

[0072] The implementation details of steps S110 and S120 are already in Figures 1 to 4 The above is clearly described in the embodiments, so it will not be repeated here.

[0073] In summary, the processing circuit utilizes shared memory to participate in pipelined data access operations across multiple cycles. The shared memory includes at least a first memory block and a second memory block. The first memory block and the second memory block each perform different data access operations during the same cycle. By utilizing the shared memory space, the processing circuit can fully utilize the bandwidth of the on-chip network. This optimizes the processing circuit's data access operations. The processing circuit utilizes the shared memory to implement parallel pipelined data access operations and asynchronous block copy operations for the tensor memory accelerator.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or perform equivalent replacements on some or all of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A processing circuit, characterized in that: The processing circuit comprises: A shared memory configured to perform a plurality of cycles of pipelined data access operations, wherein the shared memory comprises: a first memory block configured to perform a first data access operation during a single cycle among the plurality of cycles; and a second memory block configured to perform a second data access operation during the same single cycle, a third memory block configured to perform a third data access operation during the same single cycle; an execution unit, coupled to the shared memory, configured to receive first data from the first storage block and second data from the second storage block, perform an operation on the first data and the second data to generate operation data, and provide the operation data to the third storage block; The first data access operation is different from the second data access operation.

2. The processing circuit according to claim 1, wherein: The processing circuit further includes: The global memory is coupled to the shared memory and configured to perform the first data access operation together with the first memory block, perform the second data access operation together with the second memory block, and perform the third data access operation together with the third memory block.

3. The processing circuit according to claim 2, wherein: In the first data access operation, the global memory transfers the data stored in the first buffer of the global memory to the first storage block, and In the second data access operation, the global memory transfers the data stored in the second buffer of the global memory to the second memory block.

4. The processing circuit according to claim 3, wherein: The first buffer is an input buffer, and The second buffer is a first-in-first-out buffer.

5. The processing circuit according to claim 3, wherein: The third storage block is configured to perform the third data access operation with a third buffer of the global memory.

6. The processing circuit according to claim 5, characterized in that The third buffer is an output buffer.

7. The processing circuit according to claim 5, characterized in that The third storage block performs a fourth data access operation with the global memory of other processing circuits.

8. The processing circuit according to claim 2, wherein: When the second storage block is written with a first number of bytes of data, a status flag having a second number of bytes is written into the flag block of the second storage block. The first number of bytes is greater than or equal to 32 times the second number of bytes, and When the status flag having a second number of bytes is written into the flag block, the processing circuit determines that the second data access operation is completed.

9. A method for operating a processing circuit, characterized in that The processing circuit includes a shared memory and an execution unit, wherein the operating method includes: Dividing the shared memory into a plurality of blocks; and Perform pipelined data access operations over multiple cycles, The pipeline data access operations of executing the plurality of cycles include: During a single cycle among the plurality of cycles, a first data access operation is performed by a first memory block among the plurality of blocks; and During the same single cycle, performing a second data access operation by a second memory block among the plurality of blocks; During the same single cycle, performing a third data access operation by a third memory block among the plurality of blocks; The operating method further includes: receiving, by the execution unit, first data from the first storage block and second data from the second storage block, performing an operation on the first data and the second data to generate an operation result, and providing the operation result to the third storage block; The first data access operation is different from the second data access operation.

10. The operating method according to claim 9, characterized in that: The processing circuit further comprises a global memory, wherein the step of performing the first data access operation comprises: The first data access operation is performed jointly by the global memory and the first storage block.

11. The operating method according to claim 10, characterized in that: The step of jointly performing the first data access operation by the global memory and the first storage block includes: The data stored in the first buffer of the global memory is transferred from the global memory to the first storage block.

12. The operating method according to claim 10, characterized in that: The step of performing the second data access operation includes: The second data access operation is performed jointly by the global memory and the second storage block.

13. The operating method according to claim 12, characterized in that: The step of jointly performing the second data access operation by the global memory and the second storage block includes: The global memory transfers the data stored in the second buffer of the global memory to the second storage block.

14. The operating method according to claim 13, characterized in that: The third data access operation is performed by the third storage block and a third buffer of the global memory.

15. The operating method according to claim 14, characterized in that: Also includes: A fourth data access operation is performed by the third storage block and the global memory of other processing circuits.

16. The operating method according to claim 10, characterized in that: When the second storage block is written with a first number of bytes of data, a status flag having a second number of bytes is written into the flag block of the second storage block. The first number of bytes is greater than or equal to 32 times the second number of bytes, and The operating method further includes: When the status flag having a second number of bytes is written into the flag block, it is determined that the second data access operation is completed.

Images