Artificial intelligence chip and cache operation method thereof

Abstract

The present application provides an artificial intelligence chip and a cache operation method thereof. The artificial intelligence chip includes a cache, a plurality of computing cores, and a scratchpad allocation-release unit. The cache dynamically allocates its storage space into a cache mode space and a scratchpad mode space. At least one of the plurality of computing cores issues a cache mode access request to the cache for data access to the cache mode space. The scratchpad allocation-release unit performs data access to the scratchpad mode space of the cache based on a scratchpad mode access request issued by at least one of the plurality of computing cores. In response to the scratchpad mode space being insufficient to serve the scratchpad mode access request, the scratchpad allocation-release unit generates a scratchpad allocation request to the cache to dynamically increase the scratchpad mode space.

Description

TECHNICAL FIELD

[0001] The present application relates to an electronic circuit, and in particular, to an Artificial Intelligence (AI) chip and a cache operation method thereof. BACKGROUND

[0002] Static Random Access Memory (SRAM) in a processor is generally statically configured as a cache mode and a scratchpad mode to meet the memory requirements of different applications. That is, the configuration of the cache mode and the scratchpad mode will not be changed during the use of the application program. However, for actual applications, the configuration requirements of the cache mode and the scratchpad mode are dynamic. The static allocation method is difficult to achieve a good configuration balance. How to dynamically modify the storage space configuration of different modes during the application process is one of the technical issues in the field of processors. SUMMARY

[0003] The present application is directed to an Artificial Intelligence chip and a cache operation method thereof to dynamically modify the allocation of the cache mode space and the scratchpad mode space during the application process.

[0004] In embodiments according to the present application, the Artificial Intelligence chip includes a cache, a plurality of compute cores, and a scratchpad allocation-release unit. The cache dynamically allocates storage space of the cache as a cache mode space and a scratchpad mode space. The plurality of compute cores are coupled to the cache. At least one of the plurality of compute cores issues a cache mode access request to the cache for data access to the cache mode space. The scratchpad allocation-release unit is coupled to the plurality of compute cores and the cache. The scratchpad allocation-release unit performs data access to the scratchpad mode space of the cache based on a scratchpad mode access request issued by at least one of the plurality of compute cores. In response to the scratchpad mode space being insufficient to service the scratchpad mode access request, the scratchpad allocation-release unit generates a scratchpad allocation request to the cache to dynamically increase the scratchpad mode space.

[0005] In an embodiment according to the present application, the cache operation method comprises: dynamically allocating, by a cache of the artificial intelligence chip, a storage space of the cache into a cache mode space and a scratchpad mode space; issuing, by at least one of the plurality of computing cores, a cache mode access request to the cache for data access to the cache mode space; performing, by a scratchpad allocation-release unit, data access to the scratchpad mode space of the cache based on a scratchpad mode access request issued by at least one of the plurality of computing cores; and in response to the scratchpad mode space being insufficient to serve the scratchpad mode access request, generating, by the scratchpad allocation-release unit, a scratchpad allocation request to the cache to dynamically increase the scratchpad mode space.

[0006] Based on the above, the artificial intelligence chip is arranged with a scratchpad allocation-release unit. The scratchpad allocation-release unit can execute scratchpad mode access requests of the computing cores. That is, the computing cores running the application program can perform data access to the scratchpad mode space of the cache through the scratchpad allocation-release unit. During use of the application program, in response to the scratchpad mode space being insufficient to serve the scratchpad mode access request, the scratchpad allocation-release unit can generate a scratchpad allocation request to the cache to dynamically increase the scratchpad mode space. That is, the artificial intelligence chip can dynamically modify the allocation of the cache mode space and the scratchpad mode space during the application process. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is a circuit block schematic diagram of an artificial intelligence chip according to an embodiment of the present application;

[0008] Figure 2 is a flowchart schematic diagram of a cache operation method of an artificial intelligence chip according to an embodiment of the present application;

[0009] Figure 3 is a circuit block schematic diagram of a cache according to an embodiment of the present application.

[0010] BRIEF DESCRIPTION OF DRAWINGS

[0011] 100, an artificial intelligence chip,

[0012] 110, a computing core,

[0013] 120, a cache,

[0014] 121, a cache line allocation-release unit,

[0015] 122, a hit-miss check unit,

[0016] 123, a tag memory,

[0017] 124, an operation engine,

[0018] 125. memory controller,

[0019] 126. cache line array,

[0020] 130. main memory,

[0021] 140. scratchpad allocation-deallocation unit. DETAILED DESCRIPTION

[0022] Reference will now be made in detail to the exemplary embodiments of the present application, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used in the different drawings and the same or similar reference numbers will be used for the same or similar parts.

[0023] The term "coupled" or "connected" used in the entire specification (including claims) can refer to any direct or indirect connection. For example, if a first device is coupled or connected to a second device, it should be interpreted that the first device can be directly connected to the second device, or the first device can be indirectly connected to the second device through other devices or some connection means. The terms "first", "second", and the like used in the entire specification (including claims) are used to name components, or to distinguish different embodiments or ranges, and are not intended to limit the upper or lower limit of the number of components, or the order of components. In addition, wherever possible, the same reference numbers are used for the same or similar components, members, steps in the drawings and embodiments. The components, members, steps using the same reference numbers or the same terms in different embodiments can be referred to each other according to the related description. It should be understood that the features of the following embodiments can be combined with each other. For example, the features of the second embodiment can be combined with the features of the first embodiment. Those skilled in the art can select a suitable combination of features according to the actual design requirements.

[0024] Processors such as artificial intelligence chips can provide tremendous computing power. The tremendous computing power of an artificial intelligence chip is derived from a large number of hardware cores inside. An artificial intelligence chip usually contains multiple programmable processors, such as a stream processor cluster (SPC). Each programmable processor usually contains multiple compute units (CUs, or compute cores), and each compute core usually contains multiple execution units (EUs, or execution cores). The execution cores include at least one of a tensor core (Tcore), an integer (INT) core, a floating point (FP) core, and a vector core (Vcore), for example. By programming the compute cores, the artificial intelligence chip can support general-purpose computing, scientific computing, and neural network computing. The compute cores of the artificial intelligence chip usually access data of a main memory through a cache, such as a last level cache (LLC).

[0025] Figure 1 is a circuit block diagram of an artificial intelligence chip according to an embodiment of the present application. In Figure 1 The artificial intelligence chip 100 includes multiple compute cores 110, a cache 120, a main memory 130, and a scratchpad allocation-release unit 140 in the embodiment shown. The number of compute cores 110 can be determined according to actual design and application. The compute cores 110 are also referred to as compute units. Although not shown in Figure 1 Each compute core 110 usually contains multiple execution units and a shared memory. Different execution units in the same compute core can exchange data with each other through the shared memory. The cache 120 is coupled between the compute cores 110 and the main memory 130. The cache 120 can be a last level cache or other cache of the artificial intelligence chip 100. The compute cores 110 access data of the main memory 130 through the cache 120. The multiple compute cores 110 are coupled to the cache 120. The scratchpad allocation-release unit 140 is coupled to the multiple compute cores 110 and the cache 120.

[0026] Figure 2 is a flowchart of a cache operation method of an artificial intelligence chip according to an embodiment of the present application. Please refer to Figure 1 and Figure 2In step S210, the cache 120 dynamically allocates the storage space of the cache 120 into a cache mode space and a scratchpad memory (SPM) space. The SPM is a memory management manner different from the conventional cache. The compute cores 110 can directly control and use the SPM space. For example, the compute cores 110 can use the SPM space to temporarily store data or instructions.

[0027] In step S220, at least one of the compute cores 110 can issue a cache mode access request to the cache 120 to access the cache mode space of the cache 120. In addition, at least one of the compute cores 110 can issue a SPM access request to the SPM allocation-deallocation unit 140. Based on the SPM access request issued by at least one of the compute cores 110, the SPM allocation-deallocation unit 140 accesses the SPM space of the cache 120 (step S220). For example, the SPM allocation-deallocation unit 140 includes a remap table. The SPM allocation-deallocation unit 140 converts the access address of the SPM access request into a cache line address based on the remap table, and then accesses the SPM space of the cache 120 based on the cache line address.

[0028] In response to the SPM space being insufficient to serve the SPM access request, the SPM allocation-deallocation unit 140 generates a SPM allocation request to the cache 120 to dynamically increase the SPM space (step S230). That is, part of the cache mode space is allocated (moved) to the SPM space. The meaning that the SPM space is insufficient to serve the SPM access request may, for example, be that the SPM space size is smaller than the space size required by the SPM access request. In response to the SPM space of the cache 120 being greater than the SPM access requirement of the compute cores 110, the SPM allocation-deallocation unit 140 generates a SPM deallocation request to the cache 120 to dynamically decrease the SPM space. That is, part of the SPM space is released (moved) to the cache mode space. The meaning that the SPM space is greater than the SPM access requirement of the compute cores 110 may, for example, be that the SPM space size is greater than the space required by the SPM access request of the compute cores 110.

[0029] In summary, the artificial intelligence chip 100 is arranged with a scratchpad allocation-release unit 140. The scratchpad allocation-release unit 140 can perform scratchpad mode access requests of the compute core 110. That is, the compute core 110 running an application program can perform data access to the scratchpad mode space of the cache 120 through the scratchpad allocation-release unit 140. During the use of the application program, in response to the scratchpad mode space being insufficient to serve the scratchpad mode access request, the scratchpad allocation-release unit 140 can generate a scratchpad allocation request to the cache 120 to dynamically increase the scratchpad mode space. That is, the artificial intelligence chip 100 can dynamically modify the allocation of the cache mode space and the scratchpad mode space of the cache 120 during the use of the application program.

[0030] Figure 3 is a circuit block schematic diagram of a cache shown in accordance with an embodiment of the present application. Figure 3 The cache 120 shown can be used as Figure 1 is one of the implementation examples of the cache 120. Figure 3 The artificial intelligence chip 100, the compute core 110, the cache 120, the main memory 130, and the scratchpad allocation-release unit 140 shown can refer to Figure 1 The related descriptions of Figure 2 are not repeated here. Figure 3 The cache 120 shown includes a cache line allocation-release unit 121, a hit-miss check unit 122, a tag memory 123, an operation engine 124, a memory controller 125, and a cache line array 126. According to different designs, in some embodiments, the implementation of at least one of the cache line allocation-release unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125 can be a hardware circuit. In other embodiments, the implementation of at least one of the cache line allocation-release unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125 can be a combination of more of hardware, firmware, and software (i.e., a program).

[0031] In hardware form, at least one of the cache line allocation-deallocation unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125 can be implemented as logic circuits on an integrated circuit. For example, the relevant functions of at least one of the cache line allocation-deallocation unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125 can be implemented as various logic blocks, modules, and circuits in one or more hardware controllers, microcontrollers, hardware processors, microprocessors, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), central processing units (CPUs), or other processing units. The relevant functions of at least one of the cache line allocation-deallocation unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125 can be implemented as hardware circuits, such as various logic blocks, modules, and circuits in an integrated circuit, using hardware description languages (such as Verilog HDL or VHDL) or other suitable programming languages.

[0032] In software or firmware form, the functions of at least one of the cache line allocation-deallocation unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125 can be implemented as programming codes. For example, at least one of the cache line allocation-deallocation unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125 is implemented using a general programming language (e.g., C, C++, or assembly language) or other suitable programming language. The programming codes can be recorded or stored in a "non-transitory machine-readable storage medium." In some embodiments, the non-transitory machine-readable storage medium includes, for example, a semiconductor memory and / or a storage device. An electronic device (e.g., a CPU, a hardware controller, a microcontroller, a hardware processor, or a microprocessor) can read and execute the programming codes from the non-transitory machine-readable storage medium to implement the functions of at least one of the cache line allocation-deallocation unit 121, the hit-miss check unit 122, the operation engine 124, and the memory controller 125.

[0033] The cache line allocation-deallocation unit 121 is coupled to the scratchpad allocation-deallocation unit 140. The hit-miss check unit 122 is coupled to the plurality of compute cores 110 and the cache line allocation-deallocation unit 121. The operation engine 124 is coupled to the hit-miss check unit 122 and the cache line allocation-deallocation unit 121. The memory controller 125 is coupled to the operation engine 124 and the scratchpad allocation-deallocation unit 140. The cache line array 126 is coupled to the memory controller 125. The cache line array 126 includes a plurality of cache line sets, and each cache line set includes a plurality of cache lines. The storage space of the cache line array 126 is dynamically allocated as cache mode space and scratchpad mode space. At least one of the plurality of compute cores 110 can issue a scratchpad mode access request to the scratchpad allocation-deallocation unit 140. In response to at least one of the plurality of compute cores 110 issuing the scratchpad mode access request, the scratchpad allocation-deallocation unit 140 performs data access to the scratchpad mode space in the cache line array 126 through the memory controller 125.

[0034] In response to the scratchpad mode space of the cache line array 126 being insufficient to service the scratchpad mode access request, or in response to at least one of the compute cores 110 sending a scratchpad allocation command (carrying size information of scratchpad mode space needed to be allocated) to the scratchpad allocation-release unit 140, the scratchpad allocation-release unit 140 generates a scratchpad allocation request to the cache line allocation-release unit 121 to dynamically increase the scratchpad mode space. After the scratchpad allocation-release unit 140 receives the scratchpad allocation command of the compute core 110, the scratchpad allocation-release unit 140 compares the size information of the scratchpad space therein with the current scratchpad mode space. If the current scratchpad mode space cannot meet the requirement of the compute core 110 (i.e., the scratchpad mode space is insufficient to service the scratchpad mode access request of the compute core 110), the scratchpad allocation-release unit 140 generates at least one scratchpad allocation request for sending to the cache 120 to apply for scratchpad mode space.

[0035] The cache line allocation-release unit 121 executes the scratchpad allocation request to find a target cache line from the cache mode space that can be allocated to the scratchpad mode space. Under the premise of ensuring consistency, if the target cache line is clean, it is directly allocated, and if the target cache line is dirty, it is evicted to the main memory 130. The cache line allocation-release unit 121 returns the cache line address of the at least one target cache line to the scratchpad allocation-release unit 140. The scratchpad allocation-release unit 140 records the cache line address of the at least one target cache line in the remapping table to dynamically increase the scratchpad mode space. That is, part of the cache mode space is allocated (moved) to the scratchpad mode space.

[0036] In detail, the cache line allocation-release unit 121 has a cache line set state table. The cache line array 126 includes a plurality of cache line sets, and the cache line set state table counts the number of cache lines of each of the plurality of cache line sets that are configured as the scratchpad mode space. For example, the cache line set state table is used to indicate a count value of each of the cache line sets in the cache line array 126, and the count value represents how many cache lines in a certain cache line set are allocated (moved) as part of the scratchpad mode space. In response to the scratchpad allocation-release unit 140 generating the scratchpad allocation request to the cache 120, the cache line allocation-release unit 121 selects at least one target cache line set from the plurality of cache line sets based on the cache line set state table. For example (but not limited to), the cache line allocation-release unit 121 selects at least one cache line set with the least number of cache lines from the cache line set state table as the at least one target cache line set. The cache line allocation-release unit 121 provides group address information of the at least one target cache line set to the hit-miss check unit 122.

[0037] The hit-miss check unit 122 selects at least one target cache line from the plurality of cache lines of the at least one target cache line set. The hit-miss check unit 122 has a cache line busy status table including a plurality of busy bits, the plurality of busy bits of the cache line busy status table recording busy status of the plurality of cache lines of the plurality of cache line sets. The hit-miss check unit 122 multiplexes the cache line busy status table to select at least one cache line from the plurality of cache lines of the at least one target cache line set as non-busy status as the at least one target cache line. The hit-miss check unit 122 provides the cache line address of the at least one target cache line to the operation engine 124. In addition, the hit-miss check unit 122 sets at least one busy bit corresponding to the at least one target cache line in the cache line busy status table to busy status to update the cache line busy status table.

[0038] In response to the selected at least one target cache line being dirty data, the hit-miss check unit 122 first informs the operation engine 124 to store back the data of the selected at least one target cache line from the cache line array 126 to the main memory 130 through the memory controller 125 and the operation engine 124. After the dirty data of the selected at least one target cache line is stored back to the main memory 130, the hit-miss check unit 122 provides the cache line address of the selected at least one target cache line to the operation engine 124, and sets at least one busy bit corresponding to the selected at least one target cache line in the cache line busy status table to busy status. The operation engine 124 returns the cache line address of the selected at least one target cache line to the cache line allocation-release unit 121.

[0039] The cache line allocation-release unit 121 returns the cache line address of the selected at least one target cache line to the scratchpad allocation-release unit 140. In response to the scratchpad allocation-release unit 140 receiving the cache line address of the selected at least one target cache line returned by the cache line allocation-release unit 121, the scratchpad allocation-release unit 140 records the cache line address of the selected at least one target cache line in the remapping table to dynamically increase the scratchpad mode space.

[0040] In response to the scratchpad mode space of the cache 120 being greater than the scratchpad mode access requirement of the compute core 110, or in response to at least one of the compute cores 110 sending a scratchpad release command (carrying size information of scratchpad mode space needed to be released) to the scratchpad allocation-release unit 140, the scratchpad allocation-release unit 140 generates a scratchpad release request to the cache 120 to dynamically reduce the scratchpad mode space.

[0041] In detail, the scratchpad allocation-release unit 140 selects the cache line address of the pre-release target from the scratchpad mode space according to the remapping table to generate at least one scratchpad release request carrying the cache line address to the cache line allocation-release unit 121. After the cache line allocation-release unit 121 receives the scratchpad release request, the cache line corresponding to the address is changed to the cache mode. That is, part of the scratchpad mode space is released (moved) to the cache mode space.

[0042] At least one of the plurality of computing cores 110 can issue a cache mode access request to the cache 120 to access data in the cache mode space of the cache 120. In response to at least one of the plurality of computing cores 110 issuing a cache mode access request, the hit-miss check unit 122 determines whether the cache mode access request is a hit, that is, whether the target data of the cache mode access request is in the cache line array 126, based on the contents of the tag memory 123. The hit-miss check unit 122 notifies the operation engine 124 of the determination result. In response to the hit-miss check unit 122 determining that the cache mode access request is a hit, the operation engine 124 accesses the target cache line corresponding to the cache mode access request in the cache mode space of the cache line array 126 through the memory controller 125, and then the operation engine 124 returns the data of the target cache line to the computing core 110 through the hit-miss check unit 122. The main memory 130 is coupled to the operation engine 124. In response to the hit-miss check unit 122 determining that the cache mode access request is a miss, the operation engine 124 accesses the main memory 130, and then the operation engine 124 returns the data to the computing core 110 through the hit-miss check unit 122.

[0043] In summary, the scratchpad allocation-release unit 140 can dynamically apply the storage space of the cache line array 126 as the cache mode space and the scratchpad mode space. The application software completes the bidirectional conversion of the storage space of the cache line array 126 from the cache mode to the scratchpad mode by requesting application. The application software sends a scratchpad allocation request to the cache 120 through the scratchpad allocation-release unit 140 to allow the cache 120 to select at least one allocatable cache line or to retrieve a cache line that can be allocated after being retrieved, and then pass the selection information to the scratchpad allocation-release unit 140. The cache line can be dynamically configured as a scratchpad mode. When the application does not need it, the cache line allocated as the scratchpad mode can also be released by sending a scratchpad release request through the scratchpad allocation-release unit 140, and the cache line will be switched back to the cache mode. In this way, the cache 120 can achieve the purpose of dynamic sharing of storage space and maximize application performance. Without changing the original pipeline structure, the implementation logic is simple, the cost is low, and the like.

[0044] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An artificial intelligence chip, characterized in that, The artificial intelligence chip includes: The cache, wherein the cache dynamically allocates the storage space of the cache into a cache mode space and a notepad mode space; Multiple computing cores, coupled to the cache, wherein at least one of the multiple computing cores issues a cache mode access request to the cache to access data in the cache mode space; and A notepad allocation-release unit is coupled to the plurality of computing cores and the cache, wherein the notepad allocation-release unit accesses the notepad pattern space of the cache based on a notepad pattern access request issued by at least one of the plurality of computing cores, and, in response to the notepad pattern space being insufficient to serve the notepad pattern access request, the notepad allocation-release unit generates a notepad allocation request to the cache to dynamically increase the notepad pattern space. The cache includes: A cache line allocation-release unit is coupled to the notepad allocation-release unit; The hit-miss check unit is coupled to the plurality of computing cores and the cache line allocation-release unit; The operation engine is coupled to the hit-miss check unit and the cache line allocation-release unit; The memory controller is coupled to the operating engine and the notepad allocation-release unit; and A cache line array coupled to the memory controller, wherein the storage space of the cache line array is dynamically allocated into the cache mode space and the notepad mode space; The cache line array includes multiple cache line groups, and the cache line allocation-release unit has a cache line group status table. In response to the notepad allocation-release unit generating the notepad allocation request to the cache, the cache line allocation-release unit selects at least one target cache line group from the plurality of cache line groups based on the cache line group status table; the hit-miss check unit selects at least one target cache line from the plurality of cache lines in the at least one target cache line group; the operation engine returns the cache line address of the at least one target cache line to the cache line allocation-release unit; and the cache line allocation-release unit returns the cache line address of the at least one target cache line to the notepad allocation-release unit. In response to the notepad allocation-release unit receiving the cache line address of the at least one target cache line from the cache line allocation-release unit, the notepad allocation-release unit records the cache line address of the at least one target cache line in the remapping table to dynamically increase the notepad pattern space.

2. The artificial intelligence chip according to claim 1, characterized in that, In response to a notepad mode access request where the notepad mode space is larger than the number of computing cores, the notepad allocation-release unit generates a notepad release request for the cache to dynamically reduce the notepad mode space.

3. The artificial intelligence chip according to claim 1, characterized in that, The notepad allocation-release unit includes the remapping table. The notepad allocation-release unit converts the access address of the notepad mode access request into a cache line address based on the remapping table, and the notepad allocation-release unit accesses the cached notepad mode space based on the cache line address.

4. The artificial intelligence chip according to claim 1, characterized in that, In response to at least one of the plurality of computing cores issuing the notepad mode access request, the notepad allocation-release unit accesses the notepad mode space in the cache line array via the memory controller.

5. The artificial intelligence chip according to claim 1, characterized in that, The cache line group status table counts the number of cache lines in each of the plurality of cache line groups that are configured as the notepad mode space, and The cache line allocation-release unit selects at least one cache line group with the fewest cache lines from the cache line group status table as at least one target cache line group.

6. The artificial intelligence chip according to claim 1, characterized in that, The hit-miss check unit has a cache line busy status table, which includes multiple busy bits. These busy bits record the busy status of multiple cache lines in the multiple cache line groups. The hit-miss check unit reuses the cache line busy status table to select at least one non-busy cache line from a plurality of cache lines in at least one target cache line group as at least one target cache line, and The hit-miss check unit sets at least one busy bit corresponding to the at least one target cache line in the cache line busy status table to a busy state, thereby updating the cache line busy status table.

7. The artificial intelligence chip according to claim 6, characterized in that, In response to the selected at least one target cache line being dirty data, the hit-miss check unit notifies the operation engine to store the data of the at least one target cache line from the cache line array back to main memory.

8. The artificial intelligence chip according to claim 1, characterized in that, The artificial intelligence chip also includes: Main memory, coupled to the operating engine, wherein, In response to the hit-miss check unit determining that the cache mode access request is a hit, the operation engine accesses the target cache line corresponding to the cache mode access request in the cache line array via the memory controller; and In response to the hit-miss check unit determining that the cache mode access request is a miss, the operation engine accesses the main memory.

9. A caching operation method for an artificial intelligence chip, characterized in that, The caching operation method includes: The storage space of the cache is dynamically allocated into a cache mode space and a notepad mode space by the cache of the artificial intelligence chip, wherein the cache is coupled to multiple computing cores of the artificial intelligence chip and a notepad allocation-release unit, and the notepad allocation-release unit is coupled to the multiple computing cores; At least one of the plurality of computing cores issues a cache mode access request to the cache in order to access data in the cache mode space; The notepad allocation-release unit accesses data in the cached notepad pattern space based on a notepad pattern access request issued by at least one of the plurality of computing cores; and In response to the fact that the sticky note mode space is insufficient to serve the sticky note mode access request, the sticky note allocation-release unit generates a sticky note allocation request to the cache to dynamically increase the sticky note mode space; The cache includes a cache line allocation-release unit, a hit-miss check unit, an operation engine, a memory controller, and a cache line array. The cache line allocation-release unit is coupled to the notepad allocation-release unit. The hit-miss check unit is coupled to the plurality of computing cores and the cache line allocation-release unit. The operation engine is coupled to the hit-miss check unit and the cache line allocation-release unit. The memory controller is coupled to the operation engine and the notepad allocation-release unit. The cache line array is coupled to the memory controller. The storage space of the cache line array is dynamically allocated into the cache mode space and the notepad mode space. The cache line array includes multiple cache line groups, the cache line allocation-release unit has a cache line group status table, and the cache operation method further includes: In response to the notepad allocation-release unit generating the notepad allocation request to the cache, the cache line allocation-release unit selects at least one target cache line group from the plurality of cache line groups based on the cache line group status table; the hit-miss check unit selects at least one target cache line from the plurality of cache lines of the at least one target cache line group; the operation engine returns the cache line address of the at least one target cache line to the cache line allocation-release unit; and the cache line allocation-release unit returns the cache line address of the at least one target cache line to the notepad allocation-release unit. In response to the notepad allocation-release unit receiving the cache line address of the at least one target cache line from the cache line allocation-release unit, the notepad allocation-release unit records the cache line address of the at least one target cache line in the remapping table to dynamically increase the notepad pattern space.

10. The caching operation method according to claim 9, characterized in that, The caching operation method further includes: In response to a notepad mode access request where the notepad mode space is larger than the number of computing cores, the notepad allocation-release unit generates a notepad release request for the cache to dynamically reduce the notepad mode space.

11. The caching operation method according to claim 9, characterized in that, The notepad allocation-release unit includes the remapping table, and the caching operation method further includes: The notepad allocation-release unit converts the access address of the notepad mode access request into a cache line address based on the remapping table; and The notepad allocation-release unit accesses data in the cached notepad pattern space based on the cache line address.

12. The caching operation method according to claim 9, characterized in that, The caching operation method further includes: In response to at least one of the plurality of computing cores issuing the notepad mode access request, the notepad allocation-release unit accesses the notepad mode space in the cache line array via the memory controller.

13. The caching operation method according to claim 9, characterized in that, The caching operation method further includes: The number of cache lines configured as the notepad mode space in the plurality of cache lines of each of the plurality of cache lines is counted by the cache line group status table; and The cache line allocation-release unit selects at least one cache line group with the fewest cache lines from the cache line group status table as at least one target cache line group.

14. The caching operation method according to claim 9, characterized in that, The hit-miss check unit has a cache line busy status table, which includes multiple busy bits. The cache operation method further includes: The busy status of multiple cache lines in the multiple cache line groups is recorded by multiple busy bits in the cache line busy status table; The hit-miss checking unit reuses the cache line busy status table to select at least one non-busy cache line from a plurality of cache lines in at least one target cache line group as at least one target cache line; and The hit-miss check unit sets at least one busy bit corresponding to at least one target cache line in the cache line busy status table to a busy state, thereby updating the cache line busy status table.

15. The caching operation method according to claim 14, characterized in that, The caching operation method further includes: In response to the selected at least one target cache line being dirty data, the hit-miss check unit notifies the operation engine to store the data of the at least one target cache line from the cache line array back to main memory.

16. The caching operation method according to claim 9, characterized in that, The caching operation method further includes: In response to the hit-miss check unit determining that the cache mode access request is a hit, the operation engine accesses the target cache line corresponding to the cache mode access request in the cache line array via the memory controller; and In response to the hit-miss check unit determining that the cache mode access request is a miss, the operation engine accesses the main memory of the artificial intelligence chip, wherein the main memory is coupled to the operation engine.

Images