Method for cache memory and related products thereof

Abstract

The present disclosure relates to a method for a cache memory, a cache memory, a system on chip, a board card and a computing device, wherein the computing device of the present disclosure is embodied in a computing processing device included in a combined processing device, which can further include a general-purpose interconnection interface and other processing devices. The computing processing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing processing device and the other processing devices, respectively, for storing data of the computing processing device and the other processing devices. The present disclosure can improve the efficiency of the use of the cache memory.

Description

Technical Field

[0001] This disclosure generally relates to the field of chip technology. More specifically, this disclosure relates to a method for using a cache memory, a cache memory, a system-on-a-chip including the cache memory, a board including the system-on-a-chip, and a computing device including the board. Background Technology

[0002] The operational performance of a computing system largely depends on the average memory access latency. Improving the hit rate of the cache memory (or simply "cache") to effectively reduce the number of memory accesses can significantly improve system performance. To this end, processors typically employ caching mechanisms to address the speed and performance mismatch between the processor and the slower main memory. Current caches implement multi-level caching mechanisms, such as three-level caches (L1, L2, and L3), with the cache closest to main memory called the Last Level Cache (LLC). Given the frequent use and crucial role of caches in on-chip systems, effective management strategies are needed to improve cache utilization and reduce the number of main memory accesses. Furthermore, how to expand the application of LLC for different scenarios is also a problem that needs to be solved. Summary of the Invention

[0003] In view of the technical problems mentioned in the background section above, this disclosure provides a resident scheme for cache memory. The scheme of this disclosure allows a specific region in the cache memory to be configured as a locked region, and frequently used data can be resided therein, thereby improving cache hit rate and enhancing overall system performance. Based on this, this disclosure provides a scheme for cache memory in the following aspects.

[0004] In a first aspect, this disclosure provides a method for a cache memory, comprising: configuring a specific storage space in the cache memory as a latch region supporting multiple latch modes, wherein each latch mode corresponds to a latch-related operation performed on data in the latch region; receiving a latch-related request for performing a latch-related operation on the data in the latch region; and performing a latch-related operation on the data in the latch region according to the latch-related request and the corresponding latch mode.

[0005] In a second aspect, this disclosure provides a cache memory, comprising: a configuration module for configuring a specific storage space in the cache memory as a latch area supporting multiple latch modes, wherein each latch mode corresponds to a latch-related operation performed on data in the latch area; and a latch execution module for: receiving a latch-related request to perform a latch-related operation on the data in the latch area; and performing a latch-related operation on the data in the latch area according to the latch-related request and the corresponding latch mode.

[0006] In a third aspect, this disclosure provides a system-on-a-chip including a cache memory as described above and in several embodiments below; and a processor configured to generate the latch-related request; wherein the latch execution module of the cache memory is configured to perform latch-related operations on the data in the latch area according to the latch-related request and in a corresponding latch mode.

[0007] In a fourth aspect, this disclosure provides a board including a system-on-a-chip as described above and in several embodiments below.

[0008] In a fifth aspect, this disclosure provides a computing device including the board described above and in several embodiments below.

[0009] According to the solutions provided in the aforementioned aspects of this disclosure, a latch area can be used to perform latching and unlocking operations on frequently used data, thereby significantly improving cache hit rate. Furthermore, since the latch area of ​​this disclosure supports multiple latching modes, which can be selected and used according to configuration, the application scenarios of the latch area are expanded. When used in producer and consumer kernel scenarios, the latch area of ​​this disclosure can act as a medium for data transfer, thereby improving data accessibility and utilization. In addition, since the latch area improves the probability of cache hits, the solution of this disclosure also significantly improves the overall performance of the computing system. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0011] Figure 1 This is a structural diagram of a board according to an embodiment of the present disclosure;

[0012] Figure 2 This is a structural diagram illustrating an integrated circuit device according to an embodiment of the present disclosure;

[0013] Figure 3 This is a schematic diagram showing the internal structure of a single-core computing device according to an embodiment of the present disclosure;

[0014] Figure 4 This is a schematic diagram showing the internal structure of a multi-core computing device according to an embodiment of the present disclosure;

[0015] Figure 5 This is a schematic diagram illustrating the internal structure of a processor core according to an embodiment of the present disclosure;

[0016] Figure 6 This is a flowchart illustrating a method for a cache memory according to an embodiment of the present disclosure;

[0017] Figure 7 This is a simplified block diagram illustrating a cache memory according to an embodiment of the present disclosure;

[0018] Figure 8 This is a simplified block diagram illustrating a system-on-a-chip according to an embodiment of the present disclosure;

[0019] Figure 9 This is a detailed block diagram illustrating a system-on-a-chip according to an embodiment of the present disclosure;

[0020] Figure 10 This is a schematic block diagram illustrating a page pattern according to an embodiment of the present disclosure;

[0021] Figure 11 This is a schematic diagram illustrating a hash operation in window mode according to an embodiment of the present disclosure;

[0022] Figure 12 This is a simplified block diagram illustrating a system-on-a-chip according to an embodiment of the present disclosure;

[0023] Figure 13 This is a flowchart illustrating a method for a system-on-a-chip according to embodiments of the present disclosure; and

[0024] Figure 14 This is an operational block diagram illustrating a system-on-a-chip according to an embodiment of the present disclosure. Detailed Implementation

[0025] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure, and the multiple described embodiments can be appropriately combined according to different scenarios to achieve different applications. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0026] It should be understood that the terms "first," "second," and "third," etc., that may be used in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or sets thereof.

[0027] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0028] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0029] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0030] Figure 1 A schematic diagram of the structure of a board 10 according to an embodiment of the present disclosure is shown. It will be understood that... Figure 1 The structure and composition shown are merely examples and are not intended to limit the scope of this disclosure in any way.

[0031] like Figure 1As shown, board 10 includes chip 101, which can be a system-on-chip (SoC), i.e., a system-on-a-chip as described in the context of this disclosure. In one implementation scenario, it can integrate one or more combined processing devices. The aforementioned combined processing device can be an artificial intelligence computing unit used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining, especially since deep learning technology is widely used in the field of cloud intelligence. A significant characteristic of cloud intelligence applications is the large amount of input data, which places high demands on the platform's storage and computing capabilities. Board 10 of this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0032] As further shown in the figure, chip 101 is connected to external device 103 via external interface device 102. Depending on the application scenario, external device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results of chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0033] The board 10 may also include a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 can be configured to regulate the state of the chip 101. For this purpose, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0034] Figure 2 This is a structural diagram illustrating the combined processing apparatus in chip 101 according to the above embodiment. Figure 2 As shown, the combined processing device 20 may include a computing device 201, an interface device 202, a processing device 203, and a dynamic random access memory (DRAM) DRAM 204.

[0035] The computing device 201 can be configured to execute user-specified operations, primarily implemented as a single-core or multi-core intelligent processor. In some operations, it can be used to perform deep learning or machine learning calculations, and can also interact with the processing device 203 via the interface device 202 to jointly complete the user-specified operations.

[0036] Interface device 202 can be used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0037] The processing device 203, as a general-purpose processing device, performs basic controls including but not limited to data transfer and starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 may be one or more types of processors, including but not limited to digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing device 201 of this disclosure can be considered to have a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0038] DRAM 204 is used to store data to be processed. It is DDR memory, typically 16G or larger in size, and is used to store data in computing device 201 and / or processing device 203.

[0039] Figure 3A schematic diagram of the internal structure of the computing device 201 as a single core is shown. The single-core computing device 301 is used to process input data such as computer vision, speech, natural language processing, and data mining. The single-core computing device 301 includes three main modules: a control module 31, a processing module 32, and a storage module 33. The control module 31 coordinates and controls the operation of the processing module 32 and the storage module 33 to complete deep learning tasks. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 fetches instructions from the processing device 203, and the instruction decode unit 312 decodes the fetched instructions and sends the decoding result as control information to the processing module 32 and the storage module 33.

[0040] The computation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 performs vector operations and supports complex operations such as vector multiplication, addition, and nonlinear transformations. The matrix operation unit 322 is responsible for the core computations of the deep learning algorithm, namely matrix multiplication and convolution. The storage module 33 is used to store or move related data, including a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access (DMA) module 333. The NRAM 331 stores the input neurons, output neurons, and intermediate results after computation. The WRAM 332 stores the convolution kernels of the deep learning network, i.e., the weights. The DMA 333 is connected to the DRAM 204 via the bus 34 and is responsible for data transfer between the single-core computing device 301 and the DRAM 204.

[0041] Figure 4 A schematic diagram of the internal structure of the computing device 201 as a multi-core processor is shown. The multi-core computing device 41 adopts a hierarchical design. As a system-on-a-chip, the multi-core computing device 41 includes at least one cluster according to this disclosure, and each cluster includes multiple processor cores. In other words, the multi-core computing device 41 is constructed in a hierarchical structure of system-on-a-chip, cluster, and processor cores. From the perspective of the system-on-a-chip hierarchy, as... Figure 4 As shown, the multi-core computing device 41 includes an external storage controller 401, a peripheral communication module 402, an on-chip interconnect module 403, a synchronization module 404, and multiple clusters 405.

[0042] There may be multiple external storage controllers 401 (two are shown as an example in the figure), which are used to access external storage devices, i.e., off-chip memory in the context of this disclosure, in response to access requests issued by the processor core (e.g., external storage devices). Figure 2The DRAM 204 in the chip allows data to be read from or written to external devices. The peripheral communication module 402 receives control signals from the processing device 203 via the interface device 202, initiating the computing device 201 to execute tasks. The on-chip interconnect module 403 connects the external memory controller 401, the peripheral communication module 402, and multiple clusters 405 to transmit data and control signals between modules. The synchronization module 404 is a Global Barrier Controller (GBC) used to coordinate the working progress of each cluster and ensure information synchronization. The multiple clusters 405 of this disclosure are the computing cores of the multi-core computing device 41. Although in Figure 4 The illustration exemplarily shows four clusters; however, with the development of hardware, the multi-core computing device 41 of this disclosure may also include eight, sixteen, sixty-four, or even more clusters 405. In one application scenario, cluster 405 can be used to efficiently execute deep learning algorithms.

[0043] From the perspective of cluster hierarchy, such as Figure 4 As shown, each cluster 405 may include multiple processor cores (IPU cores) 406 and a memory core (MEM core) 407, which may include, for example, a cache memory (e.g., LLC) as described in the context of this disclosure.

[0044] The processor core 406 is exemplarily shown as four in the figure. This disclosure does not limit the number of processor cores 406, and its internal architecture is as follows. Figure 5 As shown. Each processor core 406 is similar to Figure 3 The single-core computing device 301 can also include three modules: a control module 51, an arithmetic module 52, and a storage module 53. The functions and structures of the control module 51, arithmetic module 52, and storage module 53 are roughly the same as those of the control module 31, arithmetic module 32, and storage module 33, and will not be described in detail here. It should be noted that the storage module 53 can include an Input / Output Direct Memory Access (IODMA) module 533 and a Move Direct Memory Access (MVDMA) module 534. The IODMA 533 controls the memory access of NRAM 531 / WRAM 532 and DRAM 204 through the broadcast bus 409; the MVDMA 534 is used to control the memory access of NRAM 531 / WRAM 532 and SRAM 408.

[0045] Back Figure 4The storage core 407 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 406, and performing communication between cluster 405 and DRAM 204, communication between clusters 405, and communication between processor cores 406. In other embodiments, the storage core 407 may have scalar operation capabilities to perform scalar operations.

[0046] Storage core 407 may include Static Random-Access Memory (SRAM) 408, a broadcast bus 409, a Cluster Direct Memory Access (CDMA) module 410, and a Global Direct Memory Access (GDMA) module 411. In one implementation scenario, SRAM 408 can act as a high-performance data relay station. Therefore, data multiplexed between different processor cores 406 within the same cluster 405 does not need to be obtained from DRAM 204 by each processor core 406 individually; instead, it is relayed between processor cores 406 via SRAM 408. Furthermore, storage core 407 only needs to quickly distribute multiplexed data from SRAM 408 to multiple processor cores 406, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip I / O access.

[0047] Broadcast bus 409, CDMA 410, and GDMA 411 are used to perform communication between processor cores 406, communication between clusters 405, and data transfer between cluster 405 and DRAM 204, respectively. These will be explained below.

[0048] The broadcast bus 409 is used to complete high-speed communication between the processor cores 406 within the cluster 405. In this embodiment, the broadcast bus 409 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (e.g., data transmission from one processor core to another) data transmission. Multicast is a communication method that transmits a piece of data from SRAM 408 to several specific processor cores 406. Broadcast is a communication method that transmits a piece of data from SRAM 408 to all processor cores 406, and is a special case of multicast.

[0049] CDMA 410 is used to control memory accesses of SRAM 408 between different clusters 405 within the same computing device 201. GDMA 411 works in conjunction with the external memory controller 401 to control memory accesses from SRAM 408 of cluster 405 to DRAM 204, or to read data from DRAM 204 into SRAM 408. As described above, communication between DRAM 204 and NRAM 431 or WRAM 432 can be achieved in two ways. The first way is through IODAM 433 directly communicating with DRAM 204 and NRAM 431 or WRAM 432; the second way is to first transmit data between DRAM 204 and SRAM 408 via GDMA 411, and then transmit data between SRAM 408 and NRAM 431 or WRAM 432 via MVDMA 534. Although the second approach may require more components and has a longer data flow, in some embodiments, the bandwidth of the second approach is significantly greater than that of the first approach. Therefore, performing communication between DRAM 204 and NRAM 431 or WRAM 432 using the second approach may be more efficient. It is understood that the data transmission methods described herein are merely exemplary, and those skilled in the art can flexibly select and apply various data transmission methods according to the specific hardware arrangement based on the teachings of this disclosure.

[0050] In other embodiments, the functions of GDMA 411 and IODMA 533 can be integrated into the same component. Although this disclosure treats GDMA 411 and IODMA 533 as different components for ease of description, those skilled in the art will recognize that any component that performs similar functions and achieves similar technical effects as this disclosure falls within its scope of protection. Furthermore, the functions of GDMA 411, IODMA 533, CDMA 410, and MVDMA 534 can also be implemented by the same component.

[0051] The above combination Figures 1-5The hardware architecture and internal structure of this disclosure are described in detail. It is understood that the above description is merely exemplary and not restrictive. Depending on different application scenarios and hardware specifications, those skilled in the art can also make changes to the board and its internal structure, and these changes still fall within the protection scope of this disclosure. For example, in the scheme described below, the corresponding hardware architecture may be a CDMA410 that does not include controlling access to SRAM 408 between different clusters 405 within the same computing device 201. Instead, the scheme below involves improving and optimizing, for example, a cache memory disposed between SRAM 408 and DRAM 204, to achieve efficient on-demand data latching and communication between different clusters through the cache memory.

[0052] To efficiently utilize cache memory (e.g., LLC) and improve data access hit rates, the present disclosure proposes configuring specific storage spaces within the cache memory as latch areas for data latching operations, particularly for data that will be frequently used. For example, the aforementioned frequently used data may be data that will be reused between at least one task with data dependencies. It is understood that when data is only needed once, it may not need to be latched into the cache memory.

[0053] Furthermore, based on the aforementioned configuration of the latch area for data latching, the following scheme of this disclosure also proposes configuring the cache memory to support multiple latching modes, so that when a latch-related request is received, the cache memory operates in the latching mode corresponding to the aforementioned latch-related request. Depending on different application scenarios and requirements, the multiple latching modes of this disclosure can have a specific priority order to satisfy different latch-related operations. In addition, to enable the cache memory to support multiple latching modes, the scheme of this disclosure also proposes several different configuration methods, allowing for more flexible use of the cache memory and its application in inter-cluster communication.

[0054] Figure 6 This is a flowchart illustrating a method 600 for a cache memory according to an embodiment of the present disclosure. Figure 6As shown, method 600 includes, at step S602, configuring a specific storage space in the cache memory as a latch region supporting multiple latch modes. In one embodiment, the aforementioned multiple latch modes may include, but are not limited to, an instruction mode that performs latch-related operations based on hardware instructions, a window mode that performs latch-related operations based on window attributes, a stream mode that performs latch-related operations based on data streams, and / or a page mode that performs latch-related operations based on cache pages. In one embodiment, the aforementioned data stream may be an instruction stream or a data stream with different types. Taking a data stream as an example, in a neural network application scenario, the data stream may be the neuron data stream, weight data stream, output result data stream, etc., of the neural network model. Furthermore, in the context of this disclosure, the data targeted by latch-related operations is data that will be used multiple times by the on-chip system's processor, and it has a relatively high priority compared to data that has not been latched. By latching (or residing) such frequently used data in the latch region of this disclosure, the cache hit rate can be significantly improved, thereby improving the overall performance of the system. In addition, by keeping the frequently used data in the latch area of ​​the LLC, the number of read and write operations between the on-chip system and off-chip memory (such as DDR or DRAM) can be reduced, thereby improving memory access efficiency.

[0055] In one application scenario, the various latch modes described above can be configured with different priorities based on user preferences or system optimizations. For example, in one implementation, the priority order could be instruction mode -> window mode -> stream mode -> page mode; in another implementation, the priority order could be instruction mode -> page mode -> stream mode -> window mode. This multi-mode and priority setting allows for more diverse use of the latch area in the cache, increasing the flexibility of latch usage to meet different application scenarios and system requirements. Furthermore, the latch modes can be iterated sequentially according to their priority order; when a high-priority latch mode is disabled, a low-priority latch mode can be used.

[0056] In one embodiment, a specific storage space can be configured as a latch area supporting a corresponding latch mode according to one of a plurality of received configuration instructions. In one scenario, the configuration instruction may include one or more configuration items to configure the aforementioned latch area. For example, the plurality of configuration items may include configuration items for enabling the latch area, disabling the latch area, and / or the latch area size. Further, corresponding latching strategies (e.g., the size of the latched data or the specific data to be latched) can be configured in the aforementioned instruction mode, window mode, stream mode, or page mode to latch different types or specific instructions, data, or data streams. Specific details on configuring corresponding latching strategies in different modes are provided below. Through such enabling, disabling, and various specific configurations, the solution of this disclosure can achieve flexible use of the cache memory, enabling it to operate in one of the various latching modes of this disclosure as needed, or in a normal mode.

[0057] Return to Figure 6 In the flowchart above, after completing the configuration operation at step S602, a latch-related request for performing latch-related operations on data in the latch area is received at step S604. According to embodiments of this disclosure, the latch-related request can be triggered by an operation aimed at residing specific data in the latch area. Alternatively, the latch-related request can also be triggered by an operation aimed at removing or releasing specific data from the latch area. As described in detail above, the latch-related request of this disclosure can also have different expressions or contents when operating in different latch modes. For example, for instruction mode, window mode, or stream mode, the latch-related request may include configuration items for indicating the behavioral attributes of the cache memory, etc.

[0058] In one embodiment, the configuration items used to indicate the behavioral attributes of the cache memory include at least one of the following configuration attributes:

[0059] Transient attribute: No caching within LLC, meaning data read and write operations are performed directly with external memory (such as DDR); used to avoid caching certain data that is only accessed once within LLC, thereby avoiding the occupation of LLC resources;

[0060] The `Lock` attribute: Retains specific data in the latch area, allowing reading and writing of data from the hit cache line. If the cache line belongs to the latch area, its attribute is configured as persistent; if it does not belong to the latch area, its attribute remains unchanged, i.e., it retains the normal attribute as described below. It should be clear that cache lines in the latch area have two attributes: persistent and normal. Cache lines with the persistent attribute in this latch area can only be accessed and replaced by latch-related requests with the `Lock` attribute.

[0061] Unlock attribute: After reading and writing data from the hit cache line, release the corresponding storage space of the data in the latch area of ​​LLC, and set the corresponding cache line attribute in the latch area to the following general attribute;

[0062] General attributes: Requests that are normally cached within the LLC can directly read and write data to external memory;

[0063] Invalid attribute: The data is invalidated immediately after reading to prevent it from being replaced and written to off-chip memory;

[0064] Clean attribute: During a write operation, data can be written to the hit cache line, and the entire contents of the cache memory are written back to off-chip memory, while the cache line's attributes remain unchanged. During a read operation, data is read from the hit cache line. If the hit cache line is dirty, it is written back to off-chip memory.

[0065] Default attribute: This default option can be used to indicate that the configuration regarding latch mode should be ignored.

[0066] By attaching the aforementioned exemplary configurable attributes to the latch-related request, the solution disclosed herein can perform corresponding latch-related operations in instruction mode based on these attached attributes.

[0067] For example, in page mode, a latch-related request can instruct that data associated with a specific page be latched in the latch area for subsequent multiple uses, or it can instruct that data associated with a specific page be unlocked from the latch area after multiple uses to free up more storage space for subsequent data latching. It is understood that through release operations, the storage space of the latch area can be used flexibly, thereby improving the utilization efficiency of the latch area disclosed herein.

[0068] Return to Figure 6In response to the latch-related request in step S604, at step S606, latch-related operations can be performed on the data in the latch area according to the latch-related request and the corresponding latch mode. According to embodiments of this disclosure, the aforementioned latch-related operations may include read and write operations for the latch area. In one embodiment, for a write operation to the latch area, method 600 may further include latching data or a selected portion of data in a designated area of ​​the latch area according to the latch-related request for subsequent multiple reads. In another embodiment, for a read operation to the latch area, method 600 may further include releasing the data or a selected portion of the data from the designated area of ​​the latch area according to the latch-related request after the read operation is completed.

[0069] Regarding the aforementioned selected portion of data, in one embodiment, a predetermined proportion of data can be randomly selected from the data to form the aforementioned portion of data to be latched in the latch area. In another embodiment, a hash algorithm can be used to select a predetermined proportion of data from the data as the aforementioned portion of data to be latched in the latch area. In a further embodiment, when the memory access address of the data to be latched is within the address range of the locking window, the aforementioned hash algorithm can be used to select the portion of data that can be latched in the latch area. The specific use of the hash algorithm will be discussed later in conjunction with the appendix. Figure 11 Let me describe it in detail.

[0070] Using the above combination Figure 6The described method of this disclosure enables the cache memory to support multiple latching modes, thereby expanding the application scenarios of the cache memory and significantly improving the cache hit rate. Furthermore, the introduction of multiple latching modes makes the use of the latch area more flexible and adaptable, thus meeting different application scenarios and user needs. In addition, effective latching of data in the latch area also promotes data sharing between the producer kernel and one or more consumer kernels, improving data accessibility and utilization. Here, the producer kernel and consumer kernel can be understood as two dependent tasks, where the output of the producer kernel serves as input to the consumer kernel, allowing the consumer kernel to use this input to complete its corresponding task. At this point, since the output of the producer kernel will be used as the input for subsequent calculations, the output of the producer kernel can be used as data that needs to be used multiple times. This data that needs to be used multiple times can be temporarily stored in the latch area of ​​the cache memory so that the consumer kernel can directly obtain the input from the cache memory without accessing the off-chip memory. This reduces the memory access interaction between the AI ​​processor and the off-chip memory, reduces the I / O memory access overhead, and thus improves the processing efficiency and performance of the AI ​​processor.

[0071] Figure 7 This is a simplified block diagram illustrating a cache memory 700 according to an embodiment of the present disclosure. It will be understood that... Figure 7 The cache memory 700 shown can be combined with Figure 6 The described cache memory, therefore regarding Figure 6 The described cache memory also applies to the following section on... Figure 7 The description.

[0072] like Figure 7 As shown, the cache memory 700 of this disclosure may include a configuration module 701 and a latch execution module 702. Furthermore, the cache memory 700 also includes storage space for performing cache operations, for example, as shown in the figure, the storage space is divided into 8 equal parts by 8 channels (way0 to way7), where each channel includes a number of cache lines.

[0073] In one embodiment, the configuration module described above can be used to configure a specific storage space in the cache memory as a latching region supporting multiple latching modes, wherein the size of the specific storage space is smaller than the total storage size of the cache memory. For example, Figure 7 Ways 0 through 5 can be configured as specific memory spaces that support latching. Correspondingly, Figure 7Ways 6 and 7 in the latch can maintain the normal properties of the cache memory, that is, be used as a general cache. As mentioned earlier, the latch mode can be instruction mode, window mode, stream mode, and / or page mode. Furthermore, the latch execution module can be used to receive latch-related requests for latch-related operations on data in the latch area. Then, the latch execution module can perform latch-related operations on the data in the latch area according to the latch-related request and the corresponding latch mode. Similar to the previous description, the latch-related operations here can include write operations to the latch area (i.e., writing data to the latch area) or releasing data from the latch area. For example, when a consumer kernel finishes using the data in the latch area, and that data will no longer be used by other consumer kernels, the space storing the data in the latch area can be released to latch other data.

[0074] Figure 8 This is a simplified block diagram illustrating a system-on-a-chip 800 according to an embodiment of the present disclosure. Figure 8 As shown, the system-on-a-chip 800 of this disclosure may include, for example: Figure 7 The cache memory 700 and processor (or processor core) 802 are shown. In one embodiment, the cache memory's latch execution module can be used to perform latch-related operations on the data in the latch area according to a latch-related request and a corresponding latch mode. Regarding the cache memory 700, the foregoing... Figure 6 and Figure 7 The description has already been provided and will not be repeated here. Regarding the processor 802, according to the scheme of this disclosure, it can be various types of processors and may include one or more processor cores to generate latch-related requests. In operation, the latch execution module of the cache memory is used to perform latch-related operations on data within the latch area according to the generated latch-related requests and the corresponding latch mode. For example, when the latch mode is instruction mode, the processor can be used to generate latch-related requests according to received hardware instructions. As another example, when the latch mode is page mode, the processor can be used to generate latch-related requests according to cache page configuration. Furthermore, when the latch mode is window mode or stream mode, the processor can be used to configure a locking window and generate latch-related requests according to the locking window.

[0075] Depending on the implementation, processor 802 may also be an intelligent processor or intelligent processing unit (“IPU”) that includes multiple computing cores, which can be configured to perform calculations in various fields of artificial intelligence (e.g., neural networks).

[0076] Figure 9This is a detailed block diagram illustrating a system-on-chip 900 according to an embodiment of the present disclosure. It will be understood that the system-on-chip 900 shown herein can be... Figure 8 The illustrated system-on-a-chip is a specific implementation method, therefore regarding Figure 8 The description also applies to Figure 9 Furthermore, for illustrative purposes only, the operation of the System-on-Chip 900 will be described in window mode (or stream mode) among multiple latching modes.

[0077] like Figure 9 As shown, the system-on-chip 900 may include a task scheduler (“Job Scheduler”) 902, which includes a scheduling unit 903 and a configurator 904. In one embodiment, the configurator 904 may be used to generate configuration instructions based on assigned configuration tasks (e.g., those available from a task queue) for sending to a configuration module (such as a CLR) in a cache memory (i.e., “LLC” 906). In one embodiment, the scheduling unit 903 may be used to schedule multiple tasks (i.e., “kernels” that will execute on the AI ​​processor) in the task scheduler for sending to the intelligent processor (IPU) 905 in the system-on-chip of this disclosure. In the scheme of this disclosure, the intelligent processor 905 may include multiple processor cores, which may be configured as follows: Figure 4 The diagram illustrates a cluster. In one implementation scenario, within the aforementioned multi-processor core architecture, the scheduling unit can allocate tasks to appropriate processor cores based on the idle time (e.g., utilization) of the multiple processor cores.

[0078] Furthermore, the System-on-Chip 900 also includes a System Memory Management Unit (SMMU), which translates the virtual address of memory access data into a physical address to access the relevant storage location based on the physical address. In one embodiment, the SMMU includes a Translation Lookaside Buffer (TLB). The TLB maintains a page table, which includes at least one page table entry. Each page table entry includes a page and its corresponding page frame. In operation, the SMMU can determine the page corresponding to the received virtual address, and then determine the physical address PA corresponding to the virtual address through the page-page-page-frame mapping relationship, thereby enabling access to the relevant storage location of the cache memory based on the physical address.

[0079] In one embodiment, access to the cache memory can be achieved through the window mode or streaming mode described above. In this case, the intelligent processor can retrieve a parameter table from the memory and configure a lock window ("Lock window") associated with the data to be latched based on the parameter table, and generate a latch-related request (i.e., an I / O access request with lock / unlock attributes) based on the configured lock window. Then, the SMMU can perform latch-related operations on the LLC based on the I / O access request. Specifically, the SMMU can send the aforementioned I / O access request to the cache policy module 907 of the LLC 906 (which performs operations similar to...). Figure 7 The same operation as the latch execution module 702 is performed. In one embodiment, the parameter table may include parameter items for configuring stream latch attributes in a lock window or stream mode. For example, parameter items may include, but are not limited to, information such as lock / unlock window, perstream lock / unlock, latch ratio, and lock window flag. In one implementation scenario, the parameters in the parameter table may be user-defined. Thus, the relevant parameters in the parameter table can be obtained during program execution, and the parameter table can be stored in memory (e.g., DDR) for use by the intelligent processor (IPU 905 in the figure) during execution.

[0080] In one embodiment, the aforementioned locking window represents the storage space that the software user wishes to latch, and the size of the locking window can be larger than the size of the latch area on the cache memory. The locking window includes one or more of the following: a base address and a size, wherein the base address of the window can be a virtual address configured by the upper-layer software (e.g., a virtual address, abbreviated as "VA"), the base address of the window corresponds to the starting address of the data for which latch-related operations are to be performed, and the size of the window can correspond to the size of the data to be latched.

[0081] Specifically, in windowed mode, the intelligent processor can determine the memory access address (which can be a virtual address) of the data in the task assigned by the task scheduler, and compare it with the address range defined by the window's locking window. If the memory access address of the data in the task is within the address range of the locking window, it means the locking window has been hit, and the locking window can be enabled (e.g., "Enabled"). Otherwise, if the memory access address of the data in the task is outside the address range of the locking window, it means the locking window has not been hit. In this case, the locking window can be ignored, meaning the data in the task will not be temporarily stored in the cache memory. Furthermore, when the memory access address of the data hits the locking window, a hash algorithm can be used to select a predetermined proportion of the data as part of the aforementioned data to be stored in the latch area. The specific use of the hash algorithm will be discussed later in conjunction with the appendix. Figure 11 This will be described in detail below. Subsequently, the intelligent processor can send a latch-related request with a Lock attribute to the cache memory LLC via the SMMU. This latch-related request with the Lock attribute can be used to instruct specific data to reside in the latch area; this specific data can be a subset of data selected according to a hash algorithm.

[0082] The following uses window mode, combined with Figure 9 This describes the latching and releasing processes of an LLC.

[0083] LLC residency (or lock-in) process:

[0084] Step 1: The task scheduler uses the configurator to configure the LLC (e.g., via the caching policy module) to enable the lock area (“Lock enable”), disable the lock area (“Lock disable”), and the size of the lock area, i.e., the number of paths (“Way”) shown in the diagram (e.g., Way0 to Way7).

[0085] Step 2: The task scheduler sends the task kernel to the IPU;

[0086] Step 3: The IPU retrieves the lock window flag from the parameter table, reads and configures the lock window. In one implementation scenario, this parameter table can be configured by software and stored at an off-chip address in Dynamic Random Access Memory (DRAM). The task scheduler then sends this address to the IPU, which can then read the parameter table to configure the lock window.

[0087] Step 4: The IPU generates latch-related requests through the Memory Management Unit (SMMU), and when sending the request to the LLC's caching strategy module, it can make the request include the lock attribute based on the lock window information.

[0088] Step 5: After receiving a latch-related request with the lock attribute, the LLC's caching strategy module stores the corresponding data in the corresponding cache line and marks the lock attribute of the cache line (i.e., the latch area), for example, by setting it to "persisting" as mentioned above.

[0089] LLC deregistration (or release) process:

[0090] Step 6: The task scheduler sends the kernel to the IPU;

[0091] Step 7: The IPU obtains the unlock window identifier from the parameter table, reads and configures the unlock window;

[0092] Step 8: When the IPU sends a request, it attaches the unlock attribute based on the unlock window information;

[0093] Step 9: After receiving a request with the unlock attribute, the LLC's caching strategy module switches the cache line with the hit lock attribute to a normal attribute, such as the normal attribute described above in conjunction with the instruction pattern.

[0094] Step 10: The task scheduler disables the latch area (i.e., LLC lockdisable) using the configurator and the CLR module. In one implementation scenario, the CLR module can clear the previous locking attribute configuration based on the configurator's instructions.

[0095] The above combination Figure 9 The latching scheme for the system-on-chip in window mode disclosed herein is described in detail. This latching operation significantly improves the cache hit probability, enhances the efficiency of the cache memory, and expands the application scenarios.

[0096] This disclosure also supports latch-related operations in streaming mode. When the enable bit corresponding to the data stream in the task of this disclosure is low, it is considered the default situation, that is, latch-related operations in streaming mode are not executed. Conversely, when the enable bit is high, the corresponding latch-related operations can be performed on the data stream in streaming mode. Specifically, the window mode and streaming mode of this disclosure have similar operations. A hash algorithm and the latch ratio of the data stream can be used to select a predetermined proportion of data from the data stream as the aforementioned partial data to be stored in the latch area. The specific use of the hash algorithm will be discussed later in conjunction with the appendix. Figure 11Let me describe it in detail.

[0097] As mentioned above, in one embodiment, this disclosure also supports latch-related operations in page mode. The following describes... Figure 10 Use this to describe the page pattern.

[0098] Figure 10 This is a schematic block diagram illustrating a page mode according to an embodiment of the present disclosure. Figure 10 As shown, according to the solution of this application, cache pages can be directly configured to have the locking attributes of this disclosure, thereby enabling cache pages mapped to memory (such as "memory") to be used for shared access to data among multiple kernels (kernels 0-2 as shown in the figure). In one embodiment, the programmer can use instructions (such as Malloc) to mark the cache page with the locking attribute. When a kernel accesses a cache page marked as locked, the SMMU can lock the data corresponding to the cache page in the latch area of ​​this disclosure. Then, when a subsequent kernel needs to access the aforementioned cache page again, it can read the previously locked data from the corresponding cache line in the latch area, thereby achieving a cache hit. Thus, through the page model, the solution of this disclosure improves the sharing and accessibility of data among multiple kernels.

[0099] Specifically, in page mode, the software driver can directly configure the information in the page table ("System Memory Management Unit", abbreviated as "SMMU") via instructions, and determine whether to perform page-based latching operations or normal operations based on this information. When the information in the page table indicates that the SMMU is bypassed, it means that there is no need to latch the cache memory, and the attributes of the cache lines in the cache memory can be normal. When the information indicates that the SMMU is linearly mapped, page-based latching operations can be set according to the SMMU linear mapping window configuration. For example, the data corresponding to the cache pages in this linear mapping window is locked in the latch area of ​​this disclosure. The SMMU can generate corresponding latch-related requests based on the information in the page table and send the latch-related requests to the LLC. The LLC's cache policy module can configure the LLC's cache lines according to the latch-related requests to perform corresponding cache-related operations.

[0100] In one embodiment, this disclosure also supports instruction mode, in which the system-on-a-chip can configure the latch area in LLC through memory access instructions (IO instructions) in the instruction set.

[0101] For example, I / O instructions may include at least one configuration field with latch-related attributes, thereby allowing for flexible configuration of the LLC. Here, various configuration fields can represent the corresponding operations that the LLC can perform when accessing data from off-chip memory (e.g., DDR space). In one implementation scenario, the instruction includes the aforementioned configuration attributes: Transient, Lock, Unlock, Normal, Invalid, Clean, or Default, etc. Since the instruction mode has the highest priority, when the I / O access instruction is specified with the Default attribute, it means that latch-related operations can be performed by other modes (such as window mode, stream mode, or page mode).

[0102] By attaching the aforementioned exemplary configurable attributes to the latch-related request, the solution disclosed herein can perform corresponding latch-related operations in instruction mode based on these attached attributes.

[0103] When the task scheduler sends a task to the Intelligent Processor (IPU), the IPU can determine latch-related requests based on the I / O instructions in the task. Specifically, when the Lock attribute configuration field in the I / O instruction is enabled, the Lock attribute can be attached to the latch-related request, allowing the LLC to store specific data in the locked area based on the latch-related request with the Lock attribute. When the Unlock attribute configuration field in the I / O instruction is enabled, the Unlock attribute can be attached to the latch-related request, allowing the LLC to release the locked area based on the latch-related request with the Unlock attribute. Depending on the application scenario, other attributes can also be attached to the latch-related requests similarly.

[0104] Furthermore, in some operational scenarios, the instruction may include a specific configuration field indicating the latch ratio. When this specific configuration field (e.g., the specific bit `inst_ratio_en`) is low, the latch operation can be considered to depend on the instruction configuration, i.e., the latch-related request is determined based on the specific I / O instruction in the task. If the aforementioned bit is high, a hash algorithm can be used to compare the data with the latch ratio indicated by the instruction, and a predetermined proportion of data can be selected from the data stream as the aforementioned partial data to be stored in the latch area. The specific use of the hash algorithm is discussed below in conjunction with the appendix. Figure 11 Let me describe it in detail.

[0105] Figure 11This illustrates hash operations in window mode or streaming mode according to embodiments of this disclosure. The scheme of this disclosure uses hash operations to perform a certain proportion of residency (i.e., locking) because one of the key issues of LLC residency is the tradeoff between bandwidth and capacity. Therefore, this disclosure proposes to perform a certain proportion of residency (i.e., Lock Ratio) to obtain different bandwidths and residency capacities for different tasks. Assuming a preset Lock Ratio value of P (e.g., as a percentage), the expected bandwidth is B = 6T*P + 2T*(1-P), where 6T is the read rate of data residing on the LLC, and 2T is the read rate of data stored in memory (e.g., DRAM), where T = 1000 Gbit / s. As mentioned above, the Lock Ratio can be configured in the lock / unlock window or for a specific data stream. Furthermore, although hash operations in window mode or streaming mode are described below, similar operations also apply to hash operations in instruction mode.

[0106] Specifically, in window mode or streaming mode, the intelligent processor core first compares the memory access address of the data with the address range defined by the locked window to determine whether the requested address is within the address range of the locked window. If the requested address is within the address range of the locked window, then... Figure 11 As shown, a hash operation can be performed on the hit window address range. Here, the memory access address for each piece of data can be a virtual address.

[0107] Specifically, using globally fixed hash rules, the VA of the memory access address can be mapped to the hash space (i.e., the "Hash Map" in the diagram), and this hashing process prioritizes retaining the lower-order bits of the address. Next, the hash value obtained at 1102 is compared with the latch ratio at 1104 to randomly select data of the corresponding proportion. Specifically, when the hash value of the memory access address is less than the latch ratio, it is considered a hit, and therefore this portion of data (i.e., data matching the ratio) can be latched into the cache. Conversely, when the hash value of the memory access address is greater than or equal to the latch ratio, it is considered a miss, and therefore this portion of data will not be latched into the cache.

[0108] For example, when the Lock Ratio is set to 10%, the data corresponding to the top 10% of hash values ​​can be selected sequentially for locking operations. This means data whose latch addresses have hash values ​​less than the Lock Ratio can be latched. In other examples, the Lock Ratio can be other values, customizable by the software user, and the selection operation can be implemented based on the hash algorithm settings. For example, the Lock Ratio could be 20% to 30%, in which case the data corresponding to the top 20% to 30% of hash values ​​can be selected sequentially for locking operations. Afterward, at point 1106, the data can be processed according to the specified request type, i.e., locking or unlocking a portion of the data.

[0109] The above combination Figures 6-11 The latching scheme for the cache memory disclosed herein is described in detail. Based on the aforementioned latching scheme and as a supplement to it, the following will combine... Figures 12-14 This disclosure describes another extended application of cache memory, namely how to use cache buffers to achieve communication between clusters within a system-on-a-chip.

[0110] Figure 12 This is a simplified block diagram illustrating a system-on-a-chip according to an embodiment of the present disclosure. In conjunction with the foregoing description, it will be understood that the system-on-a-chip here may include... Figure 2 The system-on-a-chip (SoC) in the computing device 201 shown is, for example, a SoC composed of a multi-core computing device 41. Figure 6 As shown, the system-on-chip 1200 includes four clusters 0-4, as exemplarily illustrated. Since the clusters have already been described in detail above, they will not be repeated here. Further shown is a cache memory 1201, which may, for example, be configured as described above. Figure 5 The SRAM 408 shown is used to perform inter-cluster data transfer operations. In one implementation scenario, the cache memory 1201 can also perform bidirectional on-chip and off-chip communication with DRAM (e.g., DDR), including the transfer of various types of data or instructions.

[0111] Figure 13 This is a flowchart illustrating a method 1300 for a system-on-a-chip according to an embodiment of the present disclosure. The system-on-a-chip here can be, for example... Figure 12The system-on-a-chip (SoC) shown is specifically designed to include at least multiple clusters for performing computational operations and a cache memory interconnected with these clusters. In one implementation, each cluster may include multiple processor cores for performing the computational operations. In another implementation, the latch area defined in the cache memory can be used to complete data communication between clusters, thereby eliminating the need for communication modules such as CDMA410 and GDMA411 in the SoC.

[0112] In one implementation, the latch area described above can be used to transfer data between dependent tasks, for example, between a producer kernel and a consumer kernel. Specifically, the processor can latch data that the producer kernel needs to exchange with the consumer kernel in the LLC using a configured locking window. In one scenario, after the processor finishes executing the producer kernel, it can latch the data (which may be the producer kernel's input or output data) that needs to be passed to the consumer kernel. Therefore, the processor can perform the latch-related operations of this disclosure on the LLC using a configured locking window and, for example, an SMMU, as described above, thereby latching the aforementioned data to be exchanged in the LLC in window mode for later use by the consumer kernel. Correspondingly, the processor can also release the latch area according to an unlocking window configured in the consumer kernel; that is, when the processor completes the execution of the consumer kernel by performing a read operation on the data latched in the LLC, it can release the corresponding storage space of the data in the latch area of ​​the LLC.

[0113] Based on the aforementioned latch area, which can be configured to transfer data between dependent tasks, this latch area can also be used in inter-chip communication applications. For example, data (which may be data that a producer kernel needs to exchange with a consumer kernel) can be transmitted from one cluster or processor core of the processor to processors in other clusters via the latch area for merging processing. Processors in other clusters read data from the latch area for processing, thereby realizing inter-chip data transfer. Specific details on how this latch area is used for inter-cluster communication are described below.

[0114] like Figure 13 As shown, this disclosure also includes a method for inter-cluster communication using a latch area of ​​a cache memory, the method comprising:

[0115] At step S1302, a specified storage space of the off-chip memory is mapped to a given storage area of ​​the cache memory (“cache”) (whose physical properties are the same as the locked area described above in conjunction with the accompanying drawings), so that the given storage area can be used as a cluster storage area for inter-cluster data communication. Figure 8In one implementation scenario shown, the cache memory may include an LLC, and the off-chip memory includes DDR. Based on this, the specified storage space can be... Figure 14 The storage space specified at location 1402. Correspondingly, the cluster storage area can be... Figure 14 A given storage area in the 1404-bit cache memory. In one implementation scenario, a specific storage space in the DDR can be specified via software configuration, and this specified DDR storage space can be mapped to a given space on the cache for use between clusters (e.g., Figure 14 Cluster 0 and Cluster 1 (shown in the diagram) communicate. After the cluster storage area is partitioned and determined, in step S1304, the determined cluster storage area can be used to perform cluster operations.

[0116] In one embodiment, using a clustered storage area to perform cluster operations may include using the clustered storage area for inter-cluster communication. Specifically, using the clustered storage area for inter-cluster communication may include using the clustered storage area to implement point-to-point communication between clusters. Additionally, the clustered storage area may be used to implement broadcast communication from one of the plurality of clusters to the remaining clusters. In a point-to-point communication scenario, the clustered storage area may be used to receive write operations from a first cluster for written data and, in response to read operations from a second cluster, to send previously written data from the first cluster to the second cluster.

[0117] In one example implementation of the write operation described above, the cluster storage area can also be used to receive a lock indication that the write data associated with the write operation resides in the cluster storage area, for example... Figure 14 The write lock shown is the lock-related request with the Lock attribute mentioned above. Next, the write data can be retained in the cluster storage area based on the lock indication, where the cluster storage area can be the latch area determined in the above embodiments. This retention method can significantly improve the hit rate of data that will be read multiple times in the cache memory.

[0118] In one implementation scenario, the producer kernel executing in one of the clusters can use the write locking described above to lock the data that needs to be exchanged with the consumer kernel in the LLC, so that the consumer kernel can use it later. For example, the producer kernel can transmit data to processors in other clusters via the LLC for merging processing. Processors in other clusters can then read the data from the cluster's memory for processing, thereby achieving inter-chip data transfer.

[0119] In one example implementation of the above read operation, the clustered storage area can also be used to receive a read invalidation indication that prevents the written data from being written back to off-chip memory, for example from... Figure 14The read invalidation (“read invalid”) is issued by cluster 1. This read invalidation indication can be a latch-related request with an invalid attribute; the generation method of this latch-related request is described above. The latch-related request can differ depending on the latch mode. Then, after sending the write data to cluster 1, the cluster storage area can invalidate the cache line associated with the write data based on the read invalidation indication.

[0120] To achieve synchronization of data transfer (or communication) between the aforementioned clusters, a cluster writing data to the cluster storage area (such as cluster 0) can send a synchronization command to another cluster (such as cluster 1) after the write operation is completed, for example... Figure 14 The `hsem` ("hardware semaphore") is used in this context. Upon receiving a synchronization command, Cluster 1 can send the aforementioned read invalidation request to the cluster storage area. This invalidates the cache line after reading data written into the cluster storage area by Cluster 0, thereby preventing the aforementioned data from being written back.

[0121] In the context of this disclosure, the aforementioned actions of writing data to and reading data from the cluster storage area can be collectively referred to as latch-related operations triggered by latch-related requests. The confirmation method for these latch-related requests is described above. Specifically, the latch-related request can be used to instruct a latch operation. Through the latch operation, data is latched in the cluster storage area for subsequent multiple uses. Furthermore, the latch-related request can be used to instruct a release operation, through which data can be unlocked from the cluster storage area to free up more storage space for subsequent data latching. It is understood that through the release operation, the storage space of the cluster storage area can be used flexibly, thereby improving the utilization efficiency of the cluster storage area disclosed in this disclosure.

[0122] In one implementation, for read operations in the cluster storage area, after the read operation is completed, the data or a selected portion of the data can be released from a designated area of ​​the cluster storage area according to a latch-related request. Regarding the aforementioned selected portion of data, in one embodiment, a predetermined proportion of data can be randomly selected from the data to form the aforementioned portion of data to be latched in the latch area. In another embodiment, a hash algorithm can be used to select a predetermined proportion of data from the data as the aforementioned portion of data to be latched in the cluster storage area; see the above for details. Figure 11 Partial description.

[0123] The solutions disclosed herein have been described in detail above with reference to the accompanying drawings. Depending on the application scenario, the electronic devices or apparatus disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus disclosed herein can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal applications. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present disclosure can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0124] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

[0125] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or apparatus embodiments described above, this document divides them based on logical functions, but in actual implementation, there may be other division methods. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationships between different units or components, the connections discussed above in conjunction with the accompanying drawings can be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect couplings involve communication connections utilizing interfaces, where the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0126] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this disclosure. Additionally, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically independently.

[0127] In some implementation scenarios, the integrated unit described above can be implemented as a software program module. If implemented as a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable storage device (CMSDD). Therefore, when the disclosed solution is embodied in a software product (e.g., a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to cause a computer device (e.g., a personal computer, server, or network device) to execute some or all of the steps of the method described in the embodiments of this disclosure. The aforementioned memory may include, but is not limited to, various media capable of storing program code, such as USB flash drives, flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0128] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the hardware structure of the circuit may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any suitable storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), such as resistive random access memory ("RRAM"), dynamic random access memory ("DRAM"), static random access memory ("SRAM"), enhanced dynamic random access memory ("EDRAM"), high bandwidth memory ("HBM"), hybrid memory cube ("HMC"), ROM, and RAM, etc.

[0129] The foregoing can be better understood in accordance with the following terms:

[0130] Clause A1. A method for using a cache memory, comprising:

[0131] Configure a specific storage space in the cache memory as a latch area that supports multiple latching modes;

[0132] Receive a latch-related request to perform latch-related operations on the data in the latch area; and

[0133] Based on the latch-related request, latch-related operations are performed on the data in the latch area according to the corresponding latch mode.

[0134] Clause A2. The method according to Clause A1, wherein the multiple latching modes are executed in a predetermined priority order.

[0135] Clause A3. The method according to Clause A1 or 2, wherein the multiple latching modes include an instruction mode that performs latching-related operations based on hardware instructions, a window mode that performs latching-related operations based on window attributes, a stream mode that performs latching-related operations based on data streams, and / or a page mode that performs latching-related operations based on cached pages.

[0136] Clause A4. The method described in Clause A3, wherein,

[0137] In the instruction mode, the latch-related request is determined based on the hardware instruction;

[0138] In the page mode, the latch-related requests are determined based on the cache page configuration;

[0139] In the window mode or stream mode, the latch-related request is determined based on the locked window.

[0140] Clause A5. The method according to Clause A4, wherein in the instruction mode, the window mode, or the stream mode, the latch-related request can be accompanied by a locking attribute, the locking attribute being used to indicate that specific data is to reside in the latch area, the specific data being a subset of data selected according to a hash algorithm.

[0141] Clause A6. The method described in accordance with Clause A3 or A4, wherein, in page mode, the method comprises:

[0142] The latching operation is performed based on the linear mapping window of the system memory management unit.

[0143] Clause A7. The method according to Clause A3, wherein configuring the latch area to support the multiple latch modes includes:

[0144] According to one of the received configuration instructions, a specific storage space is configured to support a latch area of ​​a corresponding latch mode, wherein the configuration instruction includes configuration items for enabling latch area, disabling latch area and / or latch area size.

[0145] Clause A8. The method according to Clause A7, wherein for a write operation to a latch, the method includes latching the data or a selected portion of the data in a designated area of ​​the latch in accordance with the latch-related request for use in subsequent multiple reads.

[0146] Clause A9. The method according to Clause A7, wherein for a read operation of a latch, the method includes, after the read operation is performed, releasing the data or a selected portion of the data from a designated area of ​​the latch according to the latch-related request.

[0147] Clause A10. A cache memory comprising:

[0148] A configuration module is used to configure a specific storage space in the cache memory as a latch area that supports multiple latching modes;

[0149] The latch execution module is used for:

[0150] Receive a latch-related request to perform latch-related operations on the data in the latch area; and

[0151] Based on the latch-related request, latch-related operations are performed on the data in the latch area according to the corresponding latch mode.

[0152] Clause A11. A system-on-a-chip, comprising:

[0153] The cache memory as described in Clause A10; and

[0154] Processor, used to generate the latch-related request;

[0155] The latch execution module of the cache memory is used to perform latch-related operations on the data in the latch area according to the latch-related request and the corresponding latch mode.

[0156] Clause A12. The system-on-a-chip according to Clause A11, wherein the latch mode includes an instruction mode, and in the instruction mode, the processor is configured to generate the latch-related request based on a received hardware instruction.

[0157] Clause A13. The on-chip system according to Clause A11, wherein the latch mode includes a page mode, and in the page mode, the processor is configured to generate the latch-related request based on the cache page configuration.

[0158] Clause A14. The on-chip system according to Clause A11, wherein the latching mode includes the window mode or the streaming mode, and in the window mode or streaming mode, the on-chip system further includes: a task scheduler, which includes a configurator and a scheduling unit, wherein:

[0159] The configurator is used to generate the configuration instructions according to the assigned configuration task, so as to send them to the configuration module of the cache memory; and

[0160] The scheduling unit is used to schedule multiple tasks in the task scheduler so that they can be sent to the processor core.

[0161] Clause A15. The system-on-a-chip as described in Clause A14, wherein the configuration instructions include configuration items for enabling latches, disabling latches, and / or latch size.

[0162] Clause A16. The system-on-a-chip as described in Clause A15, wherein said processor further includes a system memory management unit for use in window mode or streaming mode:

[0163] Configure the locking window associated with the data to be latched based on the parameter table; and

[0164] The latch-related request is generated based on the configured lock window.

[0165] Clause A17. The on-chip system according to Clause A16, wherein the configuration items for the locking window include one or more of the following:

[0166] The base address and size of the window, wherein the base address of the window corresponds to the starting address of the data to be latched, and the size of the window corresponds to the size of the data;

[0167] A latch instruction that latches data in the latch area;

[0168] An unlock instruction that unlocks data from the latch area; and

[0169] The latch ratio indicates the proportion of data that will be actually latched out of the data to be latched for latch-related operations.

[0170] Clause A18. The system-on-a-chip according to Clause A17, wherein the processor is further configured to use a hash algorithm to select a portion of the data that can be latched in the latch area when the memory access address of the data to be latched is within the address range of the latching window.

[0171] Clause A19. The system-on-a-chip according to Clause A17, wherein the processor is configured to randomly select a portion of data satisfying a predetermined latch ratio from the data to be latched according to a hash algorithm, and generate a latch-related request with locking attributes for latching in the latch area.

[0172] Clause A20. The system-on-a-chip according to Clause A14, wherein the processor is configured to perform a write operation on the data in the latch area, the latch execution module is configured to latch the data or a selected portion of the data in a designated area of ​​the latch area according to the latch-related request, and wherein the processor is further configured to perform a read operation on the data in the latch area, and the latch execution module is configured to release the data after the read operation is completed from the designated area of ​​the latch area according to the latch-related request.

[0173] Clause A21. The on-chip system according to any one of Clauses A16-A20, wherein the task comprises a producer kernel and a consumer kernel, wherein:

[0174] When executing the producer kernel, the processor is used to latch the data output by the producer kernel into the latch area via the latch-related request, for use by the consumer kernel; and

[0175] When executing the consumer kernel, the processor is used to read data from the latch area, and after reading the data, to unlock the data from the latch area through the latch-related request, so as to release the storage space in the latch area used for the data.

[0176] Clause A22. A board comprising a system-on-a-chip as described in any one of Clauses A11-A21.

[0177] Clause A23. A computing device comprising a board as described in Clause A22.

[0178] While numerous embodiments of this disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and intent of this disclosure. It should be understood that various alternatives to the embodiments of this disclosure described herein may be employed in the practice of this disclosure. The appended claims are intended to define the scope of this disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for a cache memory, comprising: Configure a specific storage space in the cache memory as a latch area that supports multiple latching modes; Receive latch-related requests for performing latch-related operations on data in the latch area; as well as According to the latch-related request, perform latch-related operations on the data in the latch area in the corresponding latch mode; The various latching modes mentioned above have a predetermined priority order; When a high-priority latching mode is disabled, a low-priority latching mode is used.

2. The method according to claim 1, wherein the multiple latching modes include an instruction mode that performs latching-related operations based on hardware instructions, a window mode that performs latching-related operations based on window attributes, a stream mode that performs latching-related operations based on data streams, and / or a page mode that performs latching-related operations based on cached pages.

3. The method according to claim 2, wherein, In the instruction mode, the latch-related request is determined based on the hardware instruction; In the page mode, the latch-related requests are determined based on the cache page configuration; In the window mode or stream mode, the latch-related request is determined based on the locked window.

4. The method of claim 3, wherein in the instruction mode, the window mode, or the stream mode, the latch-related request may include a locking attribute, the locking attribute being used to indicate that specific data is to be stored in the latch area, the specific data being a subset of data selected according to a hash algorithm.

5. The method according to claim 2 or 3, wherein in page mode, the method comprises: The latching operation is performed based on the linear mapping window of the system memory management unit.

6. The method of claim 2, wherein configuring the latch area to support the multiple latch modes comprises: Configure a specific storage space as a latch area that supports a corresponding latch mode according to one of the received configuration instructions; The configuration instructions include configuration items for enabling latches, disabling latches, and / or latch size.

7. The method of claim 6, wherein for a write operation to the latch area, the method includes latching the data or a selected portion of the data in a designated area of ​​the latch area according to the latch-related request, for use in subsequent multiple reads.

8. The method of claim 6, wherein for a read operation of the latch area, the method includes, after the read operation is performed, releasing the data or a selected portion of the data from a designated area of ​​the latch area according to the latch-related request.

9. A cache memory, comprising: A configuration module is used to configure a specific storage space in the cache memory as a latch area that supports multiple latching modes; The latch execution module is used for: Receive latch-related requests for performing latch-related operations on data in the latch area; as well as According to the latch-related request, latch-related operations are performed on the data in the latch area in the corresponding latch mode, wherein the multiple latch modes have a predetermined priority order; When a high-priority latching mode is disabled, a low-priority latching mode is used.

10. A system-on-a-chip, comprising: The cache memory according to claim 9; as well as Processor, used to generate the latch-related request; The latch execution module of the cache memory is used to perform latch-related operations on the data in the latch area according to the latch-related request and the corresponding latch mode, wherein the multiple latch modes have a predetermined priority order. When a high-priority latching mode is disabled, a low-priority latching mode is used.

11. The system-on-a-chip of claim 10, wherein the latch mode includes an instruction mode, and in the instruction mode, the processor is configured to generate the latch-related request according to a received hardware instruction.

12. The system-on-a-chip of claim 10, wherein the latch mode includes a page mode, and in the page mode, the processor is configured to generate the latch-related request based on a cache page configuration.

13. The system-on-chip of claim 10, wherein the latching mode includes a window mode or a stream mode, and in the window mode or stream mode, the system-on-chip further includes: The task scheduler includes a configurator and a scheduling unit, wherein: The configurator is used to generate configuration instructions based on the assigned configuration task, for sending to the configuration module of the cache memory; and The scheduling unit is used to schedule multiple tasks in the task scheduler so that they can be sent to the processor.

14. The system-on-a-chip according to claim 13, wherein the configuration instructions include configuration items for enabling latches, disabling latches, and / or latch size.

15. The system-on-a-chip of claim 14, wherein the processor further comprises a system memory management unit configured to: in window mode or stream mode: Configure the locking window associated with the data to be latched based on the parameter table; and The latch-related request is generated based on the configured lock window.

16. The system-on-a-chip according to claim 15, wherein the configuration items of the locking window include one or more of the following: The base address and size of the window, wherein the base address of the window corresponds to the starting address of the data to be latched, and the size of the window corresponds to the size of the data; A latch instruction that latches data in the latch area; An unlock instruction that unlocks data from the latch area; and The latch ratio indicates the proportion of data that will be actually latched out of the data to be latched for latch-related operations.

17. The system-on-a-chip according to claim 16, wherein the processor is further configured to: If the memory access address of the data to be latched is within the address range of the locking window, a hash algorithm is used to select the portion of data that can be latched in the latch area.

18. The system-on-a-chip of claim 16, wherein the processor is configured to randomly select a portion of the data to be latched that satisfies the predetermined latch ratio from the data to be latched according to a hash algorithm, and generate a latch-related request with locking attributes for latching in the latch area.

19. The system-on-a-chip of claim 13, wherein the processor is configured to perform a write operation on the data in the latch area, the latch execution module is configured to latch the data or a selected portion of the data in a designated area of ​​the latch area according to the latch-related request, and wherein the processor is further configured to perform a read operation on the data in the latch area, and the latch execution module is configured to release the data after the read operation is completed from the designated area of ​​the latch area according to the latch-related request.

20. The system-on-a-chip according to any one of claims 15-19, wherein the task comprises a producer kernel and a consumer kernel, wherein: When the producer kernel is executed, the processor is used to latch the data output by the producer kernel into the latch area through the latch-related request, so that the consumer kernel can use it; as well as When executing the consumer kernel, the processor is used to read data from the latch area, and after reading the data, to unlock the data from the latch area through the latch-related request, so as to release the storage space in the latch area used for the data.

21. A board comprising a system-on-a-chip according to any one of claims 10-20.

22. A computing device comprising the board according to claim 21.

Images