Cache, data processing apparatus and chip

Abstract

The present disclosure provides a cache, comprising at least two layer structures, each of the at least two layer structures comprising a first cache unit and a second cache unit. First cache access instructions of different layer structures enter different pipelines to access the first cache units in the corresponding layer structures, and second cache access instructions of different layer structures enter different pipelines to access the second cache units in the corresponding layer structures.

Description

Cache, data processing device and chip

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to Chinese patent application 202310946874.5 filed with the State Intellectual Property Office of China on July 28, 2023, and the disclosure of this Chinese patent application is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to the field of computer processing technology, and in particular to a cache, a data processing device, and a chip. Background Art

[0004] In the CPU (Central Processing Unit) microarchitecture, instruction execution and cache access pass through a four-stage pipeline, with each stage taking up one clock cycle. With a limited execution pipeline, CPU cache capacity is constrained by frequency increases and power consumption optimization, becoming a major bottleneck restricting CPU performance.

[0005] Summary of the Invention

[0006] The present disclosure provides a cache, a data processing device and a chip.

[0007] In a first aspect, an embodiment of the present disclosure provides a cache comprising at least two layer structures, each of the at least two layer structures comprising: a first cache unit and a second cache unit, first cache access instructions of different layer structures enter different pipelines to access the first cache unit within the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit within the corresponding layer structure.

[0008] On the other hand, an embodiment of the present disclosure further provides a data processing device, which includes the cache as described above.

[0009] On the other hand, an embodiment of the present disclosure further provides a chip, which includes the cache as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a schematic diagram of a cache access instruction entering a pipeline in the related art;

[0011] FIG2 is a first schematic diagram of the structure of a cache provided by an embodiment of the present disclosure;

[0012] FIG3 is a second schematic diagram of the structure of a cache provided by an embodiment of the present disclosure;

[0013] FIG4 is a schematic diagram of a cache including a two-layer structure provided by an example of the present disclosure;

[0014] FIG5 is a schematic diagram of a cache including a three-layer structure provided by an example of the present disclosure;

[0015] FIG6 is a schematic diagram of the structure of a data processing device and a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of this disclosure to those skilled in the art.

[0017] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0018] The terms used herein are used only to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a," "an," and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements, and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof is not excluded.

[0019] The embodiments described herein may be described with reference to plan views and / or cross-sectional views, with the aid of idealized schematic diagrams of the present disclosure. Thus, the example illustrations may be modified based on manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to the embodiments shown in the accompanying drawings, but include modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the accompanying drawings are schematic in nature, and the shapes of the regions shown in the drawings illustrate specific shapes of the regions of the elements, but are not intended to be limiting.

[0020] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0021] Figure 1 is a schematic diagram of the cache access instruction entering the pipeline in the related art. As shown in Figure 1, pipelines D1 to D4 are a four-stage pipeline for CPU instruction execution and cache access, and each stage of the pipeline is one clock cycle. The cache includes a physical address memory (Tag Ram), a data memory (Data Ram), a data merging and format conversion unit (Data Merge & Format Conversion), and an interface unit (Interface). The physical address memory is a physical module for storing physical addresses and cache status information. The data memory is a physical module for storing data. The data merging and format conversion unit is used to perform data merging and format conversion. The interface unit is used to transmit the data after merging and format conversion to other devices or modules. The cache unit of the cache includes two parts, namely, the physical address memory and the data memory. The process of a load instruction or a store instruction entering the cache pipeline during execution is shown in Figure 1, including stages D1 to D4.

[0022] In D1: the physical address memory is read according to the cache access instruction Tag Read.

[0023] In D2: the data memory is read according to the cache access instruction Data Read.

[0024] In D3: according to the physical address information output by the physical address memory, the data output by the data memory is selected, and operations such as data merging and format conversion are performed on the data output by the data memory.

[0025] In D4: the data after merging and format conversion is sent to the interface unit.

[0026] The high-performance CPU mainly aims to achieve 3GHz, so the four-stage pipeline (D1 to D4) has the following bottlenecks: in D1, the physical address memory has more input signals, which limits the frequency increase; in D2, the physical address memory has a large output load, which limits the frequency increase; in D3, the data memory has a large output load and needs to be merged and format converted (the number of logical levels is large), which limits the frequency increase; in D4, the data is finally output to the interface, the bit width is large, the layout and wiring distance is long, and the frequency increase is limited.

[0027] Therefore, when the execution pipeline is limited, the CPU cache capacity is restricted by frequency increase and power consumption optimization, becoming one of the main bottlenecks restricting CPU performance improvement.

[0028] To address the above issues, the disclosed embodiments improve the cache structure and pipeline to expand cache capacity. The cache includes at least two layers, each layer including a first cache unit and a second cache unit. First cache access instructions from different layers enter different pipelines to access the first cache unit within the corresponding layer, and second cache access instructions from different layers enter different pipelines to access the second cache unit within the corresponding layer.

[0029] Figure 2 is a schematic diagram of the cache structure of an embodiment of the present disclosure. As shown in Figure 2, the number of layer structures is n, where n ≥ 2. This means that the cache can include a first layer structure R1, a second layer structure R2, ..., and an nth layer structure Rn, each layer structure containing the same elements and structure. The first layer structure R1 includes a first cache unit M11 and a second cache unit M12, the second layer structure R2 includes a first cache unit M21 and a second cache unit M22, ..., and the nth layer structure Rn includes a first cache unit Mn1 and a second cache unit Mn2.

[0030] Simply expanding cache capacity will prevent timing convergence (frequency reduction) and performance improvement. By enabling cache access instructions in each layer to independently access their respective cache units, the frequency reduction issue caused by independent cache expansion can be effectively addressed. Furthermore, the number of expanded layers (n-1) can be customized based on actual application requirements. The total cache capacity after expansion is n times that of a single-layer structure, enhancing the flexibility and universality of this structure's product applications.

[0031] The cache provided by the embodiment of the present disclosure includes at least two layers, each layer including a first cache unit and a second cache unit. First cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure. The cache of the embodiment of the present disclosure includes multiple layers, which can expand cache capacity, improve data hit rate, and enhance overall CPU performance. Moreover, the cache access instructions of each layer structure independently access their respective cache units, which can ensure cache processing efficiency and optimize power consumption.

[0032] In some embodiments, the pipelines that the first cache access instruction of each layer enters are sequentially numbered, and the pipelines that the second cache access instruction of each layer enters are sequentially numbered. In other words, each layer can be staggered in the order of the pipelines.

[0033] To further increase the frequency, two adjacent layer structures can be staggered by one pipeline. That is, the first cache access instruction C21 of the second layer structure enters pipeline D2, and the first cache access instruction C11 of the first layer structure enters pipeline D1, and D2 is the pipeline next to D1; the second cache access instruction C22 of the second layer structure enters pipeline D3, and the second cache access instruction C12 of the first layer structure enters pipeline D2, and D3 is the pipeline next to D2. Therefore, in some embodiments, the pipeline entered by the first cache access instruction of the nth layer structure is the pipeline next to the pipeline entered by the first cache access instruction of the (n-1)th layer structure; the pipeline entered by the second cache access instruction of the nth layer structure is the pipeline next to the pipeline entered by the second cache access instruction of the (n-1)th layer structure.

[0034] In some embodiments, each layer structure further includes a data selection unit, a data processing unit (i.e., a data merging and format conversion unit), and an interface unit. The data selection unit of each layer structure is configured to select second data based on first data to obtain target data, wherein the first data is data obtained by accessing the first cache unit in the corresponding layer structure according to the first cache access instruction of the corresponding layer structure, and the second data is data obtained by accessing the second cache unit in the corresponding layer structure according to the second cache access instruction of the corresponding layer structure. In other words, for each layer structure, the second cache unit in the layer structure simultaneously outputs the physical addresses of multiple channels, compares them with the physical addresses output by the first cache unit in the layer structure, and selects the second cache unit channel output data that is equal to the comparison result as the target data. In addition, the data output by the second cache unit is merged with data from other sources, and then performs operations such as storage format conversion to register format, endian conversion, and sign conversion. The data processing unit is configured to merge the target data with third data (i.e., data from other sources), perform format conversion on the merged data to obtain fourth data, and send the fourth data to the interface unit.

[0035] As shown in Figure 3, the first layer structure R1 may further include a data selection unit S1, a data processing unit P1, and an interface unit I1, and the second layer structure R2 may further include a data selection unit S2, a data processing unit P2, and an interface unit I2. The nth layer structure Rn may further include a data selection unit Sn, a data processing unit Pn, and an interface unit In. Therefore, the processing process of a cache access instruction of a one-layer structure occupies four pipelines, the processing process of a cache access instruction of a two-layer structure occupies five pipelines, and the processing process of a cache access instruction of an n-layer structure occupies (n+3) pipelines.

[0036] In some embodiments, the first cache unit is a physical address memory, and the second cache unit is a data memory.

[0037] To clearly illustrate the technical solution of the embodiment of the present disclosure, the following description is made by taking a cache including a two-layer structure and a cache including a three-layer structure as examples in conjunction with FIG. 4 and FIG. 5 .

[0038] Figure 4 is a schematic diagram of a cache comprising two layers, provided by an example of the present disclosure. As shown in Figure 4, the cache comprises two layers, namely, a first layer structure R1 and a second layer structure R2. The first layer structure R1 comprises a physical address memory M11, a data memory M12, a data selection unit S1, a data merging and format conversion unit P1, and an interface unit I1. The second layer structure R1 comprises a physical address memory M21, a data memory M22, a data selection unit S2, a data merging and format conversion unit P2, and an interface unit I2. This doubles the overall cache capacity.

[0039] The first layer structure R1 and the second layer structure R2 are accessed independently, that is, the physical address read instruction 11 of the first layer structure accesses the physical address memory M11 in the pipeline D1, and the data read instruction 12 of the first layer structure accesses the data memory M12 in the pipeline D2; the physical address read instruction 21 of the second layer structure accesses the physical address memory M21 in the pipeline D2, and the data read instruction 22 of the second layer structure accesses the data memory M22 in the pipeline D3.

[0040] The first layer structure R1 and the second layer structure R2 are processed independently. Specifically, in pipeline D3, data from data memory M12 is selected and output by data selection unit S1, and then merged and format-converted by data merging and format conversion unit P1. Subsequently, in pipeline D4, the merged and format-converted data is output by interface unit I1. In pipeline D4, data from data memory M22 is selected and output by data selection unit S2, and then merged and format-converted by data merging and format conversion unit P2. Subsequently, in pipeline D5, the merged and format-converted data is output by interface unit I2.

[0041] After one-tier expansion, the cache capacity is double the capacity of the existing cache.

[0042] FIG5 is a schematic diagram of a cache including three layers provided by an example of the present disclosure. As shown in FIG5 , the cache includes three layers, namely, a first layer structure R1, a second layer structure R2, and a third layer structure R3. The first layer structure R1 includes a physical address memory M11, a data memory M12, a data selection unit S1, a data merging and format conversion unit P1, and an interface unit I1. The second layer structure R2 includes a physical address memory M21, a data memory M22, a data selection unit S2, a data merging and format conversion unit P2, and an interface unit I2. The third layer structure R3 includes a physical address memory M31, a data memory M32, a data selection unit S3, a data merging and format conversion unit P3, and an interface unit I3. Thus, the overall cache capacity is doubled.

[0043] The first layer structure R1, the second layer structure R2, and the third layer structure R3 are independently accessed in a staggered manner. That is, the physical address read instruction 11 of the first layer structure accesses the physical address memory M11 in pipeline D1, and the data read instruction 12 of the first layer structure accesses the data memory M12 in pipeline D2. The physical address read instruction 21 of the second layer structure accesses the physical address memory M21 in pipeline D2, and the data read instruction 22 of the second layer structure accesses the data memory M22 in pipeline D3. The physical address read instruction 31 of the third layer structure accesses the physical address memory M31 in pipeline D3, and the data read instruction 32 of the third layer structure accesses the data memory M31 in pipeline D4.

[0044] The first layer structure R1, the second layer structure R2, and the third layer structure R3 are processed independently. That is, in pipeline D3, data from data memory M11 is selected and output by data selection unit S1, and then merged and format converted by data merging and format conversion unit P1. Subsequently, in pipeline D4, the merged and format converted data is output by interface unit I1. In pipeline D4, data from data memory M22 is selected and output by data selection unit S2, and then merged and format converted by data merging and format conversion unit P2. Subsequently, in pipeline D5, the merged and format converted data is output by interface unit I2. In pipeline D5, data from data memory M32 is selected and output by data selection unit S3, and then merged and format converted by data merging and format conversion unit P3. Subsequently, in pipeline D6, the merged and format converted data is output by interface unit I3.

[0045] After two tiers of expansion, the cache capacity is three times the capacity of the existing cache.

[0046] An embodiment of the present disclosure further provides a data processing device, which includes the cache as described above.

[0047] In some embodiments, the data processing device includes but is not limited to: a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processor (NPU), and a data processing unit (DPU).

[0048] In some embodiments, the cache of the data processing device is a first-level cache, or the cache of the data processing device is a first-level cache and a second-level cache.

[0049] In the case where the data processing device is a CPU, as shown in Figure 6, the CPU may include a processing core Core, a level 1 cache L1, and a level 2 cache L2. L3 is a level 3 cache, which is provided in a SoC (System on Chip), with the level 1 cache L1, level 2 cache L2, and level 3 cache L3 interconnected in stages. It should be noted that a CPU may also include a processing core Core and a level 1 cache L1, but not a level 2 cache L2.

[0050] The relationship between the capacity of each level of cache and memory access latency is shown in Table 1:

[0051] Table 1

[0052] As can be seen in Table 1, in SoCs, the CPU's L1 cache has the shortest access latency and the best performance. Therefore, increasing the capacity of the CPU's L1 cache significantly improves SoC performance. When reading data, the first step is to search in the L1 cache. If the data is not found, the second cache is searched in the L2 cache. If the data is still not found, the third cache is searched in the L3 cache or in memory. Therefore, using the aforementioned extended layer structure for the L1 cache can significantly improve the performance of data processing devices and chips.

[0053] An embodiment of the present disclosure further provides a chip, which includes the data processing device as described above.

[0054] In some embodiments, the chip may further include a third-level cache. In the case of multiple data processing devices, each data processing device is connected to the third-level cache. It should be noted that the third-level cache may also be the cache described above.

[0055] As shown in Figure 6, the chip is a SoC, and the data processing device is a CPU. Each CPU includes a L1 cache and a L2 cache. The chip also includes a L3 cache. The L1 cache of each CPU is cascaded with the L2 cache of the same CPU, and the L2 cache of each CPU is cascaded with the L3 cache. The SoC is connected to the DRAM (dynamic random access memory) outside the chip through the L3 cache.

[0056] The cache of the disclosed embodiment can be applied to high-performance CPUs in computing-intensive fields such as cloud computing, scientific computing, and AI, and by increasing the cache capacity, the performance of the CPU, i.e., the chip, can be improved. It should be noted that the cache is not limited to high-performance CPU applications, and any CPU that uses a cache can be used. The multi-layered cache of the disclosed embodiment will not affect the high-frequency implementation of the CPU, and its scalability and applicability are strong. By increasing the cache capacity, the CPU performance improvement effect is significant.

[0057] It will be appreciated by those skilled in the art that all or some of the steps in the method disclosed above, and the functional modules / units in the device can be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0058] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly stated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present invention as set forth in the appended claims.

Claims

1. A cache comprising: At least two layers, Among them, each of the at least two layer structures includes: a first cache unit and a second cache unit, the first cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and the second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure.

2. The cache of claim 1, wherein: The sequence number of the pipeline entered by the first cache access instruction of each of the at least two layer structures increases sequentially, and the sequence number of the pipeline entered by the second cache access instruction of each of the at least two layer structures increases sequentially.

3. The cache of claim 1, wherein: The pipeline entered by the first cache access instruction of the nth layer structure among the at least two layer structures is the next pipeline of the pipeline entered by the first cache access instruction of the (n-1)th layer structure among the at least two layer structures; The pipeline entered by the second cache access instruction of the n-th layer structure is the pipeline next to the pipeline entered by the second cache access instruction of the (n-1)-th layer structure; Wherein, n is the number of the at least two layer structures.

4. The cache of claim 1, wherein: Each of the at least two layer structures further comprises: a data selection unit, a data processing unit and an interface unit, The data selection unit is used to select each second data according to the first data to obtain the target data, wherein the first data is the data obtained by accessing the first cache unit of the layer structure according to the first cache access instruction of the corresponding layer structure, and the second data is the data obtained by accessing the second cache unit of the layer structure according to the second cache access instruction of the layer structure; The data processing unit is used for combining the target data with the third data, and performing format conversion on the combined data to obtain fourth data, and sending the fourth data to the interface unit.

5. The cache according to any one of claims 1 to 4, wherein: The first cache unit is a physical address memory, and the second cache unit is a data memory.

6. A data processing device comprising the cache as claimed in any one of claims 1 to 5.

7. The data processing apparatus according to claim 6, wherein: The cache is a first-level cache, or the cache is a first-level cache and a second-level cache.

8. The data processing apparatus according to claim 6, wherein: The data processing device is at least one of a central processing unit, a graphics processing unit, an embedded neural network processor, and a data processing device.

9. A chip comprising the data processing device according to any one of claims 6 to 8.

10. The chip according to claim 9, further comprising a three-level cache, wherein the number of the data processing devices is plural, and the data processing devices are connected to the three-level cache.

Images