Cache, data processing device and chip

Abstract

The invention provides a cache which comprises at least two layer structures, each layer structure comprises a first cache unit and a second cache unit, first cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure. The second cache access instructions of different layer structures enter different assembly lines to access the second cache units in the corresponding layer structures; the cache comprises a plurality of layer structures, the cache capacity can be expanded, the data hit rate is increased, and the overall performance of a CPU is improved; moreover, the cache access instruction of each layer structure independently accesses the respective cache unit by mistake, so that the cache processing efficiency can be ensured. The invention further provides a data processing device and a chip.

Description

Technical Field

[0001] The present disclosure relates to the field of computer processing technology, and in particular to a cache, a data processing device and a chip. Background Art

[0002] In the CPU (Central Processing Unit) microarchitecture, CPU instruction execution and cache access have 4 pipelines, each pipeline is one clock cycle. In the case of limited execution pipelines, the CPU cache capacity is limited by frequency increase and power consumption optimization, becoming one of the main bottlenecks restricting CPU performance improvement. Summary of the invention

[0003] The present disclosure provides a cache, a data processing device and a chip.

[0004] In a first aspect, an embodiment of the present disclosure provides a cache comprising at least two layer structures, each of the layer structures comprising a first cache unit and a second cache unit, first cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure.

[0005] On the other hand, an embodiment of the present disclosure further provides a data processing device, comprising the cache as described above.

[0006] On the other hand, an embodiment of the present disclosure further provides a chip, comprising the cache as described above.

[0007] The cache provided by the embodiment of the present disclosure includes at least two layer structures, each layer structure includes a first cache unit and a second cache unit, first cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure; the cache of the embodiment of the present disclosure includes multiple layer structures, which can expand the cache capacity, improve the data hit rate, and enhance the overall CPU performance; moreover, the cache access instructions of each layer structure independently access their respective cache units, which can ensure the processing efficiency of the cache. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of a cache access instruction entering a pipeline in the related art;

[0009] Figure 2 Schematic diagram of the cache structure provided by the embodiment of the present disclosure Figure 1 ;

[0010] Figure 3 Schematic diagram of the cache structure provided by the embodiment of the present disclosure Figure 2 ;

[0011] Figure 4 A schematic diagram of a cache including a two-layer structure provided for a specific example of the present disclosure;

[0012] Figure 5 A schematic diagram of a cache including a three-layer structure provided for a specific example of the present disclosure;

[0013] Figure 6 A schematic diagram of the structure of a data processing device and a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0015] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0016] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0017] The embodiments described herein may be described with reference to plan views and / or cross-sectional views by means of idealized schematic diagrams of the present disclosure. Therefore, the example illustrations may be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to the embodiments shown in the accompanying drawings, but include modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the accompanying drawings have schematic properties, and the shapes of the regions shown in the figures illustrate the specific shapes of the regions of the elements, but are not intended to be limiting.

[0018] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0019] Figure 1FIG. 1 is a schematic diagram of cache access instructions entering the pipeline in the related art. Figure 1 As shown, pipeline D1-D4 is a 4-stage pipeline for CPU instruction execution and cache access, and each stage of the pipeline is one clock cycle. The cache includes Tag Ram (physical address memory), Data Ram (data memory), Data Merge & Format Convertion (data merging and format conversion unit) and Interface (interface unit). Tag Ram is a physical module for storing physical address and cache status information; Data Ram is a physical module for storing data; Data Merge & Format Convertion is used for data merging and format conversion; Interface is used to transmit the data after merging and format conversion to other devices or modules. The cache unit of the cache includes two parts, Tag Ram and Data Ram. During the execution of a Load instruction or a Store instruction, the process of entering the cache pipeline is as follows. Figure 1 As shown, it includes the following stages:

[0020] In D1: Tag Ram is read according to the cache access instruction Tag Read.

[0021] In D2: Data Ram is read according to the cache access instruction Data Read.

[0022] In D3: According to the physical address information output by Tag Ram, select the data output by Data Ram, and perform operations such as data merging and format conversion on the data output by Data Ram.

[0023] In D4: the data after merging and format conversion is sent to the interface unit.

[0024] The main goal of high-performance CPU is to achieve 3GHz, so for the 4-stage pipeline (D1-D4), there are the following bottlenecks: in D1, there are more Tag Ram input signals, and the frequency increase is limited; in D2, the Tag Ram output load is large, and the frequency increase is limited; in D3, the Data Ram output load is large, and it needs to be merged and format converted (larger logical levels), and the frequency increase is limited; in D4, the data is finally output to the interface, the bit width is large, the layout and wiring distance is long, and the frequency increase is limited.

[0025] Therefore, when the execution pipeline is limited, the CPU cache capacity is restricted by frequency increase and power consumption optimization, becoming one of the main bottlenecks restricting CPU performance improvement.

[0026] In view of the above problems, the embodiments of the present disclosure improve the cache structure and pipeline to expand the cache capacity. The cache includes at least two layer structures, each layer structure includes a first cache unit and a second cache unit, and first cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure.

[0027] Figure 2 The structure of the cache in the embodiment of the present disclosure is shown as follows Figure 1 ,like Figure 2 As shown, the number of layer structures is n, n ≥ 2, that is, the cache may include a first layer structure R1, a second layer structure R2, ..., and an nth layer structure Rn, and the elements and structures contained in each layer structure are the same. The first layer structure R1 includes a first cache unit M11 and a second cache unit M12, the second layer structure R2 includes a first cache unit M21 and a second cache unit M22, and the nth layer structure Rn includes a first cache unit Mn1 and a second cache unit Mn2.

[0028] If the cache capacity is simply expanded, the timing cannot converge (frequency decreases), and the processing performance cannot be improved. The cache access instructions of each layer structure independently access their own cache units, which can effectively solve the frequency drop problem caused by independent cache expansion. At the same time, the number of expanded layer structures (n-1) can be customized according to actual application needs. The total cache capacity after expansion is n times the cache capacity of the single-layer structure, which improves the product application flexibility and universality of this structure.

[0029] The cache provided by the embodiment of the present disclosure includes at least two layer structures, each layer structure includes a first cache unit and a second cache unit, first cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure; the cache of the embodiment of the present disclosure includes multiple layer structures, which can expand the cache capacity, improve the data hit rate, and enhance the overall CPU performance; moreover, the cache access instructions of each layer structure independently access their respective cache units, which can ensure the processing efficiency of the cache and optimize power consumption.

[0030] In some embodiments, the pipelines entered by the first cache access instructions of each layer of structure are incremented in sequence, and the pipelines entered by the second cache access instructions of each layer of structure are incremented in sequence. In other words, each layer of structure can be staggered in sequence according to the order of the pipelines.

[0031] In order to further increase the frequency, two adjacent layer structures can be staggered by one pipeline. That is, the pipeline entered by the first cache access instruction C21 of the second layer structure is D2, and the pipeline entered by the first cache access instruction C11 of the first layer structure is D1, and D2 is the next pipeline of D1; the pipeline entered by the second cache access instruction C22 of the second layer structure is D3, and the pipeline entered by the second cache access instruction C12 of the first layer structure is D2, and D3 is the next pipeline of D2. Therefore, in some embodiments, the pipeline entered by the first cache access instruction of the nth layer structure is the next pipeline of the pipeline entered by the first cache access instruction of the (n-1)th layer structure; the pipeline entered by the second cache access instruction of the nth layer structure is the next pipeline of the pipeline entered by the second cache access instruction of the (n-1)th layer structure.

[0032] In some embodiments, each layer structure further includes a data selection unit, a data processing unit and an interface unit. The data selection unit of each layer structure is used to select each second data according to the first data to obtain the target data, wherein the first data is the data obtained by accessing the first cache unit in the corresponding layer structure according to the first cache access instruction of the corresponding layer structure, and each second data is the data obtained by accessing the second cache unit in the corresponding layer structure according to the second cache access instruction of the corresponding layer structure; that is, for each layer structure, the second cache unit in the layer structure simultaneously outputs the physical addresses of four channels, and compares with the physical address output by the first cache unit in the layer structure, and selects the second cache unit channel output data equal to the comparison result. In addition, the data output by the second cache unit is merged with the data from other sources, and then the storage format is converted to the register format, the big-endian conversion, the sign conversion and other operations are performed. The data processing unit is used to merge the target data with the third data, and convert the format of the merged data to obtain the fourth data, and send the fourth data to the interface unit.

[0033] like Figure 3 As shown, the first layer structure R1 may also include a data selection unit S1, a data processing unit P1 and an interface unit I1, the second layer structure R2 may also include a data selection unit S2, a data processing unit P2 and an interface unit I2, and the first layer structure Rn may also include a data selection unit Sn, a data processing unit Pn and an interface unit In. Therefore, the processing process of a cache access instruction of a layer structure occupies four pipelines, the processing process of a cache access instruction of two layers of structures occupies five pipelines, and the processing process of a cache access instruction of n layers of structures occupies (n+3) pipelines.

[0034] In some embodiments, the first cache unit is a physical address memory, and the second cache unit is a data memory.

[0035] To clearly illustrate the technical solution of the embodiment of the present disclosure, the following Figure 4 and Figure 5 , the caches with two layers and three layers are respectively used as examples for explanation.

[0036] Figure 4 A schematic diagram of a cache including two layers of structures provided for a specific example of the present disclosure, such as Figure 4 As shown, the cache includes two layer structures, namely the first layer structure R1 and the second layer structure R2. The first layer structure R1 includes an address memory 1 (i.e., a first cache unit M11), a data memory 1 (i.e., a second cache unit M12), a data selection unit 1, a data merging and format conversion unit 1, and an interface unit 1; the second layer structure R1 includes a data address memory 2 (i.e., a first cache unit M21), a data memory 2 (i.e., a second cache unit M22), a data selection unit 2, a data merging and format conversion unit 2, and an interface unit 2. The overall cache capacity is doubled.

[0037] The first layer structure R1 and the second layer structure R2 are accessed independently, that is, the physical address read instruction Tag Read 11 of the first layer structure accesses the physical address memory 1 at D1, and the data read instruction Data Read12 of the first layer structure accesses the data memory 1 at D2; the physical address read instruction Tag Read 21 of the second layer structure accesses the physical address memory 2 at D2, and the data read instruction Data Read 22 of the second layer structure accesses the data memory 2 at D3.

[0038] The first layer structure R1 and the second layer structure R2 are processed independently, that is, the data of the data storage device 1 is selected and output by the data selection unit 1 at D3, and the data merging and format conversion processing is performed by the data merging and format conversion unit 1, and the data after merging and format conversion processing is output through the interface unit 1 at D4; the data of the data storage device 2 is selected and output by the data selection unit 2 at D4, and the data merging and format conversion processing is performed by the data merging and format conversion unit 2, and the data after merging and format conversion processing is output through the interface unit 2 at D5.

[0039] After one tier of expansion, the cache capacity is twice the capacity of the existing cache.

[0040] Figure 5 A schematic diagram of a cache including a three-layer structure provided for a specific example of the present disclosure, such as Figure 5As shown, the cache includes three layer structures, namely the first layer structure R1, the second layer structure R2 and the third layer structure R3. The first layer structure R1 includes a physical address memory 1 (i.e., a first cache unit M11), a data memory 1 (i.e., a second cache unit M12), a data selection unit 1, a data merging and format conversion unit 1 and an interface unit 1; the second layer structure R1 includes a physical address memory 2 (i.e., a first cache unit M21), a data memory 2 (i.e., a second cache unit M22), a data selection unit 2, a data merging and format conversion unit 2 and an interface unit 2; the third layer structure R3 includes a physical address memory 3 (i.e., a first cache unit M31), a data memory 3 (i.e., a second cache unit M32), a data selection unit 3, a data merging and format conversion unit 3 and an interface unit 3, and the overall cache capacity is doubled.

[0041] The first layer structure R1, the second layer structure R2 and the third layer structure R3 are accessed independently, that is, the physical address read instruction Tag Read 11 of the first layer structure accesses the physical address memory 1 at D1, and the data read instruction Data Read 12 of the first layer structure accesses the data memory 1 at D2; the physical address read instruction Tag Read21 of the second layer structure accesses the physical address memory 2 at D2, and the data read instruction Data Read 22 of the second layer structure accesses the data memory 2 at D3; the physical address read instruction Tag Read 31 of the third layer structure accesses the physical address memory 3 at D3, and the data read instruction Data Read 32 of the third layer structure accesses the data memory 3 at D4.

[0042] The first layer structure R1, the second layer structure R2 and the third layer structure R3 are processed independently, that is, the data of the data storage device 1 is selected and output by the data selection unit 1 at D3, and the data merging and format conversion processing is performed by the data merging and format conversion unit 1, and the data after the merging and format conversion processing is output through the interface unit 1 at D4; the data of the data storage device 2 is selected and output by the data selection unit 2 at D4, and the data merging and format conversion processing is performed by the data merging and format conversion unit 2, and the data after the merging and format conversion processing is output through the interface unit 2 at D5; the data of the data storage device 3 is selected and output by the data selection unit 3 at D5, and the data merging and format conversion processing is performed by the data merging and format conversion unit 3, and the data after the merging and format conversion processing is output through the interface unit 3 at D6.

[0043] After two tiers of expansion, the cache capacity is three times the capacity of the existing cache.

[0044] The embodiment of the present disclosure further provides a data processing device, which includes the cache as described above.

[0045] In some embodiments, the data processing device includes but is not limited to: a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processor (NPU), and a data processor (DPU).

[0046] In some embodiments, the cache of the data processing device is a first-level cache, or the cache of the data processing device is a first-level cache and a second-level cache.

[0047] In the case where the data processing device is a CPU, Figure 6 As shown, the CPU may include a processing core Core, a first-level cache L1 and a second-level cache L2. L3 is a third-level cache, which is set in a Soc (System on Chip, system-level chip), and the first-level cache L1, the second-level cache L2 and the third-level cache L3 are interconnected level by level. It should be noted that the CPU may also include a processing core Core and a first-level cache L1, but not a second-level cache L2.

[0048] The relationship between the capacity of each level of cache and memory access latency is shown in Table 1:

[0049] Table 1

[0050] Cache Levels capacity Memory access latency L1 <=128KB ~1ns L2 <=4MB ~5ns L3 Serveral MB ~10ns

[0051] It can be seen from Table 1 that in SoC, the CPU's L1 cache has the shortest memory access latency and the best performance. Therefore, increasing the capacity of the CPU's L1 cache has a significant effect on improving SoC performance. When reading data, it is first searched from the L1 cache. If it is not found, it is then searched from the L2 cache. If it is still not found, it is then searched from the L3 cache or memory. Therefore, the L1 cache adopts the aforementioned extended layer structure cache, which can greatly improve the performance of the data processing device and chip.

[0052] An embodiment of the present disclosure further provides a chip, which includes the data processing device as described above.

[0053] In some embodiments, the chip may further include a third-level cache, and in the case of multiple data processing devices, each data processing device is connected to the third-level cache. It should be noted that the third-level cache may also be the cache described above.

[0054] like Figure 6As shown, the chip is a SoC, the data processing device is a CPU, each CPU includes a first-level cache L1 and a second-level cache L2, and the chip also includes a third-level cache L3. The first-level cache L1 of each CPU is cascaded with the second-level cache L2 of the same CPU, and the second-level cache L2 of each CPU is cascaded with the third-level cache L3. The SoC is connected to the DRAM (dynamic random access memory) outside the chip through the third-level cache L3.

[0055] The cache of the disclosed embodiment can be applied to high-performance CPUs in computing-intensive fields such as cloud computing, scientific computing, and AI, and the performance of the CPU, i.e., the chip, can be improved by increasing the cache capacity. It should be noted that the cache is not limited to high-performance CPU applications, and any CPU that uses the cache can be used. The multi-layer structure of the cache of the disclosed embodiment will not affect the high-frequency implementation of the CPU, and it has strong scalability and applicability. By increasing the cache capacity, the CPU performance can be significantly improved.

[0056] It will be appreciated by those skilled in the art that all or some of the steps in the method disclosed above, the functional modules / units in the device may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0057] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly stated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A cache, characterized in that: It comprises at least two layer structures, each of which comprises a first cache unit and a second cache unit, first cache access instructions of different layer structures enter different pipelines to access the first cache unit in the corresponding layer structure, and second cache access instructions of different layer structures enter different pipelines to access the second cache unit in the corresponding layer structure.

2. The cache as claimed in claim 1, characterized in that: The pipelines into which the first cache access instructions of each layer structure enter are increased in sequence, and the pipelines into which the second cache access instructions of each layer structure enter are increased in sequence.

3. The cache according to claim 1 or 2, characterized in that: The pipeline entered by the first cache access instruction of the n-th layer structure is the next pipeline of the pipeline entered by the first cache access instruction of the (n-1)-th layer structure; The pipeline entered by the second cache access instruction of the n-th layer structure is the pipeline next to the pipeline entered by the second cache access instruction of the (n-1)-th layer structure; Wherein, n is the number of the layer structures.

4. The cache according to claim 1, wherein: Each of the layer structures further includes a data selection unit, a data processing unit and an interface unit, and the data selection unit of each of the layer structures is used to select each second data according to the first data to obtain target data, wherein the first data is data obtained by accessing the first cache unit in the layer structure according to the first cache access instruction of the corresponding layer structure, and each of the second data is data obtained by accessing the second cache unit in the layer structure according to the second cache access instruction of the corresponding layer structure; The data processing unit is used to combine the target data with the third data, convert the format of the combined data to obtain fourth data, and send the fourth data to the interface unit.

5. The cache according to any one of claims 1 to 2 and 4, characterized in that: The first cache unit is a physical address memory, and the second cache unit is a data memory.

6. A data processing device, characterized in that: Comprising a cache as described in any one of claims 1-5.

7. The data processing device according to claim 6, characterized in that The cache is a first-level cache, or the cache is a first-level cache and a second-level cache.

8. The data processing device according to claim 6, characterized in that The data processing device is one of the following: a central processing unit, a graphics processing unit, an embedded neural network processor, and a data processing device.

9. A chip, characterized in that: The method comprises a data processing device as claimed in any one of claims 6 to 8.

10. The chip according to claim 9, characterized in that: It also includes a three-level cache, there are multiple data processing devices, and each of the data processing devices is connected to the three-level cache.

Images