Processor and cross-die access method for processor

Abstract

The application relates to a processor and a cross-die access method for the processor. According to the application, for the cross-die access of a target data cached in another die by any die of the processor for the running of a core, the copy of the target data from the another die to the die can be triggered by the miss of the hit detection of the target data in the die, and the subsequent cross-die access of the target data by the die can be converted into the in-die access of the target data copied to the local cache of the die, so that the actual occurrence times of the cross-die access can be reduced. Moreover, when the running of the core in any die ends, the target data copied to the cache of the die can be deleted, so that the target data can adapt to the differentiated requirements of different cores on data consistency. Further, the efficiency of the cross-die access in the processor with at least two dies can be improved without losing consistency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence chips, in particular to a processor and a cross-die access method for the processor. BACKGROUND

[0002] Chiplet technology can package at least two dies as a processor, for example, as a processor with a SoC (System-on-Chip). Among them, the processor packaged with at least two dies can be referred to as a multi-die processor, and compared with the processor packaged with only one die, the processor packaged with at least two dies has higher computing power and storage capacity.

[0003] Specifically, in the processor packaged with at least two dies, each die has a processing core and a memory. Among them, the processing core of each die can not only access the memory of the die, but also realize cross-die access to the memory of other dies, that is, the memory of each die can be accessed by the processing core of all dies. Thus, multi-die fusion is realized.

[0004] However, the bandwidth of the D2D (Die to Die, die-to-die interconnect) bus between different dies is limited, thereby resulting in low efficiency of cross-die access.

[0005] As can be seen from the above, how to improve the efficiency of cross-die access in the processor packaged with at least two dies has become a technical problem to be solved in the related art. SUMMARY

[0006] The present application provides a processor and a cross-die access method for the processor, which helps to improve the efficiency of cross-die access in the processor packaged with at least two dies.

[0007] In an embodiment of the present application, a processor is provided, comprising:

[0008] a first die having a plurality of first-die-side processing cores and a plurality of first-die-side cache units, and a plurality of first-die-side main memories are respectively and one-to-one mapped to the plurality of first-die-side cache units;

[0009] a second die having a plurality of second-die-side processing cores and a plurality of second-die-side cache units, and a plurality of second-die-side main memories are respectively and one-to-one mapped to the plurality of second-die-side cache units;

[0010] The plurality of first-die-side cache units and the plurality of second-die-side cache units are configured to be one-to-one corresponding through a preset corresponding relationship between the plurality of first-die-side main memories and the plurality of second-die-side main memories, and any one of the first-die-side cache units is configured to:

[0011] In response to a read request initiated by any one of the first-die-side processing cores on target data associated with a currently running kernel, performing hit detection on the target data in the address space of the corresponding first-die-side main memory;

[0012] If the hit detection fails, a failure response is generated, which is used to copy and move the target data to the first-die-side cache unit when the access address of the target data is located in the address space of the corresponding second-die-side main memory;

[0013] If the hit detection succeeds, a read response carrying the hit target data is generated, which is used to be transmitted to the first-die-side processing core initiating the read request;

[0014] In response to the end of the running of the kernel associated with the target data, the target data copied and moved to the first-die-side cache unit is invalidated.

[0015] In some examples, optionally, the first die further has a cross-routing switch, and the cross-routing switch is configured to: route and forward, according to the access address of the target data, a read request initiated by any one of the first-die-side processing cores to the corresponding first-die-side cache unit; if any one of the first-die-side cache units generates a failure response due to the hit detection failure on the target data and the access address of the target data is located in the address space of the corresponding second-die-side main memory, route and forward the read request initiated on the target data to the corresponding second-die-side cache unit, so that the target data is copied and moved from the corresponding second-die-side cache unit to the first-die-side cache unit; if any one of the first-die-side cache units generates a read response due to the hit detection success on the target data, transmit the read response to the first-die-side processing core initiating the request on the target data.

[0016] In some examples, optionally, any one of the first-die-side cache units is further configured to: in response to the completion of the copy and movement of the target data, set a copy identifier of the tag information corresponding to the target data maintained in the first-die-side cache unit to an effective state; and in response to the end of the running of the kernel associated with the target data, invalidate the target data copied and moved to the first-die-side cache unit according to the copy identifier in the effective state.

[0017] In some examples, optionally, the tag information corresponding to the target data respectively maintained in the first die-side cache unit further comprises a core identifier corresponding to the core associated with the target data, and the operation of invalidating the target data is further based on the core identifier.

[0018] In some examples, optionally, the target data is a plurality of target data, a plurality of tag information corresponding to the plurality of target data respectively maintained in any one of the first die-side cache units, and the operation of invalidating the target data based on the replication identifier of the effective state comprises: invalidating the plurality of target data batched and replicated to the first die-side cache unit.

[0019] In some examples, optionally, the first die further has a dispatcher, and the dispatcher is configured to: dispatch a task instruction based on the core running to the plurality of first die-side processing cores; and when receiving a task response generated by any one of the first die-side processing cores due to the running of the core associated with the target data being completed, generate an invalidation command to the first die-side cache unit where the target data is located, so that the first die-side cache unit invalidates the target data replicated to the first die-side cache unit based on the received invalidation command.

[0020] In some examples, optionally, any one of the first die-side cache units comprises a main memory mapping region and a cross-die mapping region, the main memory mapping region of the first die-side cache unit is used for address space mapping of the first die-side main memory, and the cross-die mapping region of the first die-side cache unit is independent of the main memory mapping region.

[0021] In some examples, optionally, the target data associated with any one of the cores is read-only data restricted to exclusive access by the core during the running of the core.

[0022] In another embodiment of the present application, a cross-die access method for a processor is provided, the processor comprising a first die and a second die, the first die having a plurality of first die-side processing cores and a plurality of first die-side cache units, a plurality of first die-side main memories being one-to-one mapped to the plurality of first die-side cache units respectively, the second die having a plurality of second die-side processing cores and a plurality of second die-side cache units, a plurality of second die-side main memories being one-to-one mapped to the plurality of second die-side cache units respectively, the plurality of first die-side cache units and the plurality of second die-side cache units being one-to-one corresponding through a preset correspondence between the plurality of first die-side main memories and the plurality of second die-side main memories, and the cross-die access method comprising the following steps performed by any one of the first die-side cache units:

[0023] in response to a read request initiated by any one of the first-die-side processing cores in association with target data of a currently running kernel, performing a hit detection on the target data in the address space of the corresponding first-die-side main memory;

[0024] if the hit detection fails, generating a failure response, which is used to cause the target data to be copied from the corresponding second-die-side main memory to the first-die-side cache unit when the access address of the target data is located in the address space of the corresponding second-die-side main memory;

[0025] if the hit detection succeeds, generating a read response carrying the hit target data, which is used to be transmitted to the first-die-side processing core that initiates the read request;

[0026] in response to the end of the running of the kernel associated with the target data, invalidating the target data copied to the first-die-side cache unit.

[0027] In some examples, optionally, the first die further has a crossbar switch, and the cross-die access method further comprises the following steps performed by the crossbar switch: routing and forwarding, according to the access address of the target data, the read request initiated by any one of the first-die-side processing cores to the corresponding first-die-side cache unit; if any one of the first-die-side cache units generates a failure response due to the hit detection failure on the target data and the access address of the target data is located in the address space of the corresponding second-die-side main memory, routing and forwarding the read request initiated on the target data to the corresponding second-die-side cache unit so that the target data is copied from the corresponding second-die-side cache unit to the first-die-side cache unit; if any one of the first-die-side cache units generates a read response due to the hit detection success on the target data, transmitting the read response to the first-die-side processing core that initiates the request on the target data.

[0028] In some examples, optionally, the cross-die access method further comprises the following steps performed by any one of the first-die-side cache units: in response to the completion of the copying of the target data, setting a copy identifier of the tag information corresponding to the target data maintained in the first-die-side cache unit to an effective state; and in response to the end of the running of the kernel associated with the target data, invalidating the target data copied to the first-die-side cache unit according to the copy identifier in the effective state.

[0029] In some examples, optionally, the tag information corresponding to the target data respectively maintained in the first-die-side cache units further comprises a kernel identifier corresponding to the kernel associated with the target data, and the operation of invalidating the target data is further based on the kernel identifier.

[0030] In some examples, optionally, the target data is multiple, and multiple tag information is maintained for the multiple target data in any one of the first die side cache units respectively, and the operation of invalidating the target data according to the replication identifier of the effective state comprises: invalidating the multiple target data replicated and carried to the first die side cache unit in batches.

[0031] In some examples, optionally, the first die further has a dispatcher, and the cross-die access method further comprises the following steps performed by the dispatcher: dispatching a task instruction based on the kernel running to the multiple first die side processing cores; when receiving a task response generated by any one of the first die side processing cores due to the end of the kernel running associated with the target data, generating an invalid command to the first die side cache unit where the target data is located, so that the first die side cache unit invalidates the target data replicated and carried to the first die side cache unit according to the received invalid command.

[0032] In some examples, optionally, any one of the first die side cache units comprises a main memory mapping area and a cross-die mapping area, the main memory mapping area of the first die side cache unit is used for address space mapping of the first die side main memory, and the cross-die mapping area of the first die side cache unit is independent of the main memory mapping area.

[0033] In some examples, optionally, the target data associated with any one of the kernels is read-only data restricted to be accessed exclusively by the kernel during the running of the kernel.

[0034] As can be seen from the above, based on the embodiments of the present application, the cross-die access of any one of the dies of the processor to the target data cached in another die for the running of the kernel can trigger the replication and carrying of the target data from the other die to the die through the miss detection of the target data in the die, and the subsequent cross-die access of the die to the target data can be converted into the in-die access to the target data replicated and carried to the local cache of the die, so that the actual occurrence number of the cross-die access restricted by the bandwidth of the D2D bus can be reduced. Moreover, when the running of any one of the kernels associated with the target data ends, the target data replicated and carried to the cache of the die due to the running of the kernel can be deleted, so that the target data can adapt to the differentiated needs of the data coherency of different kernels, thereby the coherency of the target data in different dies can be taken into account. Furthermore, the efficiency of the cross-die access in the processor with at least two dies can be improved without losing the coherency. BRIEF DESCRIPTION OF DRAWINGS

[0035] The following drawings merely illustrate the application and are not intended to limit the scope of the application.

[0036] Figure 1 An exemplary structure schematic diagram of an internal architecture of a processor in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of a cross-die access principle of a processor in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of a cross-die access instance of a processor in an embodiment of the present application;

[0039] Figure 4 An exemplary structure schematic diagram of a cache unit of each die of a processor in an embodiment of the present application;

[0040] Figure 5 A working instance schematic diagram of a cache unit of each die of a processor in an embodiment of the present application;

[0041] Figure 6 An exemplary flow schematic diagram of a cross-die access method for a processor in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments.

[0043] Exemplarily, in the embodiments of the present application, the processor can be any one of integrated circuit chips suitable for artificial intelligence, such as GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural network Processing Unit), DPU (Deep learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose computing on Graphics Processing Units).

[0044] Figure 1 An exemplary structure schematic diagram of an internal architecture of a processor in an embodiment of the present application. Please refer to Figure 1 In the embodiments of the present application, the processor can include a first die D_p and a second die D_q, and the first die D_p and the second die D_q can be interconnected through a D2D bus.

[0045] Exemplarily, in the embodiments of the present application, the first die D_p can have a plurality of first-die-side processing cores A_p_1~A_p_n, and the second die D_q can have a plurality of second-die-side processing cores A_q_1~A_q_n. Wherein, n represents the number of the first-die-side processing cores A_p_1~A_p_n and the number of the plurality of second-die-side processing cores A_q_1~A_q_n, and n can be a positive integer greater than 1. It can be understood that the embodiments of the present application use n to represent the same number of the first-die-side processing cores A_p_1~A_p_n and the plurality of second-die-side processing cores A_q_1~A_q_n for ease of expression, but the number of the first-die-side processing cores A_p_1~A_p_n and the plurality of second-die-side processing cores A_q_1~A_q_n can also be different.

[0046] Exemplarily, in the embodiments of the present application, each of the first die D_p and the second die D_q can include one or more streaming processor clusters (SPC) for kernel computing, and any one or each streaming processor cluster can include a plurality of processing cores. In this case, the plurality of first-die-side processing cores A_p_1~A_p_n can include processing cores in at least one (for example, one or all) streaming processor cluster of the first die D_p, and similarly, the plurality of second-die-side processing cores A_q_1~A_q_n can include processing cores in at least one (for example, one or all) streaming processor cluster of the second die D_q.

[0047] Exemplarily, in the embodiments of the present application, each processing core A_p_k or A_q_k in the plurality of first-die-side processing cores A_p_1~A_p_n and the plurality of second-die-side processing cores A_q_1~A_q_n can implement data processing by running a kernel, and the kernel run by any one processing core A_p_k or A_q_k is a software kernel run by an instruction.

[0048] Exemplarily, in the embodiments of the present application, the plurality of first-die-side processing cores A_p_1~A_p_n and the plurality of second-die-side processing cores A_q_1~A_q_n can implement data processing for artificial intelligence, in which case each processing core A_p_k or A_q_k in the plurality of first-die-side processing cores A_p_1~A_p_n and the plurality of second-die-side processing cores A_q_1~A_q_n can be referred to as an artificial intelligence core (AI core), k is a positive integer greater than or equal to 1 and less than or equal to n.

[0049] Exemplarily, in the embodiments of the present application, the data processing implemented by the plurality of first-die-side processing cores A_p_1~A_p_n and the plurality of second-die-side processing cores A_q_1~A_q_n can be controlled by a dispatcher through task dispatching. Accordingly, the first die D_p can further have a first-die-side dispatcher DP_p, which can be configured to dispatch task indications based on the kernel running to the plurality of first-die-side processing cores A_p_1~A_p_n, and receive task responses generated by the plurality of first-die-side processing cores A_p_1~A_p_n respectively at the end of the respective kernel running. Similarly to the first die D_p, the second die D_q can further have a second-die-side dispatcher DP_q, which can be configured to dispatch task indications based on the kernel running to the plurality of second-die-side processing cores A_q_1~A_q_n, and receive task responses generated by the plurality of second-die-side processing cores A_q_1~A_q_n respectively at the end of the respective kernel running.

[0050] Exemplarily, in the embodiments of the present application, the first die D_p can further have a plurality of first-die-side cache units C_p_1~C_p_m, and the address spaces of the plurality of first-die-side main memories M_p_1~M_p_m can be respectively and one-to-one mapped to the plurality of first-die-side cache units C_p_1~C_p_m, i.e., the data in any one of the first-die-side main memories M_p_i can be dynamically cached in the corresponding first-die-side cache unit C_p_i, and the complete data consistency between any one of the first-die-side main memories M_p_i and the corresponding first-die-side cache unit C_p_i can not be required to be maintained. Wherein, m represents the number of the plurality of first-die-side cache units C_p_1~C_p_m and the number of the plurality of first-die-side main memories M_p_1~M_p_m whose address spaces are respectively mapped to the plurality of first-die-side cache units C_p_1~C_p_m, m can be a positive integer greater than 1, m can be the same as or different from n, and i is a positive integer greater than or equal to 1 and less than or equal to m.

[0051] Exemplarily, in the embodiments of the present application, the plurality of first-die-side processing cores A_p_1~A_p_n and the plurality of first-die-side cache units C_p_1~C_p_m can all be located in the first die D_p, and the plurality of first-die-side main memories M_p_1~M_p_m as the main memory can be located in the first die D_p or can also be independent of the first die D_p. But no matter what physical form the plurality of first-die-side main memories M_p_1~M_p_m has, the interconnection between the plurality of first-die-side main memories M_p_1~M_p_m and the plurality of first-die-side cache units C_p_1~C_p_m does not depend on the D2D bus (for example, all bypass the D2D bus).

[0052] Exemplarily, in the embodiments of the present application, the plurality of first-die-side processing cores A_p_1~A_p_n and the plurality of first-die-side cache units C_p_1~C_p_m can be interconnected through a Routing Crossbar-based Network on Chip (NOC). In this case, the first die D_p can also have at least one first-die-side Routing Crossbar RCB_p, which can be interconnected with the plurality of first-die-side cache units C_p_1~C_p_m through a first-die-side Network on Chip NOC_p in the first die D_p, and the interconnection between the first-die-side Routing Crossbar RCB_p and the plurality of first-die-side processing cores A_p_1~A_p_n can not depend on the first-die-side Network on Chip NOC_p.

[0053] Exemplarily, in the embodiment of the present application, the second die D_q can also have a plurality of second-die-side cache units C_q_1~C_q_m, and the address spaces of the plurality of second-die-side main memories M_q_1~M_q_m can be respectively and one-to-one mapped to the plurality of second-die-side cache units C_q_1~C_q_m, that is, the data in any one of the second-die-side main memories M_q_i can be dynamically cached in the corresponding second-die-side cache unit C_q_i, and the complete data consistency between any one of the second-die-side main memories M_q_i and the corresponding second-die-side cache unit C_q_i can not be required to be maintained. It can be understood that, in the embodiment of the present application, although the number of the plurality of second-die-side cache units C_q_1~C_q_m and the number of the plurality of second-die-side main memories M_q_1~M_q_m whose address spaces are respectively mapped to the plurality of second-die-side cache units C_q_1~C_q_m are both denoted by m, the embodiment of the present application does not exclude the case that the number of the plurality of first-die-side cache units C_p_1~C_p_m is different from the number of the plurality of second-die-side cache units C_q_1~C_q_m, and the number of the plurality of first-die-side main memories M_p_1~M_p_m is different from the number of the plurality of second-die-side main memories M_q_1~M_q_m.

[0054] Exemplarily, in the embodiment of the present application, similar to the first die D_p, the plurality of second-die-side processing cores A_q_1~A_q_n and the plurality of second-die-side cache units C_q_1~C_q_m of the second die D_q can all be located in the second die D_q, and the plurality of second-die-side main memories M_q_1~M_q_m can be located in the second die D_q or can also be independent of the second die D_q. Regardless of the physical form of the plurality of second-die-side main memories M_q_1~M_q_m, the interconnection between the plurality of second-die-side main memories M_q_1~M_q_m and the plurality of second-die-side cache units C_q_1~C_q_m does not depend on the D2D bus (for example, bypasses the D2D bus).

[0055] Exemplarily, in the embodiment of the present application, similar to the first die D_p, the second die D_q can also have at least one second-die-side crossbar switch RCB_q, the second-die-side crossbar switch RCB_q can be interconnected with the plurality of second-die-side cache units C_q_1~C_q_m through the second-die-side on-die network NOC_q in the second die D_q, and the interconnection between the second-die-side crossbar switch RCB_q and the plurality of second-die-side processing cores A_q_1~A_q_n can not depend on the second-die-side on-die network NOC_q.

[0056] Exemplarily, in the embodiments of the present application, the processor can realize the multi-die fusion by packaging the first die D_p and the second die D_q, and the multi-die fusion can include the processing core fusion of the plurality of first die side processing cores A_p_1~A_p_n and the plurality of second die side processing cores A_q_1~A_q_n, and the storage capacity fusion of the plurality of first die side main memories M_p_1~M_p_m and the plurality of second die side main memories M_q_1~M_q_m.

[0057] Exemplarily, in the embodiments of the present application, there can be a preset correspondence between the storage capacity fusion of the plurality of first die side main memories M_p_1~M_p_m and the plurality of second die side main memories M_q_1~M_q_m, for example, the storage capacity fusion of the plurality of first die side main memories M_p_1~M_p_m and the plurality of second die side main memories M_q_1~M_q_m can be configured in pairs in one-to-one correspondence. In this case, the address spaces of the main memories with the correspondence can be uniformly addressed, for example, the address spaces of any pair of corresponding first die side main memory M_p_i and second die side main memory M_q_i can be uniformly addressed, so that there is no address overlap in the address spaces of the corresponding first die side main memory M_p_i and second die side main memory M_q_i.

[0058] Exemplarily, in the embodiments of the present application, the plurality of first die side cache units C_p_1~C_p_m and the plurality of second die side cache units C_q_1~C_q_m can be configured in one-to-one correspondence through the preset correspondence between the plurality of first die side main memories M_p_1~M_p_m and the plurality of second die side main memories M_q_1~M_q_m. For example, if the plurality of first die side main memories M_p_1~M_p_m and the plurality of second die side main memories M_q_1~M_q_m are configured in pairs in one-to-one correspondence, then the plurality of first die side cache units C_p_1~C_p_m and the plurality of second die side cache units C_q_1~C_q_m can also be configured in pairs in one-to-one correspondence.

[0059] Exemplarily, in the embodiments of the present application, any one first die side cache unit C_p_i can be configured to:

[0060] In response to a read request initiated by any one first die side processing core A_p_k for target data associated with a currently running kernel, a hit-miss detection is performed on the address space of the corresponding first die side main memory M_p_i for the target data, that is, a hit-miss detection is performed on the target data in the first die side cache unit C_p_i and the corresponding first die side main memory M_p_i;

[0061] If the hit detection of the target data in the first die-side cache unit C_p_i and the corresponding first die-side main memory M_p_i fails, a failure response is generated, which can be used to cause the target data to be copied (e.g., from the corresponding second die-side cache unit C_q_i) to the first die-side cache unit C_p_i when the access address of the target data is located in the address space of the corresponding second die-side main memory M_q_i (i.e., the target data is a cross-die access data currently located in the corresponding second die-side main memory M_q_i or currently located in both the corresponding second die-side cache unit C_q_i and the corresponding second die-side main memory M_q_i);

[0062] If the hit detection of the target data in the first die-side cache unit C_p_i and the corresponding first die-side main memory M_p_i succeeds, e.g., the access address of the target data is located in the address space of the corresponding first die-side main memory M_p_i (i.e., the target data is an in-die access data currently located in the corresponding first die-side main memory M_p_i or currently located in both the first die-side cache unit C_p_i and the corresponding first die-side main memory M_p_i), or the target data is a cross-die access data copied from the second die D_q to the first die-side cache unit C_p_i, a read response carrying the hit target data (e.g., in-die access data or data copied through cross-die access) is generated, which is used to be transmitted to the first die-side processing core A_p_k that initiates the read request;

[0063] In response to the end of the running of the core associated with the target data, the target data copied to the first die-side cache unit C_p_i is invalidated so as to enable the target data to be deleted (e.g., invalidated any data can be deleted from the first die-side cache unit C_p_i at a time when a preset condition is met) from the first die-side cache unit C_p_i.

[0064] As can be seen from the above, based on the embodiments of the present application, for the cross-die access of a processor to target data cached in another die (e.g., the second die D_q) by any die (e.g., the first die D_p) for the running of a core, the target data can be copied from the other die (e.g., the second die D_q) to the die (e.g., the first die D_p) due to the miss of the hit detection of the target data in the die (e.g., the first die D_p), and the subsequent cross-die access of the die (e.g., the first die D_p) to the target data can be converted into the in-die access to the target data copied to the local cache of the die (e.g., the first die D_p), so that the actual number of cross-die accesses subject to the delay of the bandwidth of the D2D bus can be reduced. Moreover, when the running of any core associated with the target data in any die (e.g., the first die D_p) ends, the target data copied to the cache of the die (e.g., the first die D_p) due to the running of the core can be deleted, so that the target data can adapt to the differentiated requirements of different cores for data consistency, and thus the consistency of the target data in different dies can be taken into account. Furthermore, the efficiency of the cross-die access in the processor with at least two dies can be improved without loss of consistency.

[0065] Exemplarily, in the embodiments of the present application, the target data associated with any core can be read-only data restricted to be exclusively accessed by the core during the running of the core.

[0066] Figure 2 A schematic diagram of the principle of the cross-die access of the processor in the embodiments of the present application. Please refer to Figure 2 In the embodiments of the present application, any core running in the first die side processing core A_p_k is allocated with a cache space in the address space of the second die side main memory M_q_i, so that the first die side processing core A_p_k can write the target data associated with the core into the address space of the second die side main memory M_q_i (e.g., ① in FIG. 1) in the initial stage of running the core, and the first die side processing core A_p_k needs to read the target data as a parameter for operation multiple times in the subsequent stage of running the core. In this case, based on the embodiments of the present application, only the first reading of the target data by the first die side processing core A_p_k can be cross-die access, and the subsequent readings can be converted into in-die access due to the copy of the target data. Figure 2

[0067] Exemplarily, still referring to Figure 2 In the embodiments of the present application, the first die side cross-routing switch RCB_p can be configured to:

[0068] ​Based on the access address of the target data, any read request initiated by the first die-side processing core A_p_k (such as...) Figure 2 In step ②), the route is forwarded to the corresponding first bare-chip-side buffer unit C_p_i (e.g., Figure 2 (③ in the middle)

[0069] If any of the first die-side cache units C_p_i generates a failure response due to a failure to detect a hit with the target data (e.g.) Figure 2 ③) and the access address of the target data is located in the address space of the corresponding second-die-side main memory M_q_i (i.e., the target data is currently located in the corresponding second-die-side main memory M_q_i, or is currently located in both the corresponding second-die-side cache unit C_q_i and the corresponding second-die-side main memory M_q_i), then the read request initiated for the target data will be routed and forwarded to the corresponding second-die-side cache unit C_q_i (e.g., ...). Figure 2 (④ in the text) so that the target data is copied and moved from the corresponding second die-side cache unit C_q_i to the first die-side cache unit C_p_i (e.g., ... Figure 2 (⑤ in the middle)

[0070] If any first die-side cache unit C_p_i generates a read response due to a successful hit detection of the target data—for example, because the target data was copied and moved from the second die D_p to the first die-side cache unit C_p_i and a read response was successfully generated—then the read response is transmitted to the first die-side processing core A_p_k that initiated the request for the target data (e.g., ...). Figure 2 (⑥ in the text)

[0071] For example, in the embodiments of this application, any first die-side processing core A_p_k can repeatedly read target data by initiating read requests and successfully detecting a hit in the corresponding first die-side cache unit C_p_i (i.e., repeatedly and alternately executing). Figure 2 (as in ② and ⑥), until the kernel associated with the target data finishes running. At this time, the first raw die-side cache unit C_p_i can, in response to the end of the kernel associated with the target data, invalidate the target data copied and moved to the first raw die-side cache unit C_p_i (e.g., ...). Figure 2 (⑦ in the text)

[0072] For example, in an embodiment of this application, the first die-side dispatcher DP_p can be further configured to: when receiving a task response generated by any first die-side processing core A_p_k due to the end of kernel operation associated with the target data, generate an invalid command Inv_cmd_p to the first die-side cache unit C_p_i where the target data is located, so that the first die-side cache unit C_p_i responds to the received invalid command Inv_cmd_p by invalidating the target data copied and moved to the first die-side cache unit C_p_i.

[0073] For example, in an embodiment of this application, after the target data copied and moved to the first die-side cache unit C_p_i is invalidated due to the end of kernel execution, the first die-side processing core A_p_k that previously ran the kernel can start the execution of the next kernel. The execution of the next kernel can also be associated with the target data (e.g., using the target data as read-only data), but the assignment of the target data can be different. Furthermore, the first die-side processing core A_p_k can write the target data back into the address space of the second die-side main memory M_q_i during the initial stage of running the next kernel (e.g., repeating the process). Figure 2 ① and start over Figure 3 (Processes ① to ⑦ in the text) thus allow the same target data associated with multiple kernels to have configurable values ​​adapted to different kernels. Furthermore, the target data copied and moved to any of the first bare-chip cache units C_p_i is invalidated due to the termination of the associated kernel's operation, enabling the target data to adapt to the different data consistency requirements of different kernels.

[0074] Figure 3 This is a schematic diagram illustrating an example of cross-die access by the processor in an embodiment of this application. Please refer to [link / reference]. Figure 3 ( Figure 2 The numbers ① to ⑦ shown in the figure correspond to respectively Figure 3As shown in ①~⑦), in the embodiments of this application, in any first die-side cache unit C_p_i and the corresponding second die-side cache unit C_q_i, tag information can be maintained for any data. The tag information can be used for hit detection of the corresponding data. Furthermore, the tag information corresponding to any (e.g., target data) can include: a tag for characterizing the access address of the data (e.g., target data), a validity flag for characterizing whether the data (e.g., target data) is valid or invalid, a dirty flag for characterizing whether the data (e.g., target data) needs to be synchronized to the corresponding first die-side main memory M_p_i or second die-side main memory M_q_i, and a kernel identifier (kernel id) for characterizing the kernel that caused the data (e.g., target data) to be cached in the first die-side cache unit C_p_i or second die-side cache unit C_q_i. The operation of invalidating any data (e.g., target data) can be implemented by setting the validity flag to a digital level indicating invalidity.

[0075] For example, such as Figure 3 As shown in the embodiments of this application, the tag information maintained by any first die-side cache unit C_p_i for any data (e.g., target data) may further include a copy flag, and the copy flag is used to indicate whether the data is copied from other dies (e.g., second die D_q) to the first die-side cache unit C_p_i (e.g., copied as target data). In this case, any first die-side cache unit C_p_i can be further configured as follows:

[0076] In response to the copying and transfer of target data (e.g.) Figure 3 (⑤) Upon completion, the copy flag of the tag information corresponding to the target data maintained in the first die-side cache unit C_p_i is set to the active state, indicating that the target data was copied and moved from the corresponding second die-side cache unit C_q_i to the first die-side cache unit C_p_i; and,

[0077] In response to the termination of the kernel associated with the target data, such as in response to the task response generated by the first die-side dispatcher DP_p due to the termination of the kernel associated with the target data (e.g. Figure 4 (⑦) Based on the copy identifier of the effective state, the target data that has been copied and moved to the first bare die side cache unit C_p_i is invalidated, so that the target data that has been copied and moved to the first bare die side cache unit C_p_i is deleted from the first bare die side cache unit C_p_i due to the end of the associated kernel's operation.

[0078] Exemplarily, in the embodiment of the present application, the operation of invalidating the target data can further be based on the core identification, i.e., in response to the end of the running of the core associated with the target data, any first die-side cache unit C_p_i can invalidate the target data copied and carried to the first die-side cache unit C_p_i based on the core identification corresponding to the running-end core and the replication identification of the effective state, so that the target data copied and carried to the first die-side cache unit C_p_i is deleted from the first die-side cache unit C_p_i due to the end of the running of the associated core.

[0079] Exemplarily, in the embodiment of the present application, the target data associated with any core can be multiple, in which case the first die-side cache unit C_p_i caching multiple target data through copy and carry can correspondingly maintain multiple tag information for the multiple target data, and the operation of invalidating the target data by the first die-side cache unit C_p_i based on the core identification corresponding to the running-end core and the replication identification of the effective state can include invalidating the multiple target data copied and carried to the first die-side cache unit C_p_i in batches.

[0080] Figure 5 An exemplary structure diagram of the cache unit of each die of the processor in the embodiment of the present application. Figure 4 An exemplary working diagram of the cache unit of each die of the processor in the embodiment of the present application. Please refer to Figure 5 and Figure 5 In the embodiment of the present application, any first die-side cache unit C_p_i or second die-side cache unit C_q_i can include a multiplexer, a hit detection channel, a tag cache medium, a cache queue, and a data cache medium.

[0081] Exemplarily, in the embodiment of the present application, the multiplexer of any first die-side cache unit C_p_i or second die-side cache unit C_q_i can include two input ends and one output end, one of which is used to receive the read-write request of any first die-side cache unit C_p_i through the first die-side on-chip network NOC_p or the read-write request of any first die-side cache unit C_p_i through the second die-side on-chip network NOC_q (the connection structure related to this input end is omitted in the figure) Figure 4 The other input end is used to receive the invalidation command Inv_cmd_p or Inv_cmd_p generated by the first die-side dispatcher DP_p or the second die-side dispatcher DP_q.

[0082] Exemplarily, in the embodiments of the present application, the data cache medium of any one of the first-die-side cache units C_p_i or the second-die-side cache units C_q_i is used to store the data (including the target data) cached in the first-die-side cache unit C_p_i or the second-die-side cache unit C_q_i.

[0083] Exemplarily, in the embodiments of the present application, the cache queue of any one of the first-die-side cache units C_p_i or the second-die-side cache units C_q_i is used for data synchronization between the data cache medium and the corresponding first-die-side main memory M_p_i or the second-die-side main memory M_q_i.

[0084] Exemplarily, in the embodiments of the present application, the tag cache medium of any one of the first-die-side cache units C_p_i or the second-die-side cache units C_q_i is used to store the tag information maintained correspondingly for the data (including the target data) in the data cache medium.

[0085] Exemplarily, in the embodiments of the present application, the hit detection channel of any one of the first-die-side cache units C_p_i or the second-die-side cache units C_q_i is used to perform hit detection by using the received read-write request, so as to generate a read-write response to the splitter when the hit detection succeeds, or generate a failure response to the splitter when the hit detection fails, so that the read-write response or the failure response enters the cache queue through the splitter, thereby, the read-write response entering the cache queue can allow the hit data in the data cache medium to be read-write accessed by any one of the first-die-side cache units C_p_i through the first-die-side on-die network NOC_p, or by any one of the second-die-side cache units C_q_i through the second-die-side on-die network NOC_q. Figure 5 and Figure 4 the specific connection structure related to the read-write access is omitted), and the failure response entering the cache queue can be fed back to the first-die-side routing crossbar RCB_p or the second-die-side routing crossbar RCB_q through the first-die-side on-die network NOC_p or the second-die-side on-die network NOC_q.

[0086] Exemplarily, in the embodiments of the present application, the above description is exemplarily described only by taking the cross-die access of the first die D_p to the second die D_q as an example, but it can be understood that the cross-die access of the second die D_q to the first die D_p can be performed in the same way, i.e., the execution subject related to the first die D_p side and the execution subject related to the second die D_q side in the above description are transposed, and the relevant description of the cross-die access of the second die D_q to the first die D_p can be obtained. Therefore, Figure 4 the reference signs in the above description are represented by “ / ” in the meaning of “or”, thereby representing Figure 6The diagram in FIG. 1 is to express that the first die D_p and the second die D_q can be common.

[0087] Exemplarily, in the embodiments of the present application, any one first-die-side cache unit C_p_i can include a main memory mapping region and a cross-die mapping region, the main memory mapping region of the first-die-side cache unit C_p_i can be used for address space mapping of the first-die-side main memory M_p_i, and the cross-die mapping region of the first-die-side cache unit C_p_i is used to store target data copied and carried from the corresponding second-die-side cache unit C_q_i, and the cross-die mapping region of the first-die-side cache unit C_p_i is independent of the main memory mapping region of the first-die-side cache unit C_p_i. Similarly, if the second die D_q adopts the same cross-die access mode to the first die D_p, any one second-die-side cache unit C_q_i can include a main memory mapping region and a cross-die mapping region, the main memory mapping region of the second-die-side cache unit C_q_i can be used for address space mapping of the second-die-side main memory M_q_i, and the cross-die mapping region of the second-die-side cache unit C_q_i is used to store target data copied and carried from the corresponding first-die-side cache unit C_p_i, and the cross-die mapping region of the second-die-side cache unit C_q_i is independent of the main memory mapping region of the second-die-side cache unit C_q_i.

[0088] Exemplarily, in the embodiments of the present application, the dies included by the processor can not be limited to the first die D_p and the second die D_q shown in the diagram, i.e., the processor can also include other dies in addition to the first die D_p and the second die D_q, in which case, cross-die access can be implemented between each two dies of the at least two dies included by the processor in the manner described above for the first die D_p and the second die D_q.

[0089] In another embodiment of the present application, a cross-die access method for a processor is also provided. The processor to which the cross-die access method is applicable can have the same or similar structure as the processor described above which is packaged with a first die and a second die. For example, the processor to which the cross-die access method is applicable includes a first die and a second die. The first die has a plurality of first-die-side processing cores and a plurality of first-die-side cache units. The address spaces of a plurality of first-die-side main memories are respectively and one-to-one mapped to the plurality of first-die-side cache units. The second die has a plurality of second-die-side processing cores and a plurality of second-die-side cache units. A plurality of second-die-side main memories are respectively and one-to-one mapped to the plurality of second-die-side cache units. The plurality of first-die-side cache units and the plurality of second-die-side cache units are configured to be one-to-one corresponding through a preset corresponding relationship between the plurality of first-die-side main memories and the plurality of second-die-side main memories.

[0090] Figure 6 An exemplary flowchart of the cross-die access method for a processor in an embodiment of the present application is shown in FIG. 6. Referring to FIG. 6, the cross-die access method can include the following steps performed by any one of the first-die-side cache units: ​

[0091] S610: In response to a read request initiated by any one of the first-die-side processing cores for target data associated with a currently running kernel, performing a hit detection for the target data in the address space of the corresponding first-die-side main memory. For example, S610 can perform the hit detection for the target data in the first-die-side cache unit and the corresponding first-die-side main memory.

[0092] Exemplarily, in an embodiment of the present application, the target data associated with any kernel can be read-only data which is restricted to be accessed exclusively by the kernel during the running of the kernel.

[0093] If the hit detection for the target data in the first-die-side cache unit and the corresponding first-die-side main memory fails, a failure response can be generated by performing S630. The failure response can be used to copy and move (for example, copy and move from the corresponding second-die-side cache unit) the target data to the first-die-side cache unit when the access address of the target data is located in the address space of the corresponding second-die-side main memory (i.e., the target data is currently located in the corresponding second-die-side main memory or is currently located in the corresponding second-die-side cache unit and the corresponding second-die-side main memory). For example, the copy and movement of the target data can be achieved through cross-die transmission based on the D2D bus.

[0094] ​If the hit detection of the target data in the first-die-side cache unit and the corresponding first-die-side main memory succeeds, for example, the access address of the target data is located in the address space of the corresponding first-die-side main memory (i.e., the target data is currently located in the corresponding first-die-side main memory, or is simultaneously located in the first-die-side cache unit and the corresponding first-die-side main memory, which is the in-die access data), or the target data is the cross-die access data copied and carried from the second die to the first-die-side cache unit, a read response carrying the hit target data (e.g., in-die access data or cross-die access data) can be generated by performing S650, and the read response is used for being transmitted to the first-die-side processing core initiating the read request.

[0095] S670: in response to the end of the running of the core associated with the target data, invalidating the target data copied and carried to the first-die-side cache unit, so that the target data can be deleted from the first-die-side cache unit (e.g., any data invalidated can be deleted from the first-die-side cache unit at a time when a preset condition is met).

[0096] As can be seen above, based on the embodiments of the present application, for the cross-die access of the target data cached in another die (e.g., the second die) by a die (e.g., the first die) of the processor for the running of the core, the cross-die access can be triggered by the hit detection failure of the target data in the die (e.g., the first die), and the subsequent cross-die access of the target data by the die (e.g., the first die) can be converted into the in-die access of the target data copied and carried to the local cache of the die (e.g., the first die), so that the actual occurrence times of the cross-die access generating delay due to the bandwidth limitation of the D2D bus can be reduced. Moreover, when the running of the core associated with the target data in any die (e.g., the first die) ends, the target data copied and carried to the cache of the die (e.g., the first die) due to the running of the core can be deleted, so that the target data can adapt to the differentiated requirements of different cores for data consistency, thereby the consistency of the target data in different dies can be taken into account. Further, the efficiency of the cross-die access in the processor packaged with at least two dies can be improved without loss of consistency.

[0097] Exemplarily, in the embodiments of the present application, the cross-die access method for the processor can further include the following steps performed by the first-die-side cross-routing switch:

[0098] Before S610, the read request initiated by any first-die-side processing core is routed and forwarded to the corresponding first-die-side cache unit according to the access address of the target data;

[0099] If any one of the first-die-side cache units generates a failure response due to a failure in hit detection of the target data, and the access address of the target data is located in the address space of the corresponding second-die-side main memory (i.e., the target data is currently located in the corresponding second-die-side main memory, or is currently located in both the corresponding second-die-side cache unit and the corresponding second-die-side main memory), after S650, the read request initiated for the target data is routed and forwarded to the corresponding second-die-side cache unit, so that the target data is copied and transferred from the corresponding second-die-side cache unit to the first-die-side cache unit;

[0100] If any one of the first-die-side cache units generates a read response due to a success in hit detection of the target data, for example, generates a read response due to a success in hit of the target data copied and transferred from the second die to the first-die-side cache unit, after S630, the read response is transmitted to the first-die-side processing core that initiates the request for the target data.

[0101] Exemplarily, in the embodiments of the present application, the cross-die access method for the processor can further include the following steps performed by any one of the first-die-side cache units:

[0102] In response to completion of the copying and transferring of the target data, the copy identifier of the tag information corresponding to the target data and maintained in the first-die-side cache unit is set to an effective state; and,

[0103] In response to end of running of the core associated with the target data, the target data copied and transferred to the first-die-side cache unit is set to be invalid according to the copy identifier of the effective state, or the operation of setting the target data to be invalid is further performed according to the core identifier.

[0104] Exemplarily, in the embodiments of the present application, if the target data is multiple, and multiple tag information corresponding to the multiple target data is respectively maintained in any one of the first-die-side cache units, the operation of setting the target data to be invalid includes: setting the multiple target data copied and transferred to the first-die-side cache unit to be invalid in batches.

[0105] Exemplarily, in the embodiments of the present application, the cross-die access method for the processor can further include the following steps performed by the first-die-side dispatcher:

[0106] Dispatching, to the multiple first-die-side processing cores, a task indication based on running of the core;

[0107] When receiving the task response generated by any one of the first-die-side processing cores due to the end of the kernel running associated with the target data, an invalidation command is generated to the first-die-side cache unit where the target data is located, so that the target data copied and carried to the first-die-side cache unit is invalidated according to the received invalidation command.

[0108] It can be understood that in the embodiments of the present application, there can be an "and / or" relationship between parts of the exemplary description. In this article, the meaning of "and / or" is that there can be a "and" common definition relationship or also an "or" alternative definition relationship between the context connected by it. Therefore, the parts with "and / or" relationship can be understood as including different case combinations of "and / or" between each two parts, which represents "and" common definition relationship or "or" alternative definition relationship respectively, and such different case combinations can be considered as substantially equivalent to the definition range of "at least one of the parts".

[0109] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A processor, comprising: Comprising: a first die having a plurality of first-die-side processing cores and a plurality of first-die-side cache units, and a plurality of first-die-side main memories whose address spaces are respectively and one-to-one mapped to the plurality of first-die-side cache units; a second die having a plurality of second-die-side processing cores and a plurality of second-die-side cache units, and a plurality of second-die-side main memories respectively and one-to-one mapped to the plurality of second-die-side cache units; wherein the plurality of first-die-side cache units and the plurality of second-die-side cache units are configured to be one-to-one corresponding through a preset correspondence between the plurality of first-die-side main memories and the plurality of second-die-side main memories, and any one of the first-die-side cache units is configured to: in response to a read request initiated by any one of the first-die-side processing cores for target data associated with a currently running kernel, perform a hit detection for the target data in the address space of the corresponding first-die-side main memory; if the hit detection fails, generate a failure response, which is used to cause the target data to be copied and carried to the first-die-side cache unit when the access address of the target data is located in the address space of the corresponding second-die-side main memory; if the hit detection succeeds, generate a read response carrying the hit target data, which is used to be transmitted to the first-die-side processing core that initiates the read request; in response to the end of the running of the kernel associated with the target data, invalidate the target data copied and carried to the first-die-side cache unit.

2. The processor of claim 1, wherein the first die further has a crossbar switch, and the crossbar switch is configured to: route and forward the read request initiated by any one of the first-die-side processing cores to the corresponding first-die-side cache unit according to the access address of the target data; if any one of the first-die-side cache units generates a failure response due to the hit detection failure for the target data and the access address of the target data is located in the address space of the corresponding second-die-side main memory, route and forward the read request initiated for the target data to the corresponding second-die-side cache unit, so that the target data is copied and carried from the corresponding second-die-side cache unit to the first-die-side cache unit; if any one of the first-die-side cache units generates a read response due to the hit detection success for the target data, transmit the read response to the first-die-side processing core that initiates the request for the target data.

3. The processor of claim 1, wherein any one of the first-die-side cache units is further configured to: in response to the completion of the copying and carrying of the target data, set a copy identifier of tag information corresponding to the target data maintained in the first-die-side cache unit to an effective state; and in response to the end of the running of the kernel associated with the target data, invalidate the target data copied and carried to the first-die-side cache unit according to the copy identifier in the effective state.

4. The processor of claim 3, wherein ​ ​ ​ The tag information corresponding to the target data in the first die side cache unit further includes a core identifier corresponding to the core associated with the target data, and the operation of invalidating the target data is further based on the core identifier.

5. The processor of claim 3, wherein, The target data is a plurality of target data, and a plurality of tag information corresponding to the plurality of target data is maintained in any one of the first die side cache units, and the operation of invalidating the target data based on the replication identifier of the effective state comprises: invalidating a plurality of target data batched and replicated to the first die side cache unit.

6. The processor of any one of claims 1 to 5, wherein, The first die further has a dispatcher, and the dispatcher is configured to: dispatch a task instruction based on the core running to the plurality of first die side processing cores; when receiving a task response generated by any one of the first die side processing cores due to the end of the core associated with the target data running, generate an invalidation command to the first die side cache unit where the target data is located, so that the first die side cache unit invalidates the target data replicated to the first die side cache unit according to the received invalidation command.

7. The processor of any one of claims 1 to 5, wherein, Any one of the first die side cache units includes a main memory mapping region and a cross-die mapping region, the main memory mapping region of the first die side cache unit is used for address space mapping of the first die side main memory, and the cross-die mapping region of the first die side cache unit is independent of the main memory mapping region.

8. The processor of any one of claims 1 to 5, wherein, The target data associated with any core is read-only data restricted to exclusive access by the core during the running of the core.

9. A method for cross-die access for a processor, the method comprising: The processor includes a first die and a second die, the first die has a plurality of first die side processing cores and a plurality of first die side cache units, the address spaces of a plurality of first die side main memories are one-to-one mapped to the plurality of first die side cache units, the second die has a plurality of second die side processing cores and a plurality of second die side cache units, a plurality of second die side main memories are one-to-one mapped to the plurality of second die side cache units, the plurality of first die side cache units and the plurality of second die side cache units are configured to be one-to-one through a predetermined correspondence between the plurality of first die side main memories and the plurality of second die side main memories, and the cross-die access method includes the following steps performed by any one of the first die side cache units: in response to a read request initiated by any one of the first die side processing cores to target data associated with a currently running core, performing a hit detection on the target data in the address space of the corresponding first die side main memory; if the hit detection fails, a failure response is generated, which is used to replicate the target data to the first die side cache unit when the access address of the target data is located in the address space of the corresponding second die side main memory; If the hit detection succeeds, a read response carrying the hit target data is generated, and the read response is transmitted to the first die-side processing core which initiates the read request; In response to the end of the running of the core associated with the target data, the target data copied and carried to the first die-side cache unit is invalidated.

10. The cross-die access method of claim 9, wherein, The first die further has a crossbar switch, and the cross-die access method further comprises the following steps performed by the crossbar switch: According to the access address of the target data, the read request initiated by any one of the first die-side processing cores is routed and forwarded to the corresponding first die-side cache unit; If any one of the first die-side cache units generates a failure response due to a failure of the hit detection on the target data, and the access address of the target data is located in the address space of the corresponding second die-side main memory, the read request on the target data is routed and forwarded to the corresponding second die-side cache unit, so that the target data is copied and carried from the corresponding second die-side cache unit to the first die-side cache unit; If any one of the first die-side cache units generates a read response due to a success of the hit detection on the target data, the read response is transmitted to the first die-side processing core which initiates the request on the target data.

Images