Cache control method and artificial intelligence chip

Abstract

The application relates to the technical field of artificial intelligence, and provides a cache control method and an artificial intelligence chip, the method comprising the following steps: determining the domain identification of a kernel; executing the kernel; in the execution process of the kernel, accessing the cache based on a memory access request, and storing the domain identification into the tag of the corresponding cache line in the cache; after the execution of the kernel is completed, controlling the cache to reset the tag of the cache line containing the domain identification in the tag. The cache control method and the artificial intelligence chip provided by the application realize kernel-granularity data isolation and guarantee data security. The tag resetting is limited to the cache line belonging to the kernel whose execution is completed, unnecessary cache invalidation operations are maximally reduced, the influence of memory access on the kernel being executed is reduced, the utilization rate of the cache is effectively improved, and the parallel processing performance of the artificial intelligence chip is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a cache control method and an artificial intelligence chip. BACKGROUND

[0002] In order to improve the computing effect, the artificial intelligence chip allows multiple kernels to execute in parallel. The multiple kernels executing in parallel share cache space. When any one of the kernels completes execution, the artificial intelligence chip performs a global invalidation strategy to clean up the data in the cache, thereby avoiding the use of expired data in the cache by the subsequently executed kernels.

[0003] However, the global invalidation strategy affects other executing kernels. This indiscriminate cache management strategy causes the effective cache data of the executing kernels to be forcibly cleared, forcing the executing kernels to reload data from the main memory, significantly increasing memory access delay and reducing parallel processing performance and throughput. In addition, the mixed storage of cache data of different kernels and the lack of effective isolation mechanism also bring potential risks in data security. SUMMARY

[0004] The present application provides a cache control method and an artificial intelligence chip to solve the defects of low multi-kernel parallel processing performance and poor data security in the related art.

[0005] The present application provides a cache control method, comprising:

[0006] determining the domain identifier of the kernel;

[0007] executing the kernel, in the execution process of the kernel, accessing the cache based on a memory access request, and storing the domain identifier into the tag of the cache line corresponding to the memory access request in the cache;

[0008] after the execution of the kernel ends, controlling the cache to reset the tag of the cache line containing the domain identifier in the tag.

[0009] According to the cache control method provided by the present application, the cache is accessed based on a memory access request, and the domain identifier is stored into the tag of the cache line corresponding to the memory access request in the cache, which comprises:

[0010] generating a memory access request containing the domain identifier of the kernel, and searching for target data requested by the memory access request in the cache;

[0011] In case that the target data does not exist in the cache, the target data is loaded from a lower storage to the cache, and the domain identifier is stored in the tag of the cache line where the target data is located.

[0012] According to the cache control method provided by the application, the cache is accessed based on the memory access request, and the domain identifier is stored in the tag of the cache line corresponding to the memory access request in the cache, and the method further comprises:

[0013] In case that the target data exists in the cache, and the domain identifier in the tag of the cache line where the target data is located is different from the domain identifier in the memory access request, the domain identifier in the tag of the cache line where the target data is located is replaced by the domain identifier in the memory access request.

[0014] According to the cache control method provided by the application, the domain identifier of the core is determined, and the method comprises:

[0015] The occupation states of a plurality of candidate domain identifiers are polled, the domain identifier of the core is determined from the plurality of candidate domain identifiers based on the polling result, and the occupation state of the domain identifier of the core is updated.

[0016] According to the cache control method provided by the application, the cache is controlled to perform tag reset on the cache line whose tag contains the domain identifier, and the method comprises:

[0017] An execution end command of the core is generated, and the execution end command contains the domain identifier of the core;

[0018] Based on the execution end command, the cache is controlled to traverse the tags of all cache lines, and in case that the domain identifier in the tag is the same as the domain identifier in the execution end command, tag reset is performed.

[0019] According to the cache control method provided by the application, the method further comprises:

[0020] During the execution of the core, an execution mode contained in the memory access request is stored in the tag of the cache line corresponding to the memory access request in the cache;

[0021] After the execution of the core is completed, the cache is controlled to clear the cache line whose tag contains the domain identifier and whose execution mode is temporary data.

[0022] According to the cache control method provided by the application, one domain identifier corresponds to one or more cores.

[0023] The application further provides an artificial intelligence chip, comprising a command processor, a computing unit and a cache, wherein the command processor comprises a domain identity allocation unit.

[0024] The domain identity allocation unit is configured to determine a domain identity of the kernel.

[0025] The command processor is configured to allocate the kernel and the domain identity of the kernel to the computing unit, and trigger the computing unit to execute the kernel.

[0026] The computing unit is configured to, during execution of the kernel, access the cache based on a memory access request, and store the domain identity into a tag of a corresponding cache line of the memory access request in the cache.

[0027] The command processor is further configured to, after the kernel is executed, control the cache to reset the tag of the cache line containing the domain identity in the tag.

[0028] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cache control method according to any one of the above.

[0029] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the cache control method according to any one of the above.

[0030] The application further provides a computer program product, comprising a computer program, wherein the computer program is executable on a processor to implement the cache control method according to any one of the above.

[0031] The cache control method and the artificial intelligence chip provided by the application store the domain identity of the kernel into the tag of the corresponding cache line of the memory access request in the cache during execution of the kernel, thereby realizing the association between the kernel and the cache line, and enabling the cache to realize kernel-granularity data isolation based on the domain identity in the tag of the cache line, so as to ensure data security. After the kernel is executed, the cache is controlled to reset the tag of the cache line containing the domain identity in the tag, and the tag reset is limited to the cache line belonging to the executed kernel, thereby minimizing unnecessary cache invalidation operations, reducing the influence of memory access on the executing kernel, effectively improving the utilization rate of the cache, and further ensuring the parallel processing performance of the artificial intelligence chip. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to make the technical solutions in the present application or the related art clearer, the accompanying drawings needed in the embodiments or the related art description will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0033] Figure 1 is a flowchart of the cache control method provided by the present application.

[0034] Figure 2 is a structural diagram of the artificial intelligence chip provided by the present application.

[0035] Figure 3 is a structural diagram of the cache control device provided by the present application.

[0036] Figure 4 is a structural diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0037] In order to make the technical solutions in the present application or the related art clearer, the accompanying drawings needed in the embodiments or the related art description will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0038] In the field of high-performance computing, artificial intelligence chips are widely used due to their powerful parallel processing capabilities. The artificial intelligence chip here can be at least one of GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural network Processing Unit), DPU (Deep learning Processing Unit), and APU (Accelerated Processing Unit).

[0039] In order to fully utilize hardware resources and improve overall throughput, multiple kernels are usually executed in parallel on the artificial intelligence chip, and the multiple kernels executed in parallel share a cache space.

[0040] The procedure of executing multiple kernels in parallel can be represented as the following steps:

[0041] In the kernel dispatch stage, multiple kernels are allocated to corresponding computing units by a task scheduling algorithm; all kernels share the same cache, and the cache data is managed by physical address or virtual address.

[0042] In the kernel execution stage, when the kernel accesses the memory, the data will be loaded into the cache to speed up subsequent access. However, the cache line tag of the cache currently usually only contains address, valid bit, cache priority, data read-only, data consistency and the like, and does not contain identification information capable of distinguishing to which kernel the data belongs.

[0043] After any kernel execution is completed, a global invalidation strategy is executed to clean up the data in the cache. Here, the global invalidation strategy aims to reset the cache priority, data read-only, consistency and the like in the tag of the cache line in the entire cache space or most of the cache space, so as to realize the reset of the control strategy for the cache line and the release of the consistency conflict for the cache line.

[0044] However, this global invalidation strategy has problems in the actual application of multiple kernel parallel execution:

[0045] Firstly, it will cause significant performance degradation. It can be understood that the global invalidation operation is indiscriminate, and it cannot distinguish to which kernel the data in the cache belongs. When a kernel execution is completed and triggers the global invalidation strategy, the control strategy of the valid data of other kernels still being executed will also be reset, forcing the kernel still being executed to reload the data from the main memory when next accessed, which greatly increases the memory access delay, offsets the partial advantages brought by parallel execution, and finally causes the overall throughput to decrease.

[0046] Secondly, the above scheme introduces potential risks in data security. The data of all kernels is mixed in the shared cache, and it is difficult to realize fine-grained data management and protection, which constitutes a potential security risk in the application scenario with high security requirements.

[0047] In view of the above problems, an embodiment of the present application provides a cache control method. Figure 1 is a flowchart of the cache control method provided by the present application, as Figure 1 shown, the method can be applied to an artificial intelligence chip, and the method comprises the following steps:

[0048] Step 110, determining the domain identification of the kernel.

[0049] The kernel here refers to the kernel to be executed, which can be one or more. During the kernel scheduling phase, a domain identifier can be assigned to each kernel. That is, each kernel has a corresponding domain identifier.

[0050] A domain identifier can be a kernel identifier used during memory access. Different domain identifiers can distinguish the data of different kernels in the cache, thereby enabling the differentiation and management of cached data of each kernel in the shared cache space, and ultimately achieving kernel-level data isolation based on domain identifiers.

[0051] Step 120: Execute the kernel. During the execution of the kernel, access the cache based on the memory access request and store the domain identifier in the tag of the cache line corresponding to the memory access request in the cache.

[0052] Specifically, for each kernel, during kernel execution, memory access requests can be generated to access the cache. Unlike related technologies, when accessing the cache via memory access requests, the kernel's domain identifier can be stored in the cache line corresponding to the requested data—that is, the cache line in the cache corresponding to the memory access request. Specifically, this can be achieved by adding a kernel domain identifier field to the cache line's tag structure, thereby storing the kernel's domain identifier in the cache line's tag.

[0053] Therefore, in the cache, the label of a cache line no longer only contains information such as address, valid bit, cache priority, read-only data, and data consistency, but also includes a field identifier. It can be understood that storing the kernel's field identifier in the cache line label establishes a connection between the kernel and the cache line. A cache line whose label contains the field identifier can also be understood as belonging to the cache line of the kernel corresponding to that field identifier, and the data in that cache line is the data used by the kernel corresponding to that field identifier for execution.

[0054] In this way, each kernel can store its own domain identifier into the label of the cache line corresponding to the requested data through memory access during execution. Thus, in the cache, each kernel can divide its own cache line by adding the domain identifier to the label. Therefore, kernel-level data isolation is achieved in the cache by the domain identifier in the cache line label.

[0055] Step 130: After the kernel execution is completed, control the cache to reset the labels of the cache lines whose labels contain the domain identifier.

[0056] Specifically, for each core, after the end of the core execution, the cache line containing the domain identifier of the core in the end of execution in the tag can be found in the cache, and the cache line is tagged reset. Here, the cache line is tagged reset, including resetting the cache priority, data read-only, consistency and other information in the tag, deleting the domain identifier in the tag, and the like. Thus, the control policy reset and consistency conflict resolution for the cache line belonging to the core in the end of execution can be realized through the tag reset.

[0057] Thus, after the end of each core execution, the tag reset is only performed on the cache line belonging to the core in the cache, thereby replacing the global invalidation strategy in the related art with the tag reset only on the cache line belonging to the core in the end of execution to realize the local invalidation, and the data belonging to other cores in execution in the cache remains unchanged. This selective tag reset limits the tag reset within the scope of the cache line belonging to the core in the end of execution, maximally reduces unnecessary cache invalidation operations, reduces the impact on the memory access of the core in execution, and effectively improves the utilization of the cache.

[0058] In the method provided in the embodiment of the application, the domain identifier of the core is stored in the tag of the corresponding cache line of the memory access request in the cache during the core execution, thereby realizing the association between the core and the cache line, so that the cache realizes the core-granularity data isolation with the domain identifier in the tag of the cache line, and the data security is ensured. After the end of the core execution, the cache line containing the domain identifier in the tag is controlled to be tagged reset, and the tag reset is limited within the scope of the cache line belonging to the core in the end of execution, maximally reduces unnecessary cache invalidation operations, reduces the impact on the memory access of the core in execution, effectively improves the utilization of the cache, and further ensures the parallel processing performance of the artificial intelligence chip.

[0059] Moreover, the hardware overhead required by the method is extremely small. Assuming that four selectable domain identifiers are set, the domain identifier only needs to occupy 2-bit storage space, and the hardware modification cost is low. Compared with the complex cache management method in the related art, the cache control method provided in the embodiment of the application has higher cost performance and practicability.

[0060] Based on the above embodiment, in step 120, the cache is accessed based on the memory access request, and the domain identifier is stored in the tag of the corresponding cache line of the memory access request in the cache, including:

[0061] generating a memory access request containing the domain identifier of the core, and finding target data requested by the memory access request in the cache;

[0062] In case that the target data does not exist in the cache, the target data is loaded from a lower storage of the cache to the cache, and the domain identifier is stored in the tag of the cache line where the target data is located.

[0063] Specifically, for any one executing kernel, a memory access request can be generated during execution. Different from the memory access request in the related art, the memory access request in the embodiment of the present application contains the domain identifier of the kernel, i.e., the domain identifier of the kernel can be encapsulated in the memory access request.

[0064] Subsequently, the memory access request containing the domain identifier of the kernel can be sent to the cache. After receiving the memory access request, the cache can parse the domain identifier of the kernel contained in the memory access request. After receiving the memory access request, the cache can search whether the data requested by the memory access request exists in the cache. For the convenience of description, the data requested by the memory access request is recorded as the target data in the embodiment of the present application.

[0065] In case that the target data cannot be found in the cache, i.e., the target data does not exist in the cache, the target data can be searched from a lower storage of the cache, and the searched target data is loaded to the cache. The lower storage here refers to the lower storage of the cache, which can be the main memory for example. After the target data is loaded to the cache, the parsed domain identifier of the kernel is stored in the tag of the cache line where the target data is located, thereby associating the kernel initiating the memory access request with the requested target data through the domain identifier.

[0066] Based on any one of the above embodiments, in step 120, the cache is accessed based on the memory access request, and the domain identifier is stored in the tag of the cache line corresponding to the memory access request in the cache, further comprising:

[0067] In case that the target data exists in the cache, and the domain identifier in the tag of the cache line where the target data is located is different from the domain identifier in the memory access request, the domain identifier in the tag of the cache line where the target data is located is replaced by the domain identifier in the memory access request.

[0068] Specifically, after receiving the memory access request containing the domain identifier of the kernel, the cache can search whether the target data requested by the memory access request exists in the cache.

[0069] In the case that the target data is found in the cache, i.e. the case that the target data is in the cache, the domain identifier in the tag of the cache line where the target data is located is compared with the domain identifier of the core in the memory access request resolved by the cache, in the case that the domain identifier in the tag is the same as the domain identifier in the memory access request, the read operation or the write operation is directly performed on the target data, in the case that the domain identifier in the tag does not exist, the domain identifier in the memory access request is stored in the tag of the cache line where the target data is located, and the read operation or the write operation is performed on the target data, in the case that the domain identifier in the tag is different from the domain identifier in the memory access request, the domain identifier in the memory access request is used to replace the original domain identifier in the tag of the cache line where the target data is located, and the read operation or the write operation is performed on the target data.

[0070] Therefore, in the cache, the domain identifier in the tag of the cache line can be updated according to the domain identifier contained in the latest memory access request, so that the relationship between the cache line and the core in the cache can be updated in real time, thereby guaranteeing the reliability of the core-granularity data isolation in the cache.

[0071] Based on any of the above embodiments, in step 110, the domain identifier of the core is determined, including:

[0072] The occupation states of a plurality of candidate domain identifiers are polled, the domain identifier of the core is determined from the plurality of candidate domain identifiers based on the polling result, and the occupation state of the domain identifier of the core is updated.

[0073] Specifically, a plurality of candidate domain identifiers can be set in advance. Here, the candidate domain identifier is the domain identifier that can be assigned to the core. Moreover, the occupation state of each candidate domain identifier can be maintained in real time. For any one candidate domain identifier, the occupation state of the candidate domain identifier is used to reflect whether the candidate domain identifier is occupied by the core, or is used to reflect the number of cores occupying the candidate domain identifier.

[0074] By polling the occupation states of the plurality of candidate domain identifiers, the polling result can be obtained. The polling result can be a candidate domain identifier that is not occupied by the core from the plurality of candidate domain identifiers, or a candidate domain identifier with the smallest number of occupied cores from the plurality of candidate domain identifiers. After obtaining the polling result, the candidate domain identifier assigned to the core to be executed can be determined from the plurality of candidate domain identifiers based on the polling result.

[0075] For example, based on the polling result, a candidate domain identifier that is not occupied by the core can be selected from the plurality of candidate domain identifiers as the domain identifier of the core to be executed. For another example, based on the polling result, a candidate domain identifier with the smallest number of occupied cores can be selected from the plurality of candidate domain identifiers as the domain identifier of the core to be executed, which is not limited in the embodiments of the present application.

[0076] Further, after determining the domain identifier of the kernel, it is also needed to update the occupation state of the domain identifier of the kernel, for example, updating the occupation state of the domain identifier from not occupied by the kernel to occupied by the kernel, or for example, increasing the number of the kernels occupying the domain identifier by 1 in the occupation state of the domain identifier.

[0077] In the embodiments of the present application, the occupation states of the plurality of candidate domain identifiers are polled to allocate the domain identifier to the kernel to be executed, thereby ensuring the balance of the plurality of candidate domain identifiers when the domain identifier is allocated to the kernel, avoiding the situation that some domain is overloaded while other domains are idle, and realizing the balanced utilization of resources.

[0078] Based on any of the above embodiments, in step 130, the control of the cache to reset the tag of the cache line containing the domain identifier in the tag includes:

[0079] generating an execution end command of the kernel, the execution end command containing the domain identifier of the kernel;

[0080] based on the execution end command, controlling the cache to traverse the tags of all cache lines, and resetting the tag in the case that the domain identifier in the tag is the same as the domain identifier in the execution end command.

[0081] Specifically, for the kernel that ends execution, an execution end command of the kernel can be generated. The execution end command contains the domain identifier of the kernel to inform the cache that the kernel ends execution and needs to clean up resources for the kernel.

[0082] The execution end command of the kernel that ends execution can be sent to the cache. After receiving the execution end command, the cache can parse the domain identifier of the kernel contained in the execution end command. Further, after receiving the execution end command, the cache can traverse all cache lines. Specifically, the tags of all cache lines can be traversed, and the domain identifier contained in the tag of each cache line is compared with the domain identifier contained in the execution end command parsed, thereby obtaining the cache line whose domain identifier contained in the tag is the same as the domain identifier contained in the execution end command, that is, the cache line belonging to the kernel that ends execution. For the cache line belonging to the kernel that ends execution, the cache can reset the tag of such cache line, thereby limiting the tag reset to the cache line belonging to the kernel that ends execution, and minimizing unnecessary cache invalidation operations.

[0083] Based on any of the above embodiments, the method further includes:

[0084] During execution of the core, the execution mode contained in the memory access request is stored in the tag of the corresponding cache line of the memory access request in the cache;

[0085] After the execution of the core ends, the cache line whose tag contains the domain identifier and whose execution mode is temporary data is controlled to be cleared.

[0086] Specifically, for any one core, during execution of the core, for the case of initiating memory access, the execution mode of the requested target data can be added in the memory access request during generation of the memory access request.

[0087] Here, the execution mode is used to reflect the life cycle mode of the target data, for example, the execution mode can be temporary data, and the target data whose execution mode is temporary data is the data whose life cycle is bound to the core execution, and such target data is usually not applied after the core execution ends; the execution mode can also be non-temporary data, and the target data whose execution mode is non-temporary data is the data whose life cycle is decoupled from the core execution, and such target data can be applied to other core execution after the core execution ends.

[0088] Correspondingly, after receiving the memory access request, the cache not only stores the domain identifier of the core contained in the memory access request in the tag of the corresponding cache line, but also stores the execution mode contained in the memory access request in the tag of the corresponding cache line. Therefore, in the tag of the cache line where the target data is located, not only the domain identifier of the core to which the target data belongs exists, but also the execution mode of the target data itself exists.

[0089] After execution of any one core ends, the cache can be controlled to traverse all cache lines, so as to obtain the cache line whose tag contains the domain identifier consistent with the domain identifier of the core that ends execution and whose execution mode is temporary data. It can be understood that such cache line belongs to the core that ends execution, and the data cached in such cache line belongs to temporary data that is no longer applied after the core execution ends, and therefore, data cleaning can be performed on such cache line.

[0090] In the embodiment of the application, a differentiated cache management strategy is formulated according to the execution mode, fine cache management is achieved, and system performance is further improved.

[0091] Based on any one of the above embodiments, one domain identifier corresponds to one or more cores.

[0092] Specifically, in the case of multi-core parallel execution, the number of pre-set candidate domain identifiers can be less than the number of kernels to be parallelized. That is, there is a case where a unique domain identifier cannot be allocated to each kernel due to insufficient number of candidate domain identifiers. In this case, one domain identifier is allowed to correspond to one or more kernels. That is, the same domain identifier can be allocated to multiple kernels. For example, there are four domain identifiers A, B, C, and D, but there are five kernels to be parallelized, in which case two of the five kernels are allowed to share one domain identifier, and the remaining three kernels each correspond to one domain identifier. Thus, during the parallel execution of the five kernels, when one of the two kernels sharing the same domain identifier ends execution and the cache line tag is reset based on the shared domain identifier, the cache data of the other kernel will be affected, but the cache data of the remaining three kernels is still completely preserved. Compared with the related art in which a global invalidation strategy is performed when one kernel ends execution, resulting in all running kernels being disturbed, the unnecessary cache invalidation operation is reduced, the impact on memory access of the running kernel is reduced, the utilization of the cache is effectively improved, and the parallel processing performance of the artificial intelligence chip is guaranteed.

[0093] Based on any of the above embodiments, the embodiment of the present application provides a cache control method, comprising:

[0094] First, each kernel to be executed is allocated a domain identifier. Here, the allocation of domain identifiers can be implemented based on a round-robin allocation strategy. Specifically, the occupancy state of each domain identifier can be polled to allocate a domain identifier to each kernel, and the application of the round-robin allocation strategy enables the multiple kernels to be evenly distributed in the domains corresponding to different domain identifiers, thereby avoiding the situation where some domains are overloaded while other domains are idle, and achieving balanced utilization of resources.

[0095] Second, the multiple kernels are controlled to execute in parallel. And in the execution process of each kernel, the cache is accessed based on a memory access request containing the domain identifier of the kernel, thereby causing the cache to store the domain identifier in the tag of the cache line where the target data requested by the memory access request is located, so that the cache line is associated with the kernel through the domain identifier in the tag, thereby providing a basis for subsequent selective tag reset operation.

[0096] Then, when any kernel ends execution, an execution end command containing the domain identifier of the kernel that ends execution can be sent to the cache, thereby causing the cache to compare the domain identifier in each cache line tag with the domain identifier in the execution end command, and reset the tag for the cache line whose domain identifier in the tag is the same as the domain identifier in the execution end command, thereby implementing local cache invalidation operation within the cache line range corresponding to the kernel that ends execution.

[0097] Based on any of the above embodiments, Figure 2 is a structural schematic diagram of an artificial intelligence chip provided by the present application. As shown in the figure, Figure 2 The artificial intelligence chip 20 can be used to implement the above cache control method. In the artificial intelligence chip 20, a command processor 210, a computing unit 220 and a cache 230 are included, and the command processor 210 includes a domain identity allocation unit 211;

[0098] The domain identity allocation unit 211 is configured to determine the domain identity of the kernel;

[0099] The command processor 210 is configured to allocate the kernel and the domain identity of the kernel to the computing unit 220, and trigger the computing unit 220 to execute the kernel;

[0100] The computing unit 220 is configured to, during execution of the kernel, access the cache 230 based on a memory access request, and store the domain identity in a tag of a corresponding cache line in the cache 230 for the memory access request;

[0101] The command processor 210 is further configured to, after the kernel is executed, control the cache 230 to reset the tag of the cache line containing the domain identity in the tag.

[0102] Specifically, the command processor (CP) 210 is connected with the computing unit 220 and the cache 230 respectively, and the computing unit 220 is connected with the cache 230.

[0103] Different from the related art, in the embodiment of the present application, the domain identity allocation unit 211 is additionally arranged in the command processor 210.

[0104] The domain identity allocation unit 211 is configured to allocate a domain identity for a kernel to be executed. Specifically, in the kernel scheduling stage, the domain identity allocation unit 211 can allocate a domain identity for each kernel to be executed. That is, each kernel has a corresponding domain identity.

[0105] Here, the domain identity can be the kernel identity in memory access, and different domain identities can distinguish the data of different kernels in the cache 230, thereby realizing the distinction and management of the cache data of each kernel in the shared cache 230 space, and further realizing the kernel-level data isolation based on the domain identity.

[0106] After the domain identity allocation unit 211 allocates a domain identity for each kernel to be executed, the command processor 210 can send a kernel execution command to the computing unit 220, and the kernel execution command here can contain the domain identity of the kernel, so that the computing unit 220 executes the kernel.

[0107] The computing unit 220 can execute the kernel after receiving the kernel execution command. In addition, the computing unit 220 can also parse the domain identifier of the executed kernel from the kernel execution command.

[0108] For each kernel, the computing unit 220 can generate a memory access request to access the cache 230 during the execution of the kernel. Different from the related art, during the process of accessing the cache 230 through the memory access request, the cache line in the cache 230 where the data requested by the memory access request is stored can store the domain identifier of the kernel in the tag of the cache line. Specifically, a field of the domain identifier of the kernel can be added to the tag structure of the cache line, so that the domain identifier of the kernel is stored in the tag of the cache line.

[0109] Therefore, in the cache 230, the tag of the cache line no longer only contains address, valid bit, cache priority, data read-only, data consistency and the like, but also contains the domain identifier. It can be understood that storing the domain identifier of the kernel in the tag of the cache line, i.e. establishing the association between the kernel and the cache line through the domain identifier, the cache line whose tag contains the domain identifier can also be understood as the cache line belonging to the kernel corresponding to the domain identifier, and the data in the cache line is the data used for the execution of the kernel corresponding to the domain identifier.

[0110] In this way, each kernel can store its domain identifier in the tag of the cache line corresponding to the requested data through memory access during the execution process, and each kernel can divide the cache line belonging to itself in the cache 230 by adding the domain identifier in the tag, so that the kernel-level data isolation is realized in the cache 230 through the domain identifier in the tag of the cache line.

[0111] In addition, after the execution of the kernel ends, the command processor 210 can also generate an execution end command, and the execution end command contains the domain identifier of the kernel whose execution ends. The command processor 210 can send the execution end command to the cache 230, and the cache 230 can find the cache line whose tag contains the domain identifier of the kernel whose execution ends after receiving the execution end command, and reset the tag of the cache line. Here, resetting the tag of the cache line includes resetting the cache priority, data read-only, consistency and the like in the tag, deleting the domain identifier in the tag, and the like, so that the control strategy resetting and the consistency conflict resolution for the cache line belonging to the kernel whose execution ends can be realized through the tag resetting.

[0112] Thus, after the end of each kernel execution, only the cache lines belonging to the kernel in the cache 230 are reset for the tag, replacing the global invalidation strategy in the related art with a local invalidation by resetting the tags only for the cache lines belonging to the kernel whose execution ends, while the data belonging to other kernels being executed remain unchanged in the cache 230. In this way of selectively resetting the tags, the resetting of the tags is limited to the cache lines belonging to the kernel whose execution ends, thereby minimizing unnecessary cache invalidation operations, reducing the impact on the memory access of the kernel being executed, and effectively improving the utilization of the cache.

[0113] In the artificial intelligence chip provided in the embodiments, the domain identifier of the kernel is stored in the tag of the corresponding cache line in the cache in the process of kernel execution, thereby realizing the association between the kernel and the cache line, so that the cache realizes kernel-granularity data isolation based on the domain identifier in the tag of the cache line, and data security is ensured. After the end of kernel execution, the cache is controlled to reset the tags of the cache lines containing the domain identifier, and the resetting of the tags is limited to the cache lines belonging to the kernel whose execution ends, thereby minimizing unnecessary cache invalidation operations, reducing the impact on the memory access of the kernel being executed, effectively improving the utilization of the cache, and further ensuring the parallel processing performance of the artificial intelligence chip.

[0114] Based on any of the above embodiments, the present application provides an artificial intelligence chip for implementing the cache control method in the above embodiments. The artificial intelligence chip can be connected with a host, so as to realize the cache control method based on the artificial intelligence chip under the control of the host. The host can be a CPU (Central Processing Unit, CPU), and the method comprises:

[0115] First, a determination process of the domain identifier is performed:

[0116] 1. The artificial intelligence chip receives a kernel dispatch request sent by the host, and sends the kernel dispatch request to the domain identifier allocation unit in the command processor.

[0117] 2. The domain identifier allocation unit selects an available domain identifier from a plurality of candidate domain identifiers using a polling algorithm. For example, the plurality of candidate domain identifiers are 0, 1, 2, and 3. In this case, the condition for selecting the domain identifier is that the domain identifier is within the range of 0-3, and the algorithm can select a polling strategy, thereby ensuring the balance of allocation.

[0118] 3. After the available domain identifier is selected, the domain identifier allocation unit can allocate the domain identifier to the kernel dispatched by the kernel dispatch request.

[0119] 4. After the domain identifier allocation is completed, the domain identifier allocation unit updates the occupation state record of each candidate domain identifier, and updates the occupation state of the allocated domain identifier to occupied.

[0120] 5. The domain identifier allocation unit establishes a mapping relationship between the scheduled kernel and the domain identifier, and maintains this mapping relationship during the execution of the kernel.

[0121] 6. The command processor sends a kernel execution command containing the domain identifier of the kernel to the computing unit.

[0122] 7. The domain identifier determination process ends.

[0123] Secondly, execute the memory access process:

[0124] 1. The computing unit receives the kernel execution command and executes the kernel. During the execution of the kernel, a memory access request is generated, which contains the domain identifier of the kernel.

[0125] 2. The cache checks whether the target data requested by the memory access request is in the cache.

[0126] 3. Cache Miss processing: for the case that the target data is not in the cache, load the target data from the lower-level storage to the cache, and store the domain identifier of the kernel contained in the memory access request in the tag of the cache line where the target data is located, thereby establishing an association between the cache data and the kernel.

[0127] 4. Cache Hit processing: for the case that the target data is in the cache, check whether the domain identifier in the cache line tag where the target data is located matches the domain identifier of the kernel contained in the memory access request. When the two domain identifiers match, directly perform the read operation or write operation of the memory access on the target data; in the case that the two domain identifiers do not match, replace the domain identifier in the cache line tag where the target data is located with the domain identifier of the kernel contained in the memory access request, and perform the read operation or write operation of the memory access on the target data.

[0128] 5. Update the cache access state information of the cache, which can include LRU (Least Recently Used) and other states.

[0129] 6. The memory access process ends.

[0130] Finally, execute the cache invalidation process:

[0131] 1. After the end of the execution of the kernel, the command processor generates an execution end command. Here, the execution end command is generated on the premise that the kernel is executed normally, and the domain identifier of the kernel that is executed is included in the execution end command.

[0132] 2. The command processor sends the execution end command to the cache.

[0133] 3. The cache traverses all cache lines, and checks whether the domain identifier in the tag of each cache line matches the domain identifier in the execution end command.

[0134] 4. The cache performs an invalidation operation on the cache line with the matching domain identifier, and keeps the cache line with the non-matching domain identifier unchanged. Here, the invalidation operation is specifically a tag reset operation on the cache line.

[0135] 5. After the selective and partial invalidation operation on the kernel that is executed is completed, the cache invalidation process at the end of the kernel is ended.

[0136] In the artificial intelligence chip and the cache control method based on the artificial intelligence chip provided in the embodiments of the present application, the tag reset is limited to the cache line belonging to the kernel that is executed, unnecessary cache invalidation operations are minimized, the cache data of the kernel that is executed is kept, and the utilization rate of the cache can be significantly improved. Moreover, since the influence of the global invalidation on the memory access of other kernels that are executed is avoided, the kernel execution efficiency is significantly improved, and the overall performance is improved. Furthermore, by allocating the domain identifier to the kernel, the cache data of different kernels is effectively isolated in the shared cache space, the data isolation between the kernels is enhanced, and the data security and system stability are improved.

[0137] The cache control device provided in the present application is described below. The cache control device described below can be referred to in correspondence with the cache control method described above.

[0138] Figure 3 is a structural schematic diagram of the cache control device provided in the present application, as shown in Figure 3 The cache control device comprises:

[0139] an identifier determination module 310, configured to determine the domain identifier of the kernel;

[0140] a memory access module 320, configured to execute the kernel, and access the cache based on a memory access request in the execution process of the kernel, and store the domain identifier in the tag of the corresponding cache line in the cache.

[0141] The tag reset module 330 is configured to control the cache to perform tag reset on the cache line containing the domain identifier in the tag after the execution of the kernel ends.

[0142] In the device provided in the embodiments of the present application, the domain identifier of the kernel is stored in the tag of the corresponding cache line in the cache during the execution of the kernel, thereby realizing the association between the kernel and the cache line, and enabling the cache to realize kernel-granularity data isolation based on the domain identifier in the tag of the cache line, thereby ensuring data security. After the execution of the kernel ends, the cache is controlled to perform tag reset on the cache line containing the domain identifier in the tag, and the tag reset is limited to the cache line belonging to the kernel whose execution ends, thereby minimizing unnecessary cache invalidation operations, reducing the impact of memory access on the kernel being executed, effectively improving the utilization of the cache, and further ensuring the parallel processing performance of the artificial intelligence chip.

[0143] Based on any of the above embodiments, the memory access module is specifically configured to:

[0144] generate a memory access request containing the domain identifier of the kernel, and search for target data requested by the memory access request in the cache;

[0145] in a case where the target data does not exist in the cache, load the target data from a lower-level storage to the cache, and store the domain identifier in the tag of the cache line where the target data is located.

[0146] Based on any of the above embodiments, the memory access module is further configured to:

[0147] in a case where the target data exists in the cache and the domain identifier in the tag of the cache line where the target data is located is different from the domain identifier in the memory access request, replace the domain identifier in the tag of the cache line where the target data is located with the domain identifier in the memory access request.

[0148] Based on any of the above embodiments, the identifier determination module is specifically configured to:

[0149] poll the occupation states of a plurality of candidate domain identifiers, determine the domain identifier of the kernel from the plurality of candidate domain identifiers based on the polling result, and update the occupation state of the domain identifier of the kernel.

[0150] Based on any of the above embodiments, the tag reset module is specifically configured to:

[0151] generate an execution end command of the kernel, the execution end command containing the domain identifier of the kernel;

[0152] Based on the execution end command, the cache is controlled to traverse the labels of all cache lines, and the label is reset if the field identifier in the label is the same as the field identifier in the execution end command.

[0153] Based on any of the above embodiments, the memory access module is further configured to:

[0154] During the execution of the kernel, the execution mode contained in the memory access request is stored in the tag of the cache line corresponding to the memory access request in the cache.

[0155] The tag reset module is also used for:

[0156] After the kernel execution is completed, control the cache clearing tag to contain the domain identifier and the execution mode is temporary data for the cache line.

[0157] Based on any of the above embodiments, a domain identifier corresponds to one or more kernels.

[0158] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a cache control method, which includes:

[0159] Determine the kernel's domain identifier;

[0160] The kernel is executed, and during the execution of the kernel, the cache is accessed based on the memory access request, and the domain identifier is stored in the label of the cache line corresponding to the memory access request in the cache.

[0161] After the kernel execution is complete, the cache is controlled to reset the labels of cache lines whose labels contain the domain identifier.

[0162] Further, the logic instructions in the memory 430 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the related art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0163] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the cache control method provided by the above-mentioned methods, and the method comprises:

[0164] determining a domain identifier of a kernel;

[0165] executing the kernel, and in the execution process of the kernel, accessing a cache based on a memory access request and storing the domain identifier into a tag of a cache line corresponding to the memory access request in the cache;

[0166] after the execution of the kernel ends, controlling the cache to perform tag reset on the cache line containing the domain identifier in the tag.

[0167] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the cache control method provided by the above-mentioned methods, and the method comprises:

[0168] determining a domain identifier of a kernel;

[0169] executing the kernel, and in the execution process of the kernel, accessing a cache based on a memory access request and storing the domain identifier into a tag of a cache line corresponding to the memory access request in the cache;

[0170] after the execution of the kernel ends, controlling the cache to perform tag reset on the cache line containing the domain identifier in the tag.

[0171] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0172] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0173] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cache control method characterized by, The method comprises: determining a domain identifier of a kernel; executing the kernel, wherein during the execution of the kernel, a cache is accessed based on a memory access request, and the domain identifier is stored in a tag of a corresponding cache line of the memory access request in the cache; after the execution of the kernel ends, controlling the cache to reset the tag of the cache line containing the domain identifier in the tag; wherein the cache is accessed based on the memory access request, and the domain identifier is stored in the tag of the corresponding cache line of the memory access request in the cache, comprising: generating a memory access request containing the domain identifier of the kernel, and searching for target data requested by the memory access request in the cache; in a case where the target data exists in the cache and the domain identifier in the tag of the cache line where the target data is located is different from the domain identifier in the memory access request, replacing the domain identifier in the tag of the cache line where the target data is located with the domain identifier in the memory access request.

2. The cache control method according to claim 1, characterized by, The cache is accessed based on the memory access request, and the domain identifier is stored in the tag of the corresponding cache line of the memory access request in the cache, and further comprising: in a case where the target data does not exist in the cache, loading the target data from a lower-level storage to the cache, and storing the domain identifier in the tag of the cache line where the target data is located.

3. The cache control method of claim 1, wherein, The domain identifier of the kernel is determined, comprising: polling occupation states of a plurality of candidate domain identifiers, determining the domain identifier of the kernel from the plurality of candidate domain identifiers based on a polling result, and updating the occupation state of the domain identifier of the kernel.

4. The cache control method of claim 1, wherein, The cache is controlled to reset the tag of the cache line containing the domain identifier in the tag, comprising: generating an execution end command of the kernel, wherein the execution end command contains the domain identifier of the kernel; based on the execution end command, controlling the cache to traverse the tags of all cache lines, and performing tag reset in a case where the domain identifier in the tag is the same as the domain identifier in the execution end command.

5. The cache control method according to any one of claims 1 to 4, characterized by, Further comprising: during the execution of the kernel, storing an execution mode contained in the memory access request in the tag of the corresponding cache line of the memory access request in the cache; after the execution of the kernel ends, controlling the cache to clear the cache line containing the domain identifier in the tag and having the execution mode as temporary data.

6. The cache control method according to any one of claims 1 to 4, characterized by, One domain identifier corresponds to one or more kernels.

7. An artificial intelligence chip, characterized by, A command processor, a computing unit and a cache are included, and the command processor comprises a domain identifier allocation unit; The domain identifier allocation unit is used to determine a domain identifier of a kernel; The command processor is used to allocate the kernel and the domain identifier of the kernel to the computing unit, and trigger the computing unit to execute the kernel; The computing unit is used to access the cache based on a memory access request during the execution of the kernel, and store the domain identifier in the tag of the corresponding cache line of the memory access request in the cache; The command processor is further configured to control the cache to reset the tag of the cache line containing the domain identifier in the tag after the end of the core execution. The method comprises: generating a memory access request containing the domain identifier of the core, and searching for target data requested by the memory access request in the cache; in a case where the target data exists in the cache and the domain identifier in the tag of the cache line where the target data is located is different from the domain identifier in the memory access request, replacing the domain identifier in the tag of the cache line where the target data is located with the domain identifier in the memory access request.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executed on the processor, wherein, The processor implements the cache control method according to any one of claims 1 to 6 when executing the computer program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the cache control method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the cache control method according to any one of claims 1 to 6.

Images