A cache-based asynchronous direct write method and an artificial intelligence chip

Abstract

This application provides a cache-based asynchronous direct write method and an artificial intelligence chip, relating to the field of artificial intelligence chip technology. The method includes: detecting the status bits of multiple cache groups in the cache; when the status bit of a cache group indicates that there is a cache line in the cache group that meets a preset condition, detecting the link status between the cache and main memory, the preset condition being: containing refresh prompt information and being marked as dirty; when the link status is idle and no data request is received from the processing core, evicting the cache line in the cache group that meets the preset condition to main memory. By promptly evicting cache lines containing refresh prompt information and marked as dirty to main memory when the link between the cache and main memory is idle, the interconnect bus bandwidth between the cache and main memory is kept in a dynamically balanced state, fully utilizing the interconnect bus bandwidth between the cache and main memory while avoiding excessive pressure on the interconnect bus bandwidth.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence chip technology, and in particular to a cache-based asynchronous direct write method and an artificial intelligence chip. Background Technology

[0002] In storage architecture, cache write mode determines how the processing core handles updates to main memory when writing data to the cache.

[0003] Under related technologies, artificial intelligence chips adopt a write-back cache writing mode. In write-back mode, the processing core writes data to the cache. Only when the data in the cache exceeds the cache limit or the software actively flushes the cache will the data be written to main memory.

[0004] When using AI chips to implement AI model computation, the workload of the AI ​​model is uneven over time. Thus, when the model's workload is low (i.e., the workload is less than the cache limit), the bandwidth of the interconnect bus between the cache and main memory is idle, resulting in wasted bandwidth resources. When the model's workload exceeds the cache limit, a large amount of data needs to be evicted into main memory in a short period of time, which puts a lot of pressure on the bandwidth of the interconnect bus between the cache and main memory. Summary of the Invention

[0005] This application provides an asynchronous direct write method based on caching and an artificial intelligence chip, which is used to keep the interconnect bus bandwidth between the cache and main memory in a dynamically balanced state when the workload of the model changes dynamically. This fully utilizes the interconnect bus bandwidth between the cache and main memory while avoiding excessive pressure on the interconnect bus bandwidth between the cache and main memory.

[0006] On the one hand, embodiments of this application provide a cache-based asynchronous write-through method, which includes:

[0007] Detect the status bits of each of the multiple cache groups in the cache;

[0008] For each cache group, when the status bit of the cache group indicates the first state, the link status between the cache and the main memory is detected. The first state is: there is a cache line in the cache group that meets a preset condition; the preset condition is: it contains refresh prompt information and is marked as dirty.

[0009] When the link is in an idle state and no data request is received from the processing core, cache lines in the cache group that meet the preset conditions are evicted to main memory.

[0010] Optionally, after evicting cache lines that meet the preset conditions from the cache group to main memory, the method further includes:

[0011] Update the cache line from the dirty state to a non-dirty state;

[0012] Update the status bit of the cache group so that the status bit of the cache group indicates a second state; the second state is that there is no cache line in the cache group that meets the preset condition.

[0013] Optionally, detecting the link status between the cache and main memory includes:

[0014] Read the count result of the incomplete counter, which is used to record the number of incomplete requests between the cache and the main memory;

[0015] Based on the count result of the incomplete counter, the link status between the cache and the main memory is determined.

[0016] Optionally, determining the link status between the cache and the main memory based on the count result of the incomplete counter includes:

[0017] When the count result of the incomplete counter is less than a preset threshold, the link status between the cache and the main memory is determined to be idle.

[0018] Optionally, the incomplete requests between the cache and the main memory are triggered by the processing core.

[0019] Optionally, it also includes:

[0020] Receive a data request sent by the processing core, the data request carrying refresh prompt information;

[0021] When the data request hits or is assigned a cache line, the refresh prompt information is added to the tag information of the cache line;

[0022] When a cache line is marked as dirty, update the status bit of the cache group to which the cache line belongs, so that the status bit of the cache group indicates the first state.

[0023] Optionally, the data request further includes: a memory address; the step of adding the refresh prompt information to the tagging information of the cache line when the data request hits or is allocated a cache line includes:

[0024] Read the tag information for each cache line from the tag memory;

[0025] A hit check is performed on the memory address based on the tag information of each cache line to obtain the check result;

[0026] When the check result is that the memory address matches the tag information of a cache line, the refresh prompt information is added to the tag information of the cache line;

[0027] When the check result is that there is no matching tag information for the memory address, a cache line is allocated for the data request, and the refresh prompt information is added to the tag information of the cache line.

[0028] Optionally, the data request is a read request; the method further includes:

[0029] The data of a cache line is read from the cache memory and returned to the processing core.

[0030] Optionally, the data request is a write request; the method further includes:

[0031] The target data carried by the write request is written into the storage area corresponding to the cache line in the cache memory.

[0032] On one hand, embodiments of this application provide an artificial intelligence chip, including:

[0033] Processes the core, cache, and main memory;

[0034] The cache is used to execute the cache-based asynchronous write-through method described above.

[0035] On one hand, embodiments of this application provide a computer device, including a memory, an artificial intelligence chip, and a computer program stored on the memory and executable on the artificial intelligence chip. When the artificial intelligence chip executes the program, it implements the steps of the above-described cache-based asynchronous direct write method.

[0036] In this embodiment, when a cache line containing refresh prompt information and marked as dirty is detected in the cache group, and the link between the cache and main memory is idle, and no data request is currently received from the processing core, the cache spontaneously initiates an eviction process to evict the cache line containing refresh prompt information and marked as dirty to the main memory. This fully utilizes the interconnect bus bandwidth between the cache and main memory and reduces bandwidth resource waste.

[0037] Secondly, when the link between the cache and main memory is idle, cache lines that contain refresh prompts and are marked as dirty should be promptly evicted to main memory. This keeps the number of cache lines in a dirty state in the cache to a low level. As the workload of the model gradually increases, or even exceeds the cache limit, there is no need to evict a large amount of data to main memory in a short period of time, thus effectively alleviating the pressure on the interconnect bus bandwidth between the cache and main memory.

[0038] Overall, this application can maintain a dynamic balance in the interconnect bus bandwidth between the cache and main memory even when the workload of the model changes dynamically. This fully utilizes the interconnect bus bandwidth between the cache and main memory while avoiding excessive pressure on the interconnect bus bandwidth between the cache and main memory. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A schematic diagram of the structure of an artificial intelligence chip provided in this application embodiment. Figure 1 ;

[0041] Figure 2 A schematic diagram of the structure of an artificial intelligence chip provided in this application embodiment. Figure 2 ;

[0042] Figure 3 This is a schematic diagram of the structure of an asynchronous direct write engine provided in an embodiment of this application;

[0043] Figure 4 A flowchart illustrating a cache-based asynchronous direct write method provided in an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0046] refer to Figure 1 The diagram shows the structure of an artificial intelligence chip applicable to the embodiments of this application. The artificial intelligence chip 100 includes at least a processing core 101, a cache 102, and main memory 103. The processing core 101 has the ability to perform artificial intelligence-related calculations, such as matrix multiplication and convolution operations, which are common calculation operations in artificial intelligence models.

[0047] Cache 102 is a small, fast storage area located between processing core 101 and main memory 103. It is used to temporarily store data or instructions that processing core 101 or other hardware devices may frequently access in the near future, with the aim of speeding up access and reducing access latency. Main memory 103 is the next level below cache 102. Compared to cache 102, main memory 103 generally has a larger capacity but is slower.

[0048] In this embodiment, cache 102 detects the status bits of multiple cache groups. For each cache group, when the status bit of the cache group indicates a first state, the link status between cache 102 and main memory 103 is detected. The first state is: there is a cache line in the cache group that meets a preset condition. The preset condition is: it contains refresh prompt information and is marked as dirty. When the link status is idle and no data request is received from processing core 101, the cache line in the cache group that meets the preset condition is evicted to main memory 103.

[0049] In some embodiments, see Figure 2 The cache 102 includes: a selector 201, a tag ram 202, a hit-miss check unit 203, a cache ram 204, and an asynchronous write-through engine 205.

[0050] Specifically, the tag memory 202 is used to store tag information for each cache line in multiple cache sets, where a cache set is a logical group consisting of multiple cache lines. A cache line is the smallest unit of data storage in cache 102 and also the smallest unit of operation in cache 102.

[0051] A cache line includes data and tagging information. The tagging information includes a tag, a valid bit, and a dirty bit. The tag is used to identify the address range of the data in main memory. The valid bit indicates whether the data is valid. The dirty bit indicates whether the data has been modified, i.e., whether it is consistent with main memory.

[0052] The cache memory 204 stores the data in each cache line. The tag information stored in the tag memory 202 has a one-to-one correspondence with the data stored in the cache memory 204.

[0053] In this embodiment, the cache 102 receives a data request sent by the processing core 101, and the data request carries refresh prompt information; the selector 201 preferentially selects to process the data request; thus, the hit checking unit 203 reads the tag information of each cache line from the tag memory 202; and then performs a hit check on the data request based on the read tag information.

[0054] When a data request hits or is assigned a cache line, a refresh message is added to the tag information of that cache line, and the updated tag information is updated in the tag memory 202.

[0055] When a cache line is marked as dirty, the asynchronous write engine 205 is triggered to update the status bit of the cache group to which the cache line belongs, so that the status bit of the cache group indicates the first state, that is, there is a cache line in the cache group that meets the preset conditions.

[0056] When a data request hits or is allocated a cache line, if the data request is a read request, the data of the cache line is read from cache memory 204, which may have been previously read from main memory 103; then, a request response is returned to processing core 101, which carries the read data.

[0057] If the data request is a write request, the target data carried by the write request will be written to the storage area corresponding to the cache line in the cache memory 204, and a request response will be returned to the processing core 101, which indicates that the write is complete.

[0058] In some embodiments, see Figure 3 The asynchronous direct write engine 205 includes: a cache group status detection unit 301, an occupancy status detection unit 302, and a decision unit 303.

[0059] The cache group status detection unit 301 is used to detect the status bits of each cache group; the occupancy status detection unit 302 is used to detect the link status between cache 102 and main memory 103, and also to detect the request status between processing core 101 and cache 102.

[0060] When the link between cache 102 and main memory 103 is occupied by a write request caused by processing core 101, cache 102 and main memory 103 are in an occupied state; when the link between cache 102 and main memory 103 is not occupied by a write request caused by processing core 101, cache 102 and main memory 103 are in an idle state.

[0061] The request status between processing core 101 and cache 102 includes: currently receiving a data request from processing core 101 and currently not receiving a data request from processing core 101.

[0062] If the cache group status detection unit 301 determines that the status bit of a cache bit indicates the first state; and the occupancy status detection unit 302 determines that the link status is idle and no data request has been received from the processing core 101, then the decision unit 303 initiates an asynchronous write-through request for the cache group.

[0063] At this time, selector 201 selects to process the asynchronous write-through request, so that hit checking unit 203 reads the tag information of each cache line in the corresponding cache group from tag memory 202 according to the cache group index carried in the asynchronous write-through request.

[0064] For each cache line in the cache group, if the tag information contains refresh prompt information and the dirty bit in the tag information is set to dirty state, the data of the cache line stored in the cache memory 204 is synchronized to the main memory 103; correspondingly, in the tag memory 202, the dirty bit in the tag information is set to non-dirty state.

[0065] When there are no cache lines marked as dirty in the cache group, the cache group status detection unit 301 updates the status bit of the cache group so that the status bit of the cache group indicates the second status; the second status is that there are no cache lines in the cache group that meet the preset conditions.

[0066] In this embodiment, when the asynchronous write-through engine 205 detects that there is a cache line in the cache group that contains refresh prompt information and is marked as dirty, and when the channel between cache 102 and main memory 103 is idle and no data request from core 101 is being processed, it spontaneously initiates an eviction process to evict the cache line containing refresh prompt information and marked as dirty to main memory 103. This fully utilizes the interconnect bus bandwidth between cache 102 and main memory 103 and reduces bandwidth resource waste.

[0067] Secondly, when the channel between cache 102 and main memory 103 is idle, cache lines that contain refresh prompts and are marked as dirty are promptly evicted to main memory 103. This keeps the number of dirty cache lines in cache 102 within a small range. As the workload of the model gradually increases or even exceeds the cache limit, it is not necessary to evict a large amount of data to main memory in a short period of time, thereby effectively alleviating the pressure on the interconnect bus bandwidth between cache 102 and main memory 103.

[0068] Overall, this application can still keep the interconnect bus bandwidth between cache 102 and main memory 103 in a dynamic balance when the workload of the model changes dynamically. This fully utilizes the interconnect bus bandwidth between cache 102 and main memory 103 while avoiding excessive pressure on the interconnect bus bandwidth between cache 102 and main memory 103.

[0069] In addition to the structure described above, the artificial intelligence chip 100 in this application may also include other structures, and this application does not specifically limit such structures.

[0070] Artificial intelligence chips 100 can be: Graphics Processing Unit (GPU), General-Purpose Graphics Processing Units (GPGPU), Domain Specific Architecture (DSA), etc.

[0071] The following is based on Figure 1 The diagram shown illustrates the architecture of an AI chip and details the process of a cache-based asynchronous write-through method. (See also...) Figure 4 This method is executed by a cache in an artificial intelligence chip and includes the following steps:

[0072] Step 401: Detect the status bits of each of the multiple cache groups in the cache.

[0073] Specifically, each cache group corresponds to a status bit, and the status bit is modified by updating the trigger (flop) where the status bit is located.

[0074] Step 402: For each cache group, when the status bit of the cache group indicates the first state, detect the link status between the cache and the main memory.

[0075] Specifically, the status bits of a cache group include a first state and a second state. The first state indicates that there are cache lines in the cache group that meet preset conditions; the second state indicates that there are no cache lines in the cache group that meet preset conditions. The status bits of a cache group can correspond to different values, and the meaning of the status bits is different when the values ​​are different. For example, when the status bit of the cache group is 1, it indicates the first state, that is, there are cache lines in the cache group that meet preset conditions; when the status bit of the cache group is 0, it indicates the second state, that is, there are no cache lines in the cache group that meet preset conditions, where the preset conditions are: containing refresh prompt information and being marked as dirty.

[0076] In some embodiments, a data request sent by the processing core is received, the data request carrying refresh prompt information; when the data request hits or is allocated a cache line, the refresh prompt information is added to the tag information of the cache line; when the cache line is marked as dirty, the status bit of the cache group to which the cache line belongs is updated so that the status bit of the cache group indicates the first state.

[0077] Specifically, a refresh message indicates that data in the cache line can be automatically flushed to main memory by the hardware (i.e., the cache). Generally, if the data requested in a data request will not be written again in a short period of time, this data can be automatically flushed to main memory by the hardware. In this case, the data request carries a refresh message.

[0078] If the data requested in the data request may be subject to multiple near-memory calculations, frequent calculations, or writes, then this data does not need to be automatically flushed to main memory by the hardware. In this case, the data request will not carry a refresh prompt message.

[0079] In some embodiments, the data request further includes: a memory address; reading tag information for each cache line from a tag storage; and then performing a hit check on the memory address based on the tag information for each cache line to obtain the check result.

[0080] Specifically, the memory address includes a tag and a cache set index. During a hit check, the corresponding cache set is first located based on the cache set index in the memory address; then, the tag in the memory address is compared one by one with the tag information of each cache line under the cache set to obtain the check result.

[0081] When the check result is that the memory address matches the tag information of a cache line, that is, there exists a cache line whose tag information contains the tag in the memory address and the cache line is valid, then a refresh prompt message will be added to the tag information of the cache line.

[0082] When the check result is: no memory address hit tag information, that is, no cache line tag information contains the tag at that memory address, then a cache line is allocated for the data request, and refresh prompt information is added to the tag information of the cache line.

[0083] Additionally, if a cache line that is hit or allocated is marked as dirty (i.e. inconsistent with main memory), the status bit of the cache group containing that cache line is updated so that the status bit of the cache group indicates the first state.

[0084] It should be noted that if the cache line is marked as dirty, the status bit of the cache group to which the cache line belongs already indicates the first state (that is, in the cache group, in addition to the cache line, there are other cache lines that have met the preset conditions). At this time, there is no need to update the status of the cache group.

[0085] In this embodiment, after a hit check is performed on a data request, when the data request hits or is allocated a cache line, the tag information of the cache line is updated accordingly. At the same time, the existence of a cache line that can be evicted to main memory is recorded at the granularity of the cache group, instead of recording whether each cache line can be evicted to main memory. This reduces the number of status bits, thereby reducing the consumption of trigger resources.

[0086] In some embodiments, the data request is a read request; when the data request hits or is allocated a cache line, the data of that cache line is read from the cache memory and returned to the processing core.

[0087] Specifically, when a data request hits or is allocated a cache line, the data of that cache line is read from the cache memory based on the tag information of that cache line, and the read data is returned to the processing core.

[0088] In some embodiments, the data request is a write request; when the data request hits or is allocated a cache line, the target data carried by the write request is written to the storage area corresponding to the cache line in the cache memory.

[0089] Specifically, when a data request hits or is allocated a cache line, the storage area corresponding to the cache line in the cache memory is determined based on the tag information of the cache line; then the target data is written to the storage area, and the cache line is marked as dirty.

[0090] In this embodiment, when the cache receives a data request from the processing core, it uses a write-back mode to ensure low latency; at the same time, it uses an asynchronous write-through mode to evict cache lines in the first state to main memory, thereby maximizing the utilization of the interconnect bus bandwidth between the cache and main memory.

[0091] In some embodiments, when the cache asynchronously writes cache lines that meet preset conditions (i.e., contain refresh prompt information and are marked as dirty), it should not affect the processing of data requests initiated by the processing core. Therefore, when there are cache lines in the cache group that can be evicted to main memory, it is necessary to further determine the link status between the cache and main memory, as well as the request status between the processing core and the cache.

[0092] The embodiments of this application employ at least the following methods to determine the link state between the cache and main memory:

[0093] Read the result of the outstanding counter, which records the number of outstanding requests between the cache and main memory; then determine the link status between the cache and main memory based on the result of the outstanding counter.

[0094] Specifically, incomplete requests between the cache and main memory include: incomplete write requests between the cache and main memory, that is, requests from the cache to write data to the main memory.

[0095] When the count result of the incomplete counter is less than the preset threshold, the link status between the cache and main memory is determined to be idle; when the count result of the incomplete counter is not less than the preset threshold, the link status between the cache and main memory is determined to be occupied. The preset threshold can be configured and modified by software.

[0096] In some embodiments, incomplete requests between the cache and main memory recorded by the incomplete counter are triggered by the processing core. That is, when the cache receives a data request from the processing core, the data request may cause a write request to evict cache lines to main memory, i.e., write the data in the cache line to main memory using the link between the cache and main memory.

[0097] For example, when the cache receives a write request (carrying target data) sent by the processing core, the write request hits a cache line. If the cache line has been marked as dirty, the link between the cache and main memory is used to write the data of the cache line to main memory (i.e., the write request between the cache and main memory triggered by the processing core), and then the target data is written to the cache line.

[0098] Write requests between the cache and main memory caused by the processing core, as well as asynchronous write requests to the cache (asynchronously writing cache lines that meet preset conditions to main memory), all require the use of the link between the cache and main memory, which can lead to link conflicts.

[0099] Therefore, in this embodiment, asynchronous write requests have a lower priority than write requests between the cache and main memory caused by the processing core. When the count of the incomplete counter is not less than a preset threshold, it indicates that the link between the cache and main memory needs to be prioritized for write requests caused by the processing core; therefore, the link between the cache and main memory is in an occupied state. When the count of the incomplete counter is less than the preset threshold, it indicates that the link between the cache and main memory does not need to be allocated to write requests caused by the processing core; therefore, the link between the cache and main memory is in an idle state.

[0100] It should be noted that during the execution of asynchronous write-through requests in the cache, the link between the cache and main memory is occupied by the asynchronous write-through requests. However, the incomplete counter does not count the number of incomplete asynchronous write-through requests, but only counts the write requests between the cache and main memory caused by the processing core.

[0101] Step 403: When the link status is idle and no data request is received from the processing core, cache lines that meet the preset conditions in the cache group are evicted to main memory.

[0102] Specifically, by Figure 2It is known that when a data request is received from the processing core, the selector 201, tag memory 202, hit check unit 203, and cache memory 204 are used to process the data request. After initiating an asynchronous write request, the asynchronous write engine 205 reuses the selector 201, tag memory 202, hit check unit 203, and cache memory 204 to process the asynchronous write request. In other words, the data request from the processing core and the asynchronous write request will reuse the same structural units for processing, which will lead to structural unit conflicts.

[0103] Therefore, in this embodiment, asynchronous write-through requests have a lower priority than data requests to the processing core. Before initiating an asynchronous write-through request, the request status between the processing core and the cache is checked; if a data request to the processing core is received, an asynchronous write-through request cannot be initiated.

[0104] When no data request is received from the processing core, and it is determined, as described above, that there is a cache line in the cache group that meets the preset conditions, and the link between the cache and main memory is in an idle state, an asynchronous write-through request is initiated, carrying the cache group index in the asynchronous write-through request.

[0105] The corresponding cache group is located based on the cache group index, and the tagging information of each cache line in the cache group is read. For the tagging information of each cache line read, if the tagging information contains a refresh hint and the dirty bit in the tagging information is set to dirty, then the data of the cache line is written to main memory.

[0106] In this way, by limiting the conditions for initiating asynchronous write requests in multiple dimensions, cache lines that contain refresh prompts and are marked as dirty can be evicted to main memory without affecting the data requests (i.e., normal access) of the processing core, thereby maximizing the utilization of the interconnect bus bandwidth between the cache and main memory.

[0107] In some embodiments, after evicting cache lines that meet preset conditions from the cache group to main memory, the cache lines remain consistent between the cache and main memory. Therefore, the cache lines are updated from a dirty state to a non-dirty state.

[0108] Additionally, update the status bit of the cache group so that the status bit of the cache group indicates the second state, that is, there are no cache lines in the cache group that meet the preset conditions.

[0109] It should be noted that if a cache group contains a cache line that meets a preset condition, after that cache line is evicted to main memory, the status bit of the cache group is updated so that the status bit of the cache group indicates the second state.

[0110] If the cache group contains multiple cache lines that meet the preset conditions, then all of these cache lines will be evicted to main memory before the status bit of the cache group is updated to indicate the second status.

[0111] In this embodiment, when a cache line containing refresh prompt information and marked as dirty is detected in the cache group, and the link between the cache and main memory is idle, and no data request is currently received from the processing core, the cache spontaneously initiates an eviction process to evict the cache line containing refresh prompt information and marked as dirty to the main memory. This fully utilizes the interconnect bus bandwidth between the cache and main memory and reduces bandwidth resource waste.

[0112] Secondly, when the link between the cache and main memory is idle, cache lines that contain refresh prompts and are marked as dirty should be promptly evicted to main memory. This keeps the number of cache lines in a dirty state in the cache to a low level. As the workload of the model gradually increases, or even exceeds the cache limit, there is no need to evict a large amount of data to main memory in a short period of time, thus effectively alleviating the pressure on the interconnect bus bandwidth between the cache and main memory.

[0113] When the workload of the model changes dynamically, the bandwidth of the interconnect bus between the cache and main memory is kept in a dynamic balance, so as to make full use of the bandwidth of the interconnect bus between the cache and main memory, while avoiding excessive pressure on the bandwidth of the interconnect bus between the cache and main memory.

[0114] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 5 As shown, it includes at least one artificial intelligence chip 100 and a memory 501 connected to at least one artificial intelligence chip 100. In this embodiment, the specific connection medium between the artificial intelligence chip 100 and the memory 501 is not limited. Figure 5 Taking the connection between the AI ​​chip 100 and the memory 501 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0115] In this embodiment of the application, the memory 501 stores instructions that can be executed by at least one artificial intelligence chip 100. By executing the instructions stored in the memory 501, at least one artificial intelligence chip 100 can perform the steps of the above-mentioned cache-based asynchronous direct write method.

[0116] As one embodiment, the AI ​​chip 100 detects the status bits of multiple cache groups in the cache. When the status bit of a cache group indicates that there is a cache line in the cache group that meets a preset condition, it detects the link status between the cache and main memory. The preset condition is that the cache line contains a refresh prompt and is marked as dirty. When the link status is idle and no data request is received from the processing core, the cache line in the cache group that meets the preset condition is evicted to main memory. When the link between the cache and main memory is idle, the cache line containing a refresh prompt and marked as dirty is promptly evicted to main memory.

[0117] The artificial intelligence chip 100 serves as the control center of the computer device, connecting to various parts of the device via various interfaces and lines. It enables asynchronous direct writing based on a cache by running or executing instructions stored in the memory 501 and retrieving data stored in the memory 501. Optionally, the artificial intelligence chip 100 may include one or more processing units. It may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 501 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0118] The artificial intelligence chip 100 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0119] Memory 501, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 501 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 501 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. Memory 501 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0120] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device. When the computer program is run on the computer device, it causes the computer device to perform the steps of the above-described cache-based asynchronous direct write method.

[0121] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer device or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer device or other programmable data processing equipment to cause a series of operational steps to be performed on the computer device or other programmable equipment to produce a process implemented by the computer device, thereby providing instructions that execute on the computer device or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0126] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A cache-based asynchronous direct write method, characterized in that, include: Detect the status bits of each of the multiple cache groups in the cache; For each cache group, when the status bit of the cache group indicates the first state, the link status between the cache and the main memory is detected. The first state is: there is a cache line in the cache group that meets a preset condition. The preset condition is: it contains refresh prompt information and is marked as dirty. The refresh prompt information indicates that the data in the cache line is automatically refreshed to the main memory by the cache. The refresh prompt information is carried in the data request sent by the processing core to the cache. When the link is in an idle state and no data request is received from the processing core, cache lines in the cache group that meet the preset conditions are evicted to main memory.

2. The method as described in claim 1, characterized in that, After evicting cache lines that meet the preset conditions from the cache group to main memory, the process further includes: Update the cache line from the dirty state to a non-dirty state; Update the status bit of the cache group so that the status bit of the cache group indicates a second state; the second state is: there is no cache line in the cache group that meets the preset condition.

3. The method as described in claim 1, characterized in that, The detection of the link status between the cache and main memory includes: Read the count result of the incomplete counter, which is used to record the number of incomplete requests between the cache and the main memory; Based on the count result of the incomplete counter, the link status between the cache and the main memory is determined.

4. The method as described in claim 3, characterized in that, Determining the link status between the cache and the main memory based on the count result of the incomplete counter includes: When the count result of the incomplete counter is less than a preset threshold, the link status between the cache and the main memory is determined to be idle.

5. The method as described in claim 3, characterized in that, The incomplete requests between the cache and the main memory are triggered by the processing core.

6. The method as described in claim 1, characterized in that, Also includes: Receive a data request sent by the processing core, the data request carrying refresh prompt information; When the data request hits or is assigned a cache line, the refresh prompt information is added to the tag information of the cache line; When a cache line is marked as dirty, update the status bit of the cache group to which the cache line belongs, so that the status bit of the cache group indicates the first state.

7. The method as described in claim 6, characterized in that, The data request further includes: a memory address; when the data request hits or is allocated a cache line, adding the refresh prompt information to the tag information of the cache line includes: Read the tag information for each cache line from the tag memory; A hit check is performed on the memory address based on the tag information of each cache line to obtain the check result; When the check result is that the memory address matches the tag information of a cache line, the refresh prompt information is added to the tag information of the cache line; When the check result is that there is no matching tag information for the memory address, a cache line is allocated for the data request, and the refresh prompt information is added to the tag information of the cache line.

8. The method as described in claim 7, characterized in that, The data request is a read request; the method further includes: The data of a cache line is read from the cache memory and returned to the processing core.

9. The method as described in claim 7, characterized in that, The data request is a write request; the method further includes: The target data carried by the write request is written into the storage area corresponding to the cache line in the cache memory.

10. An artificial intelligence chip, characterized in that, include: Processes the core, cache, and main memory; The cache is used to perform the method according to any one of claims 1 to 9.

Images